Rational Survivability

Why Security Awareness Campaigns Matter

The topic of security awareness training has floated up to the surface on a number of related topics lately and I'm compelled to comment on what can only be described as a diametrically opposed set of opinions on the matter.

Here's a perfect illustration taken from some comments on this blog entry where I suggested that many CIO's simply think that "awareness initiatives are good for sexual harassment and copier training, not security."

Firstly, here is someone who thinks that awareness training is a waste of time:

As to educating users, it's one of the dumbest ideas in security. As Marcus Ranum has famously pointed out, if it was going to work...it would have worked by now. If you are relying on user education as part of your strategy, you are doomed. See "The Six Dumbest Ideas in Security" for a fine explanation of this.

...and here is the counterpoint offered by another reader suggesting a different perspective:

Completely disagree. Of course you're not going to get through to everyone, but if you get through to maybe 80-90% then that's an awful lot of attacks you've prevented, with actually very little effort. The reason I think it hasn't worked yet is because people are not doing it effectively, or that they'll 'get around to it' once the CEO has signed off all the important projects, the ones that mean the IT Security team get to play with cool new toys.

What's my take?

I think this is very much a case of setting the appropriate expectations for what the deliverable and results should be from the awareness training.  I think security awareness and education can bear substantial fruit.  Further, like the second reader, if the goals are appropriately and realistically set, suggesting that 100% of the trainees will yield 100% compliance is simply nonsense.

Again, we see that too often the "success" of a security initiative is only evaluated on a binary scale of 0 or 100% which is simply stupid. We all know and accept that we'll never been 100% secure, so why would we suggest that 100% of our employees will remember and act on 100% of their awareness training?

What if I showed (and I have) that the number of tailgates through access controlled access points went down over 30% since awareness training? What if I showed that the number of phishing attempt reports to IT Security increased 62% and click-throughs decreased by the same amount since awareness training?  What if I showed that the number of reports of lost/stolen company property decreased by 18% since awareness training?  How about when all our developers were sent to SDLC training and our software deficiencies per line of code went down double digits?

What if I told you that I spent very little amounts of money and time implementing this training and did it both interactively and through group meetings and everyone was accountable and felt more empowered because we linked the topics to the things that matter to THEM as well as the company?

As to Marcus' arguments regarding the efficacy of education/awareness, he's basically suggesting that the reason awareness doesn't work is (1) human stupidity and (2) a failure of properly implementing technology that should ultimately prevent #1 from even being an issue.   

I suggest that as #2 becomes less of an issue as people get smarter about how they deploy technology (which is also an "awareness" problem) and the technology gets better, then implementing training and education for issue #1 becomes the element that will help reduce the residual gap.

To simply dismiss security awareness training as a waste of time is short-sighted and I've yet to find anyone who relies solely upon awareness training as their only strategy for securing their assets. It's one of many tools that can effectively be used to manage risk.

What's your take?

VMware's VMsafe: The Good, the Bad, the Bubbly...

Back in August before VMworld 2007, I wrote about the notion that given Cisco's investment in VMware, we'd soon see the ability for third parties to substitute their own virtual switches for VMware's. 

Further, I discussed the news that VMware began to circulate regarding their release of an API originally called Vsafe that promised to allow third party security and networking applications to interact with functions exposed by the Hypervisor. 

Both of these points really put some interesting wrinkles -- both positive and potentially negative -- in how virtualization (and not just VMware's) will continue to evolve as the security and networking functions evolve along with it.

What a difference a consonant makes...

Several months later, they've added the consonant 'm' to the branding and VMsafe is born.  Sort of.  However, it's become more than 'just' an API.  Let's take a look at what I mean.

What exactly is VMsafe? Well, it's a marketing and partnership platform, a business development lever, an architecture, a technology that provides an API and state of mind:

VMsafe is a new program that leverages the properties of VMware Infrastructure to protect machines in ways previously not possible with physical machines. VMsafe provides a new security architecture for virtualized environments and an application program interface (API)-sharing program to enable partners to develop security products for virtualized environments.

What it also provides is a way to make many existing older and consolidating security technologies relevant in the virtualized world given that today their value is suspect in the long term without it:

The VMsafe Security Architecture provides an open approach that gives security vendors the ability to leverage the inherent properties of virtualization in their security offerings.

Customers running their businesses on VMware Infrastructure will be assured that they are getting the best protection available – even better than what they have on physical infrastructure.

VMsafe adds a new layer of defense that complements existing physical security solutions such as network and host protection, shielding virtual machines from a variety of network, host and applications threats. This additional layer of protection can help enterprise organizations to dramatically increase the security posture of their IT environments.

There is then hope that a good deal of the visibility we lost in the limited exposure existing security solutions have across virtualized environments, we may get back.

Of course, this good news will be limited to those running VMware's solutions and re-tooled versions of your favorite security vendor's software.  What that means for other virtualization platforms is a little more suspect.  Since it requires the third party software to be re-written/adapted to use the VMsafe API, I can't see many jumping on the wagon to support 3-4 VMM platforms.

Ack!  This is why we need a virtualization standard like OVF and a cross-platform signaling, telemetry and control plane definition!

So how does VMsafe work?

VMsafe enables third-party security products to gain the same visibility as the hypervisor into the operation of a virtual machine to identify and eliminate malware, such as viruses, trojans and key-loggers. For instance, security vendors can leverage VMsafe to detect and eliminate malware that is undetectable on physical machines. This advanced protection is achieved through fine-grained visibility into the virtual hardware resources of memory, CPU, disk and I/O systems of the virtual machine that can be used to monitor every aspect of the execution of the system and stop malware before it can execute on a machine to steal data.

...

VSAFE enables partners to build a virtualization-aware security solution in the form of a security virtual machine that can access, correlate and modify information based on the following virtual hardware:

• Memory and CPU: VMsafe provides introspection of guest VM memory pages and cpu states.
• Networking: Network packet-filtering for both in-hypervisor and within a Security VM.
• Process execution (guest handling): in-guest, in-process APIs that enable complete monitoring and control of process execution.
• Storage: Virtual machine disk files (VMDK) can be mounted, manipulated and modified as they persist on storage devices.

So where do we go from here?

With nothing more than a press release and some flashy demos, it's a little early to opine on the extensibility of VMsafe, but I am encouraged by the fact that we will have some more tools in the arsenal, even if they are, in essence, re-branded versions of many that we already have.

However, engineering better isolation combined with brokered visibility and specific authorized/controlled access to the VMM is both a worthy endeavor that yields all sorts of opportunities, but given my original ramblings, makes me a bit nervous.

Alessandro from the virtualization.info blog gave us a description of the demo given at VMworld Europe that illustrates some of this new isolation and anti-malware capability:

A Windows XP virtual machine gets attacked with a malicious code that copies away corporate documents but another virtual machine with security engine is able to transparently recognize (a virtual memory scan through VMsafe APis access) the threat and stop it before it compromises the guest OS.

Steps in the right direction, for sure.  Since details regarding the full extent of anti-malware capabilities are still forthcoming, I will reserve judgment until I get to speak with Nand @ VMware to fully understand the scope of the capabilities.

I am sure we will see more claims surface soon suggesting with technology such as this will produce virtualized environments that are "more secure" than their non-virtualized counterparts.  The proof is in the pudding, as they say.  At this point, what we have is a very tantalizing recipe.

I'd suggest we'll also see a fresh batch of rootkit discussions popping up soon...I'd really like to see the documentation surrounding the API.  I wonder how much it costs to sign up and be authorized to participate with VMsafe?

Some of the Determina functionality is for sure making its way into VMsafe and it will be really interesting to see who will be first first out of the gate to commercially offer solutions that will utilize VMsafe  once it's available -- and it's a little unclear when that will be.

Given who demoed at VMworld Europe, I'd say it's safe to bet that McAfee will be one of the first along with some of the more agile startups that might find it easier to include in their products.  The initial lineup of vendor support makes up some of the who's-who in the security space -- all except for, um, Cisco.  Also, where the heck is SourceFire?

When do the NAC and DLP vendors start knocking?

More detailed analysis soon.

/Hoff

News Flash: If You Don't Follow Suggested Security Hardening Guidelines, Bad Things Can Happen...

The virtualization security (VirtSec) FUD meter is in overdrive this week...

Part I:
     So, I was at a conference a couple of weeks ago.  I sat in on a lot of talks.
     Some of them had demos.  What amazed me about these demos is that in
     many cases, in order for the attacks to work, it was disclosed that the
     attack target was configured by a monkey with all defaults enabled and no
     security controls in place.  "...of course, if you checked this one box, the
     exploit doesn't work..." *gulp*

Part II:
     We've observed a lot of interesting PoC attack demonstrations such as those at shows being picked
     up by the press and covered in blogs and such.  Many of these stories simply ham it up for the
     sensational title.  Some of the artistic license and innacuracies are just plain recockulous. 
     That's right.  There's ridiculous, then there's recockulous. 

Example:
     Here's a by-line from an article which details the PoC attack/code that Jon Oberheide used to show
     how, if you don't follow VMware's (and the CIS benchmark) recommendations for securing your
     VMotion network, you might be susceptible to interception of traffic and bad things since -- as
     VMware clearly states -- VMotion traffic (and machine state) is sent in the clear.

     This was demonstrated at BlackHat DC and here's how the article portrayed it:

Jon Oberheide, a researcher and PhD candidate at the University of Michigan, is releasing a proof-of-concept tool called Xensploit that lets an attacker take over the VM’s hypervisor and applications, and grab sensitive data from the live VMs.

     Really?  Take over the hypervisor, eh?  Hmmmm.  That sounds super-serious!  Oh, the humanity!

However, here's how the VMTN blog rationally describes the situation in a measured response that does it better than I could:

Recently a researcher published a proof-of-concept called Xensploit which allows an attacker to view or manipulate a VM undergoing live migration (i.e. VMware’s VMotion) from one server to another. This was shown to work with both VMware’s and Xen’s version of live migration. Although impressive, this work by no means represents any new security risk in the datacenter. It should be emphasized this proof-of-concept does NOT “take over the hypervisor” nor present unencrypted traffic as a vulnerability needing patching, as some news reports incorrectly assert. Rather, it a reminder of how an already-compromised network, if left unchecked, could be used to stage additional severe attacks in any environment, virtual or physical. ...

Encryption of all data-in-transit is certainly one well-understood mitigation for man-in-the-middle attacks.  But the fact that plenty of data flows unencrypted within the enterprise – indeed perhaps the majority of data – suggests that there are other adequate mitigations. Unencrypted VMotion traffic is not a flaw, but allowing VMotion to occur on a compromised network can be. So this is a good time to re-emphasize hardening best practices for VMware Infrastructure and what benefit they serve in this scenario.

I'm going to give you one guess as to why this traffic is unencrypted...see if you can guess right in the comments.

Now, I will concede that this sort of thing represents a new risk in the datacenter if you happen to not pay attention to what you're doing, but I think Jon's PoC is a great example of substantiating why you should follow both common sense, security hardening recommendations and NOT BELIEVE EVERYTHING YOU READ.

If you don't stand for something, you'll fall for anything.

/Hoff

McGovern's "Ten Mistakes That CIOs Consistently Make That Weaken Enterprise Security"

James McGovern over at the Enterprise Architect blog wrote a really fantastic Letterman's Top 10 of mistakes that CIO's make regarding enterprise security.  I've listed his in its entirety below and added a couple mineself... ;)

Use process as a substitute for competence: The answer to every problem is almost always methodology, so you must focus savagely on CMMi and ITIL while not understanding the fact that hackers attack software.

Ostritch Principle: Since you were so busy aligning with the business which really means that you are neither a real IT professional nor business professional, you have spent much of your time perfecting memorization of cliche phrases and nomenclature and hoping that the problem will go away if you ignore it.

Putting network engineers in charge of security: When will you learn that folks with a network background can't possibly make your enterprise secure. If a hacker attacks software and steals data yet you respond with hardware, whom do you really think is going to win the battle.

Over Rely on your vendors by relabelling them as partners: You trust your software vendors and outsourcing firms so much that you won't even perform due diligence on their staff to understand whether they have actually received one iota of training

Rely primarily on a firewall and antivirus: Here is a revelation. Firewalls are not security devices, they are more for network hygiene. Ever consider that a firewall can't possibly stop attacks related to cross site scripting, SQL injection and so on. Network devices only protect the network and can't do much nowadays to protect applications.

Stepping in your own leadership: Authorize reactive, short-term fixes so problems re-emerge rapidly

Thinking that security is expensive while also thinking that CMMi isn't: Why do you continue to fail to realize how much money their information and organizational reputations are worth.

The only thing you need is an insulting firm to provide you with a strategy: Fail to deal with the operational aspects of security: make a few fixes and then not allow the follow through necessary to ensure the problems stay fixed

Getting it twisted to realize that Business / IT alignment is best accomplished by talking about Security and not SOA: Failing to understand the relationship of information security to the business problem -- they understand physical security but do not see the consequences of poor information security. Let's be honest, your SOA is all about integration as you aren't smart enough to do anything else.

Put people in roles and give them titles, but don't actually train them: Assign untrained people to maintain security and provide neither the training nor the time to make it possible to do the job.

Here are some of my favorites that I've added.  I'll work on adding the expanded explanations later:

1. Keep talking about threats and vulnerabilities and not about risk
2. Manage your security investments like throw-away CapEx cornflakes and not as a portfolio
3. Maintain that security is a technology issue
4. Awareness initiatives are good for sexual harassment and copier training, not security
5. Security is top secret, we can't talk about what we do
6. All we need to do is invest just enough to be compliant, we don't need to be secure
7. We can't measure security effectiveness
8. Virtualization changes nothing in the security space.
9. We've built our three year security strategy and we're aligned to the business
10. One audit a year from a trusted third party indicates our commitment to security

Got any more?

/Hoff

(A)vailability > (C)onfidentiality + (I)ntegrity...Part Deux: Film/Video NOT At 11...

We had a little chat a few weeks ago at the apparent shock suffered by many a security professional in discovering that the three-legged stool of security was constructed of unequally leveraged legs of C, I and A.

Some reckon that by all practical accounts C, I and A should not be evaluated or assessed in a vacuum, but depending upon your line of business, your line of work and how you view the world, often this is how things get done -- we have very siloed organizations, so it leads to siloed decision matrices.

Specifically, availability (or service delivery) in reality -- despite what theory and purists espouse -- often trumps "security" (the C and I functions.)  As distasteful as that sounds, this is endemic.  From operating systems focused on "usability" rather than security to routing protocols focused on rapid convergence and assumed trust as opposed to secure and authenticated mechanisms.

To wit (from the Renesys Blog):

Pakistan hijacks YouTube

Late in the (UTC) day on 24 February 2008, Pakistan Telecom (AS 17557) began advertising a small part of YouTube's (AS 36561) assigned network. This story is almost as old as BGP. Old hands will recognize this as, fundamentally, the same problem as the infamous AS 7007 from 1997, a more recent ConEd mistake of early 2006 and even TTNet's Christmas Eve gift 2004.

Just before 18:48 UTC, Pakistan Telecom, in response to government order to block access to YouTube (see news item) started advertising a route for 208.65.153.0/24 to its provider, PCCW (AS 3491). For those unfamiliar with BGP, this is a more specific route than the ones used by YouTube (208.65.152.0/22), and therefore most routers would choose to send traffic to Pakistan Telecom for this slice of YouTube's network.                                                          

Yes, this is really a demonstration of unavailability, but what I'm getting at here is that fundamentally, the core routing protocol we depend upon for the backbone Internet transport is roughly governed by the same rules that we depend upon whilst driving down a road separated by nothing more than painted lines...you simply hope/trust that nobody crosses the line and crashes into you head-on.

There is very little preventing someone from re-routing traffic.  This could result in either a denial of service (as the traffic would not reach its destination) or even something akin to an interception, "storage" and eventual forwarding for nefarious means.

So, here we have a case where again we depend upon a protocol that was designed to provide (A)vailability, yet C and I are left floundering in the wings.  We'll no doubt see another round of folks who will try and evangelize the need for secure BGP -- just like secure DNS, secure SMTP, secure...

This will hit deaf ears until we see the same thing happen again...

/Hoff

VMWare Hosted Virtualization Platform Vulnerability = Guest System Break-Out via Shared Folders...

There's a little bit of serendipity floating about today and timing is everything.

Ed Skoudis (IntelGuardians) and I were chatting last week at ShmooCon regarding his previous research on VM guest escapes in hosted platforms and I raised a concern regarding my use of Parallel shared folders between my hosted XP installation and the underlying OSX host operating system.

I reckoned that this would be a very interesting vector for potential exploitation as it provides a direct pipeline to the underlying host OS and filesystem. 

While this bit of news isn't about Parallels, it is about VMware's comparable products (workstation, ACE, player, etc.) and it exploits the same vector.  From Computerworld:

February 24, 2008 (Computerworld)  A critical vulnerability in VMware Inc.'s virtualization software for Windows lets attackers escape the "guest" operating system and modify or add files to the underlying "host" OS, the company has acknowledged.

As of Sunday, there was no patch available for the flaw, which affects VMware's Windows client virtualization programs, including Workstation, Player and ACE. The company's virtual machine software for Windows servers, and for Mac- and Linux-based hosts, are not at risk.

The bug was reported by Core Security Technologies, makers of the penetration testing framework CORE IMPACT, said VMware in a security alert issued last Friday. "Exploitation of this vulnerability allows attackers to break out of an isolated Guest system to compromise the underlying Host system that controls it," claimed Core Security.

According to VMware, the bug is in the shared folder feature of its Windows client-based virtualization software. Shared folders lets users access certain files -- typically documents and other application-generated files -- from the host OS and any virtual machine on that physical system.

"On Windows hosts, if you have configured a VMware host-to-guest shared folder, it is possible for a program running in the guest to gain access to the host's complete file system and create or modify executable files in sensitive locations," confirmed VMware.

There is currently no patch available.  The mitigation strategy is to disable shared folders.

It's important to reiterate that this vulnerability does not affect VMware's Type 1 (bare metal) virtualization platforms such as ESX.  However, on Friday, VMware released fixes for 5 vulnerabilities in ESX, some of which could be exploited to bypass security controls, gain access to data or result in denial of service.

/Hoff

{image from Anthony Martin Escapes}

UPDATE: Coverage of this is being hammed up quite a bit in the press to sound like it's going to shake the very foundations of virtualization...not so much.  It's an issue that is reasonably easy to address and represents what can be generally referred to as a relatively small attack surface.  Yes, it reinforces the need to think about VirtSec in the Type 2 (hosted) virtualization world, but as I said in the comments, it really depends upon how and why you've deployed client-side virtualization.

Travel: UK (London) From 2/25-2/27

Right, so I'm in the UK from the 25th to the 27th. I'll be tagging my usual suspects, but if you're up for something in the evening, send me an email [choff @ packetfilter.com] or call my call router and it will find me:
+1.978.631.0302

/Hoff

Pondering Implications On Standards & Products Due To Cold Boot Attacks On Encryption Keys

You've no doubt seen the latest handywork of Ed Felten and his team from the Princeton Center for Information Technology Policy regarding cold boot attacks on encryption keys:

Abstract: Contrary to popular assumption, DRAMs used in most modern computers retain their contents for seconds to minutes after power is lost, even at operating temperatures and even if removed from a motherboard. Although DRAMs become less reliable when they are not refreshed, they are not immediately erased, and their contents persist sufficiently for malicious (or forensic) acquisition of usable full-system memory images. We show that this phenomenon limits the ability of an operating system to protect cryptographic key material from an attacker with physical access. We use cold reboots to mount attacks on popular disk encryption systems — BitLocker, FileVault, dm-crypt, and TrueCrypt — using no special devices or materials. We experimentally characterize the extent and predictability of memory remanence and report that remanence times can be increased dramatically with simple techniques. We offer new algorithms for finding cryptographic keys in memory images and for correcting errors caused by bit decay. Though we discuss several strategies for partially mitigating these risks, we know of no simple remedy that would eliminate them.

Check out the video below (if you have scripting disabled, here's the link.)  Fascinating and scary stuff.

Would a TPM implementation mitigate this if they keys weren't stored (even temporarily) in RAM?

Given the surge lately toward full disk encryption products, I wonder how the market will react to this.  I am interested in both the broad industry impact and response from vendors.  I won't be surprised if we see new products crop up in a matter of days advertising magical defenses against such attacks as well as vendors scrambling to do damage control.

This might be a bit of a reach, but equally as interesting to me are the potential implications upon DoD/Military crypto standards such as FIPS140.2 ( I believe the draft of 140.3 is circulating...)  In the case of certain products at specific security levels, it's obvious based on the video that one wouldn't necessarily need physical access to a crypto module (or RAM) in order to potentially attack it.

It's always amazing to me when really smart people think of really creative, innovative and (in some cases) obvious ways of examining what we all take for granted.

It Appears I'm Giving Two Keynotes @ RSA 2008, But They Spelled My Name Wrong...

I was browsing through the RSA 2008 conference agenda today and noticed that two of my talks and topics I blog about constantly were being featured as RSA keynotes!

How cool is that!?

It seems besides the talk I'm already giving, the fine folks @ RSA forgot to tell me that I was to deliver these, also.

They also accidentally attributed the speaking roles to someone else:

KEY-101 The Role of Security in Business Innovation: From Villain to Hero Keynote Art Coviello, EMC/RSA

- and -

KEY-102 Information Centric Security: The Next Wave John Thompson, Symantec Corporation

I'll be busy sorting out this correction.

In the meantime, you can just preview them here:

Security and Disruptive Innovation

Information Centricity

;)

/Hoff

Clarification from Catbird's CTO on HypervisorShield...

Last week I posted about a press release announcing a new product from Catbird called HypervisorShield.

I was having difficulty understanding some of the points raised in the press release/product brief, so I reached out to Michael Berman, Catbird's CTO (also a blogger,) for a little clarification. 

Michael was kind enough to respond  to the points in my blog posting.

Rather than repost the entire blog entry, I have paraphrased the points Michael responded to and left his comments intact.  I think some of them invite further clarification and I'll be following up with a Take5 interview shortly.  Some of the answers just beg for a little more digging...

Just to ground us all, here's the skinny on HypervisorShield:

Catbird, provider of the only comprehensive security solution for virtual and physical networks, and developer of the V-Agent™ virtual appliance, today announced the launch of HypervisorShield™, the industry’s first dedicated comprehensive security solution specifically designed to guard against unauthorized hypervisor network access and attack.

Here are my points and Michael's responses:

1. Hoff: The press release speaks to HypervisorShield's ability to protect both the hypervisor and the "hypervisor management network" which I assume is actually referring to the the virtual interface of the management functions like VMware's service console? Are we talking about protecting the service console or the network functions provided by the vKernel?
   
   Berman: We've built a monitor function that uses VMware APIs to watch for changes to/management of the virtual machines. We also have signature templates and customizable policies for network connections to the service console and the host.
    
   
2. Hoff: The press release makes it sound like protecting the hypervisor is accomplished via an IPS function that isolates VM's from one another like Reflex and Blue Lane?
   
   Berman: With all due respect to our colleagues in this space, intrusion detection and protection is one element.  Catbird combines several technologies to extend separation of duties, dual control and strict change control to the virtual infrastructure. Deploying a signature for VMSA-2008-0001 is nice, but detection or prevention of a rogue virtual center administrator from pulling off a Societe Generale hack is priceless.
    
3. Hoff: What exactly does Catbird do (in partnering with IPS companies like SourceFire) that folks like Reflex and BlueLane? don't already do.
   
   Berman: Rather than talk about the differences, let’s talk about the most important similarity.  I think I speak for all of us when I say that it’s like we are in a time warp to 1996 and I am explaining why you need a firewall for your DMZ.  Customers have little appreciation for the magnitude of the threats facing their virtual infrastructure.  Once we get past that, then we can talk about why Catbird is the best. (hint: we're smarter, faster and stronger)
    
4. Hoff: How do you monitor the Hypervisor?
   
   Berman: We deploy a virtual machine that hooks into the vSwitch environment and that also monitors the ESX hypervisor via the VI API.
    
5. Hoff: You say in the press release that "hypervisor exploits have grown 35% in the last several years."  Which hypervisor exploits, exactly? You mean exploits against the big, fat, Linux-based service console from VMware? That's not the hypervisor!
   
   Berman: I believe that the real threat to the virtual infrastructure comes from the collapse of separation of duties and the breakdown in implicit and explicit security controls within the virtual data center.  That being said, the hypervisor management application is probably the most significant area of the attack surface. If I can own the management GUI I own the hypervisor. If I can pull a stack smash against the ESX web server I own the hypervisor. If some poor shlemozzle configures Samba and NFS for the storage network then they become part of the attack surface too. You can blame us for some hyperbole, but the stat came from the CVE database. Gartner/451/Edison report that virtual infrastructure (VI) is less secure than physical and we have private data that shows people are deploying VI with no network security at all – this is just wrong.  I also think that writing about, or writing off the only risk as being some sort of red pill/blue pill hack is also wrong.

Thanks again to Michael for responding.  Look for a follow-on Take5 shortly to dig a little deeper.

/Hoff

BeanSec! Wednesday, February 20th, 2008 - 6PM to ?

Yo!  BeanSec! is once again upon us.  Wednesday, February 20th, 2008.

BeanSec! is an informal meetup of information security professionals, researchers and academics in the Greater Boston area that meets the third Wednesday of each month. 

I say again, BeanSec! is hosted the third Wednesday of every month.  Add it to your calendar.

Come get your grub on.  Lots of good people show up.  Really.

Unlike other meetings, you will not be expected to pay dues, “join up”, present a zero-day exploit, or defend your dissertation to attend. Map to the Enormous Room in Cambridge.

Enormous Room: 567 Mass Ave, Cambridge 02139.  Look for the Elephant on the left door next to the Central Kitchen entrance.  Come upstairs. We sit on the left hand side...

Don't worry about being "late" because most people just show up when they can.  6:30 is a good time to aim for.  We'll try and save you a seat.  There is a parking garage across the street and 1 block down or you can try the streets (or take the T)

In case you're wondering, we're getting about 30-40 people on average per BeanSec!  Weld, 0Day and I have been at this for just over a year and without actually *doing* anything, it's turned out swell.

We've had some really interesting people of note attend lately (I'm not going to tell you who...you'll just have to come and find out.)  At around 9:00pm or so, the DJ shows up...as do the rather nice looking people from the Cambridge area, so if that's your scene, you can geek out first and then get your thang on.

The food selection is basically high-end finger-food appetizers and the drinks are really good; an attentive staff and eclectic clientèle make the joint fun for people watching.  I'll generally annoy you into participating somehow, even if it's just fetching napkins. ;)

See you there.

/Hoff

A Worm By Any Other Name Is...An Information Epidemic?

Martin McKeay took exception to some interesting Microsoft research that suggested that the similar methodologies and tactics used by malicious software such as worms/viri, could also be used as an effective distributed defense against them:

Microsoft researchers are hoping to use "information epidemics" to distribute software patches more efficiently.

Milan Vojnović and colleagues from Microsoft Research in Cambridge, UK, want to make useful pieces of information such as software updates behave more like computer worms: spreading between computers instead of being downloaded from central servers.

The research may also help defend against malicious types of worm, the researchers say.

Software worms spread by self-replicating. After infecting one computer they probe others to find new hosts. Most existing worms randomly probe computers when looking for new hosts to infect, but that is inefficient, says Vojnović, because they waste time exploring groups or "subnets" of computers that contain few uninfected hosts.

Despite the really cool moniker (information epidemic,) this isn't a particularly novel distribution approach and in fact, we've seen malware do this.  However, it is interesting to see that an OS vendor (Microsoft) is continuing to actively engage in research to explore this approach despite the opinions of others who simply claim it's a bad idea.  I'm not convinced either way, however.

I, for one, am all for resilient computing environments that are aware of their vulnerabilities and can actively defend against them.  I will be interested to see how this new paper builds off of work previously produced on the subject and its corresponding criticism.

Vojnović's team have designed smarter strategies that can exploit the way some subnets provide richer pickings than others.

The ideal approach uses prior knowledge of the way uninfected computers are spread across different subnets. A worm with that information can focus its attention on the most fruitful subnets – infecting a given proportion of a network using the smallest possible number of probes.

But although prior knowledge could be available in some cases – a company distributing a patch after a previous worm attack, for example – usually such perfect information will not be available. So the researchers have also developed strategies that mean the worms can learn from experience.

In the best of these, a worm starts by randomly contacting potential new hosts. After finding one, it uses a more targeted approach, contacting only other computers in the same subnet. If the worm finds plenty of uninfected hosts there, it keeps spreading in that subnet, but if not, it changes tack.

That being the case, here's some of Martin's heartburn:

But the problem is, if both beneficial and malign software show the same basic behavior patterns, how do you differentiate between the two? And what’s to stop the worm from being mutated once it’s started, since bad guys will be able to capture the worms and possibly subverting their programs.

The article isn’t clear on how the worms will secure their network, but I don’t believe this is the best way to solve the problem that’s being expressed. The problem being solved here appears to be one of network traffic spikes caused by the download of patches. We already have a widely used protocols that solve this problem, bittorrents and P2P programs. So why create a potentially hazardous situation using worms when a better solution already exists. Yes, torrents can be subverted too, but these are problems that we’re a lot closer to solving than what’s being suggested.

I don’t want something that’s viral infecting my computer, whether it’s for my benefit or not. The behavior isn’t something to be encouraged. Maybe there’s a whole lot more to the paper, which hasn’t been released yet, but I’m not comfortable with the basic idea being suggested. Worm wars are not the way to secure the network.

I think that some of the points that Martin raises are valid, but I also think that he's reacting mostly out of fear to the word 'worm.'  What if we called it "distributed autonomic shielding?" ;)

Some features/functions of our defensive portfolio are going to need to become more self-organizing, autonomic and intelligent and that goes for the distribution of intelligence and disposition, also.  If we're not going to advocate being offensive, then we should at least be offensively defensive.  This is one way of potentially doing this.

Interestingly, this dovetails into some discussions we've had recently with Andy Jaquith and Amrit Williams; the notion of herds or biotic propagation and response are really quite fascinating.  See my post titled "Thinning the Herd & Chlorinating the Gene Pool"

I've left out most of the juicy bits of the story so you should go read it and churn on some of the very interesting points raised as part of the discussion.

/Hoff

Update: Schneier thinks this is a lousy idea. That doesn't move me one direction or the other, but I think this is cementing my opinion that had the author not used the word 'worm' in his analog the idea might not be dismissed so quickly...

Also, Wismer via a comment on Martin's blog pointed to an interesting read from Vesselin Bontchev titled "Are "Good" Computer Viruses Still a Bad Idea?"

Update #2: See the comments section about how I think the use case argued by Schneier et. al. is, um, slightly missing the point.  Strangely enough, check out the Network World article that just popped up which says ""This was not the primary scenario targeted for this research," according to a statement."

Duh.

Announcing the Security Star Chamber...

I had an idea today; a platform upon which to launch a little security parody mixed with an even dose of introspective navel gazing and the odd spoonful of guffaw. The goal is to provide a healthy whilst humorous appraisal of the state of the security industry.

Think InfoSec Sellout meets Monty Python and The Apprentice.

Did you ever see the movie The Star Chamber?

In one of his earlier features,Michael Douglas plays a young judge who becomes disillusioned with the law system he used to so admire when he finds himself continually having to aquit particularly dispicable criminals on the grounds of ridiculous technicalities.

Sensing his frustration,a close friend (Hal Holbrook) informs him of a secret judicial society that meets and dishes out the appropriate punishment to those who have escaped the clutches of the law. 

Inspired by some conversations this last week at ShmooCon with friends new and old, I am creating the Rational Survivability version of the "Security Star Chamber."

I'm going to play the disillusioned (young) judge.  I've recruited my not-so-secret judicial society who will, on a weekly basis, cast judgment against a specific market of the security industry; we'll pick on a segment in a no-holds barred look at the belly of beast, not to dispense punishment, but to rather provide perspective.

If we can't take ourselves seriously, we may as well play the fool instead.

We expect to communicate our judgment in the most pompous, self-important and aggrandizing style as we possibly can.  Fair and balanced?  This ain't Fox News (if you can't sift through that irony, you're sure as hell going to hate the SSC...)

Here's the catch...each of the jury has to summarize his or her argument in one sentence.

This may lend itself to some awkward dialog, but it ought to be mildly interesting for sure.

You'll meet the other judges shortly ;)

/Hoff

 

Security Innovation & the Bendy Hammer

See that odd looking hammer to the left?  It's called the MaxiStrike from Redback Tools.

No, it hasn't been run over by a Panzer, nor was there grease on the lens  during the photography session. 

Believe it or not, that odd little bend enables this 20 ounce mallet with the following features:

     > maximize strike force

     > reduce missed hits

     > leave clearance for nailing in cramped areas

All from that one little left hand turn from linear thought in product design.

You remember that series of posts I did on Disruptive Innovation?

This is a perfect illustration of how innovation can be "evolutionary" as opposed to revolutionary.

Incrementalism can be just as impacting as one of those tipping point "big-bang" events that have desensitized us to some of the really cool things that pop up and can actually make a difference.

So I know this hammer isn't going to cure cancer, but it makes for easier, more efficient and more accurate nailing.  Sometimes that's worth a hell of a lot to someone who does a lot of hammering...

Things like this happen around us all the time -- even in our little security puddle of an industry. 

It's often quite fun when you spot them.

I bet if you tried, you can come up with some examples in security.

Well?

Virtualization Hits the Mainstream...

Sad, but true...

Catbird Says It Has a Better Virtualization Security Mousetrap - "Dedicated Hypervisor Security Solution"

I spent quite a bit of time in the Catbird booth at VMworld, initially lured by their rather daring advertising campaign of "running naked."  I came away intrigued by the Security SaaS-like business model provided by their V-Agent offering and saw that as the primary differentiator.

I was particularly interested today when I read a latest press release from Catbird that suggests that their new "HypervisorShield" is specifically designed to secure the hypervisor from network access and attack:

Catbird, provider of the only comprehensive security solution for virtual and physical networks, and developer of the V-Agent™ virtual appliance, today announced the launch of HypervisorShield™, the industry’s first dedicated comprehensive security solution specifically designed to guard against unauthorized hypervisor network access and attack.

The paragraph above seems to be talking about protecting the "hypervisor" itself from network-borne compromise which is very interesting to me for reasons that should be obvious at this point. 

However, the following paragraph seems to refer to the "hypervisor management network" which I assume is actually referring to the the virtual interface of the management functions like VMware's service console?   Are we talking about protecting the service console or the network functions provided by the vKernel? 

HypervisorShield, the latest service in Catbird’s V-Security product, extends best practice security protection to virtualization’s critical hypervisor layer, thwarting both inadvertent management error and malicious threats. Delivering continuous, automated 24x7 monitoring focused on the precise vulnerabilities, known attack signatures and guest machine access of the hypervisor management network, HypervisorShield is the only service to proactively secure this essential component of a virtualization deployment.

Here's where it gets a little more confusing because the wording seems again to suggest they are protecting the hypervisor itself -- or do they mean the virtual switch as a component of the Hypervisor?:

HypervisorShield is the first virtualized security technology which can monitor and control access to the hypervisor network, detect malicious network activity directed at the hypervisor from virtual machines and validate that the hypervisor network is configured according to best practices and site security policy.

...sounds like an IPS function that isolates VM's from one another like Reflex and Blue Lane? 

OK, but here's where it gets really interesting.  Catbird is suggesting that they are able to "...see inside the hypervisor" which implies they have hooks and exposure to elements within the hypervisor itself versus the vSwitch plumbing that everyone has access to.

Via the groundbreaking Catbird V-Agent virtual appliance, protection is delivered within the virtual network itself. By contrast, traditional security solutions retrofitted for virtual deployments cannot see inside the hypervisor. Monitoring from the inside yields significantly more effective coverage and eliminates the need to reroute traffic onto the physical network for validation. As an example of the benefits of running right on the virtual subnet, HypervisorShield’s exclusive network access control (NAC) will instantly quarantine unauthorized devices on the management network.

They do talk about NAC from the VM perspective, which is something I've been advocating.

From Catbird's website we see some more detail regarding HypervisorShield which again introduces an interesting assertion:

How do you monitor the Hypervisor?

Securing a virtual host does not only involve applying the same security controls to virtual networks as were applied to their physical counterparts. Virtualization introduces a new layer of abstraction entirely—the Hypervisor. Hypervisor exploits have grown 35% in the last several years, with more surely on their way. Catbird’s patent-pending HypervisorShield protects and defends this essential component of a virtual deployment.

Really?  Hypervisor exploits have grown 35% in the last several years?  Which hypervisor exploits, exactly?  You mean exploits against the big, fat, Linux-based service console from VMware?  That's not the hypervisor!

I'm trying to give Catbird the benefit of the doubt here, but this is confusing as heck as to what exactly Catbird does (with partnering with companies like SourceFire) that folks like Reflex and BlueLane don't already do.

If anyone, especially Catbird, has some clarification for me, I'd be mighty appreciative.

/Hoff

Google Security: Frightening Statistics On Drive-By Malware Downloads...

Read a scary report from Google's security team today titled "All your iFrame Are Point to Us" regarding the evolving trends in search-delivered drive-by malware downloads.  Check out the full post here, but the synopsis follows:

It has been over a year and a half since we started to identify web pages that infect vulnerable hosts via drive-by downloads, i.e. web pages that attempt to exploit their visitors by installing and running malware automatically. During that time we have investigated billions of URLs and found more than three million unique URLs on over 180,000 web sites automatically installing malware. During the course of our research, we have investigated not only the prevalence of drive-by downloads but also how users are being exposed to malware and how it is being distributed. Our research paper is currently under peer review, but we are making a technical report [PDF] available now.  Although our technical report contains a lot more detail, we present some high-level findings here:

The above graph shows the percentage of daily queries that contain at least one search result labeled as harmful. In the past few months, more than 1% of all search results contained at least one result that we believe to point to malicious content and the trend seems to be increasing.

Ugh.  The technical report offers some really good background data on infrastructure and methodology,  geographic distribution, properties and delivery mechanisms.  Fascinating reading.

/Hoff

Off The Cuff Review: Nemertes Research's "Virtualization Risk Analysis"

I just finished reading a research paper from Andreas Antonopoulous from Nemertes titled "A risk analysis of large-scaled and dynamic virtual server environments."  You can find the piece here: 

Executive Summary

As virtualization has gained acceptance in corporate data centers, security has gone from afterthought to serious concern. Much of the focus has been on the technologies of virtualization rather than the operational, organizational and economic context. This comprehensive risk analysis examines the areas of risk in deployments of virtualized infrastructures and provides recommendations

I was interested by two things immediately:

1. While I completely agree with the fact that in regards to virtualization and security the focus has been about the "...technologies of virtualization rather than the operational, organizational and economic context" I'm not convinced there is an overwhelming consensus that "...security has gone from afterthought to serious concern" mostly because we're just now getting to see "large-scaled and dynamic virtual server environments.'  It's still painted on, not baked in.  At least that's how people react at my talks.
    
2. Virtualization is about so much more than just servers, and in order to truly paint a picture of analyzing risk within "large-scaled and dynamic virtual server environments" much of the complexity and issues associated specifically with security stem from the operational and organizational elements associated with virtualizing storage, networking, applications, policies, data and the wholesale shift in operationalizing security and who owns it within these environments.

I've excerpted the most relevant element of the issue Nemertes wanted to discuss:

With all the hype surrounding server virtualization come the inevitable security concerns: are virtual servers less secure? Are we introducing higher risk into the data center? For server virtualization to deliver benefits we have to examine the security risks. As with any new technology there is much uncertainty mixed in with promise. Part of the uncertainty arises because most companies do not have a good understanding of the real risks surrounding virtualization.

I'm easily confused...

While I feel the paper does a good job of describing the various stages of deployment and many of the "concerns" associated with server virtualization within these contexts, I'm left unsatisfied that I'm anymore prepared to assess and manage risk regarding server virtualization.  I'm concerned that the term "risk" is being spread about rather liberally because there is the presence of a bit of math.

The formulaic "Virtualization Risk Assessment" section is suggested to establish a quantatative basis for computing "relative risk," in the assessment summary.  However, since the variables introduced in the formulae are subjective and specific per asset, it's odd that the summary table is then seemingly presented generically so as to describe all assets:

Scenario	Vulnerability	Impact	Probability of Attack	Overall Risk
Single virtual server (hypervisor risk)	Low	High	Low	Low/Medium
Basic services virtualized	Low	High	Medium	Medium
Production applications virtualized	Medium	High	High	Medium/High
Complete virtualization	High	High	High	High

I'm trying to follow this and then get smacked about by this statement, which explains why people just continue to meander along applying the same security strategies toward virtualized servers as they do in conventional environments:

This conclusion might appear to be pessimistic at first glance. However, note that we are comparing various stages of deployment of virtual servers. A large deployment of physical servers will suffer from many of the same challenges that the “Complete Virtualization” environment suffers from.

Furthermore, it's unclear to me how to factor in compensating controls into this rendering given what follows:

What is new here is that there are fewer solutions for providing virtual security than there are for providing physical security with firewalls and intrusion prevention appliances in the network. On the other hand, the cost of implementing virtualized security can be significantly lower than the cost of dedicated hardware appliances, just like the cost of managing a virtual server is lower than a physical server.

The security solutions available today are limited by how much integration exists with the virtualization platforms today.  We've yet to see the VMM's/Hypervisors opened up to allow true low-level integration and topology-sensitive security interaction with flow classification, provisioning, and disposition.

Almost all supposed "virtualization-ready" security solutions today are nothing more than virtual appliance versions of existing solutions or simply the same host-based solutions which run in the VM and manage not to cock it up.  Folding your management piece into something like VMware's VirtualCenter doesn't count.

In general, I simply disagree that the costs of implementing virtualized security (today) can be significantly lower than the cost of dedicated hardware appliances -- not if you're expecting the same levels of security you get in the conventional, non-virtualized world.

The reasons (as I give in my VirtSec presentations):  Loss of visibility, constraint of the virtual networking configurations, coverage, load on the hosts, licensing.  All really important.

Cutting to the Chase

I'm left waiting for the punchline, much like I was with Burton's "Immutable Laws of Virtualization," and I think the reason why is that despite these formulae, the somewhat shallow definition of risk seems to still come down to nothing more than reasonably-informed speculation or subjective perception:

So, in the above risk analysis, one must also consider that the benefits in virtualization far outweigh the risks. The question is not so much whether companies should proceed with virtualization – the market is already answering that resoundingly in the affirmative. The question is how to do that while minimizing the risk inherent in such a strategy.

These few sentences above seem to almost obviate the need for risk analysis at all and suggests that for most, security is still an afterthought.  High risk or not, the show must go on?

So given the fact that virtualization is happening at breakneck pace, we have few good security solutions available, we speak of risk "relatively," and that operationally the entire role and duty of "security" within virtualized environments is now shifting, how do we end up with this next statement?

In the long run, virtualized security solutions will not only help mitigate the risk of broadly deployed infrastructure virtualization, but will also provide new and innovative approaches to information security that is in itself virtual. The dynamic, flexible and portable nature of virtual servers is already leading to a new generation of dynamic, flexible and portable security solutions.

I like the awareness Andreas tries to bring in this paper, but I fear that I am not left with any new information or tools for assessing risk (let alone quantifying it) in a virtual environment. 

So what do I do?!  I still have no answer to the main points of this paper, "With all the hype surrounding server virtualization come the inevitable security concerns: are virtual servers less secure? Are we introducing higher risk into the data center?"

Well?  Are they?  Am I?

/Hoff

Do The Shmoo: Who's Going to ShmooCon?

I'll be at in D.C. at ShmooCon the latter part of this week. 

I'm arriving in D.C. the afternoon of the 14th and leaving on Saturday the 17th.

If you're going to be there, ping me [choff @ packetfilter.com] or call my voice router @ +1.978.631.0302.  I'm looking forward to a number of talks.

See you there.

/Hoff

On the Chatham House Rule

James Gardner reminded me of something that I wanted to bring up but had forgotten about for some time.  Yes, he's Australian, but he can't help that.

You'd understand why that was funny if you knew that I grew up in New Zealand.  Or perhaps not.

Let me first begin by suggesting that we owe many things to the empire of Great Britain. 

There's the Queen, crumpets, French jokes, that wonderful derivative affectation that causes all the women to swoon, the incessant need for either a cuppa tea or litres of beer, and some interesting cultural and business customs.

One of those customs is that of the Chatham House Rule. 

If you've ever been to the UK and attended a business meeting discussing sensitive subject matter, there's a good chance that someone pronounced that all those participating are cloaked under the Chatham House Rule.

If, as a gracious guest, you were not (at least by modern standards) subject to Her Majesty's sovereign rule, you may have simply smiled and nodded politely not knowing who, what, or where this oddly-named domicile was and what it may have had to do with your meeting.

The same could be said for that guy Robert and all his suggestions, I suppose.

At any rate, for all of you who have wondered just what in Tony Blair's closet you just agreed to when you attended one of these meeting governed by this odd architectural framework defined in the spirit of Chatham, you may now wonder no longer.

The Chatham House Rule reads as follows:

"When a meeting, or part thereof, is held under the Chatham House Rule, participants are free to use the information received, but neither the identity nor the affiliation of the speaker(s), nor that of any other participant, may be revealed".

The world-famous Chatham House Rule may be invoked at meetings to encourage openness and the sharing of information.

EXPLANATION of the Rule

The Chatham House Rule originated at Chatham House with the aim of providing anonymity to speakers and to encourage openness and the sharing of information. It is now used throughout the world as an aid to free discussion. Meetings do not have to take place at Chatham House to be held under the Rule.

Meetings, events and discussions held at Chatham House are normally conducted 'on the record' with the Rule occasionally invoked at the speaker's request. In cases where the Rule is not considered sufficiently strict, an event may be held 'off the record'.

If you're interested in what the Chatham House is, besides the link to the rule (above) you can check out the following link to learn about the home of the Royal Institute of International Affairs.

Three things will likely come of this post:

1. You can confidently acknowledge your understanding of The Rule and use it in the spirit under which it was constructed
2. You've now realized that all that stuff you blabbed about from those prior meetings under The Rule (which you didn't understand) is someday going to come back and punt you right in the blender
3. You can now start evoking the Chatham House rule in random places regarding all manner of activities and confuse the hell out of people.  I quite like declaring it before ordering Chili Poppers and girlie drinks at TGI Friday's, for example.

You can probably guess why I'm writing this.

Some people just never learn.

My work here is done.

Carry on.

/Hoff