Rational Survivability

Patching The Cloud?

Just to confuse you, as a lead-in on this topic, please first read my recent rant titled "Will You All Please Shut-Up About Securing THE Cloud...NO SUCH THING..." 

Let's say the grand vision comes to fruition where enterprises begin to outsource to a cloud operator the hosting of previously internal complex mission-critical enterprise applications. 

After all, that's what we're being told is the Next Big Thing™

In this version of the universe, the enterprise no longer owns the operational elements involved in making the infrastructure tick -- the lights blink, packets get delivered, data is served up and it costs less for what is advertised is the same if not better reliability, performance and resilience.

Oh yes, "Security" is magically provided as an integrated functional delivery of service.

Tastes great, less datacenter filling.

So, in a corner case example, what does a boundary condition like the out-of-cycle patch release of MS08-067 mean when your infrastructure and applications are no longer yours to manage and the ownership of the "stack" disintermediates you from being able to control how, when or even if vulnerability remediation anywhere in the stack (from the network on up to the app) is assessed, tested or deployed.

Your application is sitting atop an operating system and underlying infrastructure that is managed by the cloud operator.  This "datacenter OS" may not be virtualized or could actually be sitting atop a hypervisor which is integrated into the operating system (Xen, Hyper-V, KVM) or perhaps reliant upon a third party solution such as VMware.  The notion of cloud implies shared infrastructure and hosting platforms, although it does not imply virtualization.

A patch affecting any one of the infrastructure elements could cause a ripple effect on your hosted applications.  Without understanding the underlying infrastructure dependencies in this model, how does one assess risk and determine what any patch might do up or down the stack?  How does an enterprise that has no insight into the "black box" model of the cloud operator, setup a dev/test/staging environment that acceptably mimics the operating environment? 

How does one negotiate the process for determining when and how a patch is deployed?  Where does the cloud operator draw the line?   If the cloud fabric is democratized across constituent enterprise customers, however isolated, how does a cloud provider ensure consistent distributed service?  If an application can be dynamically provisioned anywhere in the fabric, consistency of the platform is critical.

I hate to get all "Star Trek II: The Wrath of Khan" on you, but as Spock said, "The needs of the many outweigh the needs of the few."  How, when and if a provider might roll a patch has a broad impact across the entire customer base -- as it has had in the hosting markets for years -- but again the types of applications we are talking about here are far different than what we we're used to today where the applications and the infrastructure are inextricably joined at the hip.

Hosting/SaaS providers today can scale because of one thing: standardization.  Certainly COTS applications can be easily built on standardized tiered models for compute, storage and networking, but again, we're being told that enterprises will move all their applications to the cloud, and that includes bespoke creations. 

If that's not the case, and we end up with still having to host some apps internally and some apps in the cloud, we've gained nothing because we won't be able to eliminate the infrastructure needed to support either.

What about it?  Do you see cloud computing as just an extension of SaaS and hosting of today?  Do you see dramatically different issues arise based upon the types of information and applications that are being described in this model?  We've seen issues such as data ownership, privacy and portability bubble up, but these are much more basic operational questions.

This is obviously a loaded set of questions for which I have much to say -- some of which is obvious -- but I'd like to start a discussion, not a rant.

/Hoff

*This little ditty was inspired by a Twitter exchange with Bob Rudis who was complaining that Amazon's EC2 service did not have the MS08-067 patch built into the AMI...Check out this forum entry from Amazon, however, as it's rather apropos regarding the very subject of this blog...

On Patch Tuesdays for Virtualization Platforms...

In December I made note of an interesting post on the virtualization.info blog titled "Patch Tuesday for VMware."  This issue popped up today in conversation with a customer and I thought to bubble it back up for discussion.

The post focused on some work done by Ronald Oglesby and Dan Pianfetti from GlassHouse Technologies regarding the volume, frequency and distribution of patches across VMware's ESX platform. 

When you combine Ronald and Dan's data with Kris Lamb's from ISS that I wrote about a few months ago, it's quite interesting.

The assertion that Ronald/Dan are making in their post is that platforms like VMware's ESX have to date required just as much care and feeding from a patching/vulnerability management perspective as a common operating system such as a Windows Server:

So why make this chart and look at the time between patches? Let's take a hypothetical server built on July 2nd of 2007, 5 months ago almost exactly. Since being built on that day and put into production that server would have been put into maintenance mode and patched/updated eight times. That's right eight (8) times in 5 months. How did this happen? Let's look at the following timeline:

Maybe it's time to slow down and look at this as a QA issue? Maybe it's time to stop thinking about these platforms as rock solid, few moving parts systems? Maybe it's better for us not to draw attention to it, and instead let it play out and the markets decide whether all this patching is a good thing or not. Obviously patching is a necessary evil, and maybe because we are so used to it in the Windows world, we have ignored this so far. But a patch every 18.75 days for our "hypothetical" server is a bit much, don't you think?

I think this may come as a shock to some who have long held the belief that bare-metal, Type 1 virtualization platforms require little or no patching and that because of this, the "security" and availability of virtualized hosts was greater than that of their non-virtualized counterparts.

The reality of the situation and the effort and potential downtime (despite tools that help) have led to unexpected service level deviance, hidden costs and latent insecurity in deployed virtualized environments.  I think Ronald/Dan said it best:

If a client is buying into the idea of server virtualization as a piece of infrastructure (like a SAN or a switch) only to see the types of patching we see in Windows, they are going to get smacked in the face with the reality that these are SERVERS. The reality that the vendors are sticking so much into the OS that patches are going to happen just as often as with Windows Servers? Or, if the client believes the stability/rock solidness and skips a majority of general patches, they wind up with goofy time issues or other problems with iSCSI, until they catch up.

As a counterpoint to this argument I had hoped to convince Kris Lamb to extend his patch analysis of VMware's releases and see if he could tell how many patched vulnerabilities existed in the service console (the big ol' fat Linux OS globbed onto the side of ESX) versus the actual VMM implementation itself.  For some reason, he's busy with his day job. ;) This is really an important data point.  I guess I'll have to do that myself ;(

The reason why this is important is exactly the reason that you're seeing VMware and other industry virtualization players moving to embedded hypervisors; skinnying down the VMM's to yield less code, less attack surface and hopefully less vulnerabilities.  So, to be fair, the evolution of the virtualization platforms is really on-par with what one ought to expect with a technology that's still fairly nascent.

In fact, that's exactly Nand Mulchandani, VMware's Sr. Director of Security Product Management & Marketing in response to Ronald/Dan's post:

As the article points out, "patching is a necessary evil" - and that the existence of ESX patches should not come as a shock to anyone. So let’s talk about the sinister plan behind the increase in ESX patches. Fortunately, the answer is in the article itself. Our patches contain a lot of different things, from hardware compatibility updates, feature enhancements, security fixes, etc.

...

We also want customers to view ESX as an appliance - or more accurately, as a product that has appliance-like characteristics.

...

Speaking of appliances, another thing to consider is that we are now offering ESX in a number of different form-factors, including the brand new ESX Server 3i. 3i will have a significantly different patch characteristics - it does not have a Console OS and has a different patching mechanism than ESX that will be very attractive to customers.

I see this as a reasonable and rational response to the issue, but it does point out that whether you use VMware or any other vendor's virtualization platform, you should make sure to recognize that patching and vulnerability management of the underlying virtualization platforms is another -- if not really critical -- issue that will require operational attention and potential cost allocation.

/Hoff

P.S. Mike D. does a great job of stacking up other vendors against Microsoft in this vein such as Microsoft, Virtual Iron, and SWSoft.

Virtualization Threat Surface Expands: We Weren't Kidding...

First the Virtualization Security Public Service Announcement:

By now you've no doubt heard that Ryan Smith and Neel Mehta from IBM/ISS X-Force have discovered vulnerabilities in VMware's DHCP implementation that could allow for "...specially crafted packets to gain system-level privileges" and allow an attacker to execute arbitrary code on the system with elevated privileges thereby gaining control of the system.   

Further, Dark Reading details that Rafal Wojtczvk (whose last name's spelling is a vulnerability in and of itself!) from McAfee discovered the following vulnerability:

A vulnerability that could allow a guest operating system user with administrative privileges to cause memory corruption in a host process, and potentially execute arbitrary code on the host. Another fix addresses a denial-of-service vulnerability that could allow a guest operating system to cause a host process to become unresponsive or crash.

...and yet another from the Goodfellas Security Research Team:

An additional update, according to the advisory, addresses a security vulnerability that could allow a remote hacker to exploit the library file IntraProcessLogging.dll to overwrite files in a system. It also fixes a similar bug in the library file vielib.dll.

It is important to note that these vulnerabilities have been mitigated by VMWare at the time of this announcement.  Further information regarding mitigation of all of these vulnerabilities can be found here.

---

You can find details regarding these vulnerabilities via the National Vulnerability Database here:

CVE-2007-0061 - The DHCP server in EMC VMware Workstation before 5.5.5 Build 56455 and 6.x before 6.0.1 Build 55017, Player before 1.0.5 Build 56455 and Player 2 before 2.0.1 Build 55017, ACE before 1.0.3 Build 54075 and ACE 2 before 2.0.1 Build 55017, and Server before 1.0.4 Build 56528 allows remote attackers to execute arbitrary code via a malformed packet that triggers "corrupt stack memory.

CVE-2007-0062 - Integer overflow in the DHCP server in EMC VMware Workstation before 5.5.5 Build 56455 and 6.x before 6.0.1 Build 55017, Player before 1.0.5 Build 56455 and Player 2 before 2.0.1 Build 55017, ACE before 1.0.3 Build 54075 and ACE 2 before 2.0.1 Build 55017, and Server before 1.0.4 Build 56528 allows remote attackers to execute arbitrary code via a malformed DHCP packet that triggers a stack-based buffer overflow.

CVE-2007-0063 - Integer underflow in the DHCP server in EMC VMware Workstation before 5.5.5 Build 56455 and 6.x before 6.0.1 Build 55017, Player before 1.0.5 Build 56455 and Player 2 before 2.0.1 Build 55017, ACE before 1.0.3 Build 54075 and ACE 2 before 2.0.1 Build 55017, and Server before 1.0.4 Build 56528 allows remote attackers to execute arbitrary code via a malformed DHCP packet that triggers a stack-based buffer overflow.

CVE-2007-4496 - Unspecified vulnerability in EMC VMware Workstation before 5.5.5 Build 56455 and 6.x before 6.0.1 Build 55017, Player before 1.0.5 Build 56455 and Player 2 before 2.0.1 Build 55017, ACE before 1.0.3 Build 54075 and ACE 2 before 2.0.1 Build 55017, and Server before 1.0.4 Build 56528 allows authenticated users with administrative privileges on a guest operating system to corrupt memory and possibly execute arbitrary code on the host operating system via unspecified vectors.

CVE-2007-4155 - Absolute path traversal vulnerability in a certain ActiveX control in vielib.dll in EMC VMware 6.0.0 allows remote attackers to execute arbitrary local programs via a full pathname in the first two arguments to the (1) CreateProcess or (2) CreateProcessEx method.

---

I am happy to see that VMware moved on these vulnerabilities (I do not have the timeframe of this disclosure and mitigation available.)  I am convinced that their security team and product managers truly take this sort of thing seriously.

However, this just goes to show you that as the virtualization platforms enter further-highlighted mainstream adoption, exploitable vulnerabilities will continue to follow as those who follow the money begin to pick up the scent. 

This is another phrase that's going to make a me a victim of my own Captain Obvious Award, but it seems like we've been fighting this premise for too long now.  I recognize that this is not the first set of security vulnerabilities we've seen from VMware, but I'm going to highlight them for a reason.

It seems that due to a lack of well-articulated vulnerabilities that extended beyond theoretical assertions or POC's, the sensationalism of research such as Blue Pill has desensitized folks to the emerging realities of virtualization platform attack surfaces.

I've blogged about this over the last year and a half, with the latest found here and an interview here.  It's really just an awareness campaign.  One I'm more than willing to wage given the stakes.  If that makes me the noisy canary in the coal mine, so be it.

These very real examples are why I feel it's ludicrous to take seriously any comments that suggest by generalization that virtualized environments are "more secure" by design; it's software, just like anything else, and it's going to be vulnerable.

I'm not trying to signal that the sky is falling, just the opposite.  I do, however, want to make sure we bring these issues to your attention.

Happy Patching!

/Hoff

Take5 (Episode #5) - Five Questions for Allwyn Sequeira, SVP of Product Operations, Blue Lane

This fifth episode of Take5 interviews Allwyn Sequeira, SVP of Product Operations for Blue Lane.  

First a little background on the victim:

Mr. Palmer is the President and Chief Executive Officer of Blue Lane Technologies. Previously, Jeff served as President of GetThere, the leading online corporate travel procurement solution, based in Menlo Park, CA. While at the company, GetThere grew to become the largest factor in the corporate online travel market, holding 70% market share among the top 100 US corporations, #1 share in the overall market with more than 3000 customers worldwide, and transaction volumes exceeding one million per month. GetThere went public on the NASDAQ in 1999 and was acquired by Sabre Holdings (NYSE: TSG) for $750 million in 2000.
         
Earlier in his career, Mr. Palmer served in a variety of general management and marketing officer roles at Tristrata Security, Memco Software, Pilot Software and BBN, establishing experience in enterprise networking, security, and applications software.
         
Mr. Palmer holds a Bachelor of Science degree from the Massachusetts Institute of Technology and an MBA from Harvard Business School.

Allwyn Sequeira is Senior Vice President of Product Operations at Blue Lane Technologies, responsible for managing the overall product life cycle, from concept through research, development and test, to delivery and support. He was previously the Senior Vice President of Technology and Operations at netVmg, an intelligent route control company acquired by InterNap in 2003, where he was responsible for the architecture, development and deployment of the industry-leading flow control platform. Prior to netVmg, he was founder, Chief Technology Officer and Executive Vice President of Products and Operations at First Virtual Corporation (FVC), a multi-service networking company that had a successful IPO in 1998. Prior to FVC, he was Director of the Network Management Business Unit at Ungermann-Bass, the first independent local area network company. Mr. Sequeira has previously served as a Director on the boards of FVC and netVmg.

Mr. Sequeira started his career as a software developer at HP in the Information Networks Division, working on the development of TCP/IP protocols. During the early 1980's, he worked on the CSNET project, an early realization of the Internet concept. Mr. Sequeira is a recognized expert in data networking, with twenty five years of experience in the industry, and has been a featured speaker at industry leading forums like Networld+Interop, Next Generation Networks, ISP Con and RSA Conference.

Mr. Sequeira holds a Bachelor of Technology degree in Computer Science from the Indian Institute of Technology, Bombay, and a Master of Science in Computer Science from the University of Wisconsin, Madison.   

Allwyn, despite all this good schoolin' forgot to send me a picture, so he gets what he deserves ;)
(Ed: Yes, those of you quick enough were smart enough to detect that the previous picture was of Brad Pitt and not Allwyn.  I apologize for the unnecessary froth-factor.)

 Questions:

1) Blue Lane has two distinct product lines, VirtualShield and PatchPoint.  The former is a software-based solution which provides protection for VMware Infrastructure 3 virtual servers as an ESX VM plug-in whilst the latter offers a network appliance-based solution for physical servers.  How are these products different than either virtual switch IPS' like Virtual Iron or in-line network-based IPS's?

IPS technologies have been charged with the incredible mission of trying to protect everything from anything.  Overall they've done well, considering how much the perimeter of the network has changed and how sophisticated hackers have become. Much of their core technology, however, was relevant and useful when hackers could be easily identified by their signatures. As many have proclaimed, those days are coming to an end.

A defense department official recently quipped, "If you offer the same protection for your toothbrushes and your diamonds you are bound to lose fewer toothbrushes and more diamonds."  We think that data center security similarly demands specialized solutions.  The concept of an enterprise network has become so ambiguous when it comes to endpoints and devices and supply chain partners, etc. we think its time to think more realistically in terms of trusted, yet highly available zones within the data center.

It seems clear at this point that different parts of the network need very different security capabilities.  Servers, for example need highly accurate solutions that do not block or impede good traffic and can correct bad traffic, especially when it comes to closing network-facing vulnerability windows.  They need to maintain availability with minimal latency for starters; and that has been a sort of Achilles heel for signature-based approaches.  Of course, signatures also bring considerable management burdens over and beyond their security capabilities.

No one is advocating turning off the IPS, but rather approaching servers with more specialized capabilities.  We started focusing on servers years ago and established very sophisticated application and protocol intelligence, which has allowed us to correct traffic inline without the noise, suspense and delay that general purpose network security appliance users have come to expect.

IPS solutions depend on deep packet inspection typically at the perimeter based on regexp pattern matching for exploits.  Emerging challenges with this approach have made alert and block modes absolutely necessary as most IPS solutions aren't accurate enough to be trusted in full library block. 

Blue Lane uses a vastly different approach.  We call it deep flow inspection/correction for known server vulnerabilities based on stateful decoding up to layer 7.  We can alert, block and correct, but most of are deployments are in correct mode, with our full capabilities enabled. From an operational standpoint we have substantially different impacts.

A typical IPS may have 10K signatures while experts recommend turning on just a few hundred.  That kind of marketing shell game (find out what really works) means that there will be plenty of false alarms, false positives and negatives and plenty of tuning.  With polymorphic attacks signature libraries can increase exponentially while not delivering meaningful improvements in protection. 

Blue Lane supports about 1000 inline security patches across dozens of very specific server vulnerabilities, applications and operating systems.  We generate very few false alarms and minimal latency.  We don't require ANY tuning.  Our customers run our solution in automated, correct mode.

The traditional static signature IPS category has evolved into an ASIC war between some very capable players for the reasons we just discussed.Exploding variations of exploits and vectors means that exploit-centric approaches will require more processing power.

Virtualization is pulling the data center into an entirely different direction, driven by commodity processors.  So of course our VirtualShield solution was a much cleaner setup with a hypervisor; we can plug into the hypervisor layer and run on top of existing hardware, again with minimal latency and footprint.

You don't have to be a Metasploit genius to evade IPS signatures.  Our higher layer 7 stateful decoding is much more resilient. 

2) With zero-days on the rise, pay-for-play vulnerability research and now Zero-Bay (WabiSabiLabi) vulnerability auctions and the like, do you see an uptake in customer demand for vulnerability shielding solutions?

Exploit-signature technologies are meaningless in the face of evanescent, polymorphic threats, resulting in 0-day exploits. Slight modifications to signatures can bypass IPSes, even against known vulnerabilities.  Blue Lane technology provides 0-day protection for any variant of an exploit against known vulnerabilities.  No technology can provide ultimate protection against 0-day exploits based on 0-day vulnerabilities. However, this requires a different class of hacker.

3) As large companies start to put their virtualization strategies in play, how do you see customers addressing securing their virtualized infrastructure?  Do they try to adapt existing layered security methodologies and where do these fall down in a virtualized world?

I've explored this topic in depth at the Next Generation Data Center conference last week. Also, your readers might be interested in listening to a recent podcast: The Myths and Realities of Virtualization Security: An Interview. 

To summarize, there are a few things that change with virtualization, that folks need to be aware of.  It represents a new architecture.  The hypervisor layer represents the un-tethering and clustering of VMs, and centralized control.  It introduces a new virtual network layer.  There are entirely new states of servers, not anticipated by traditional static security approaches (like instant create, destroy, clone, suspend, snapshot and revert to snapshot). 

Then you'll see unprecedented levels of mobility and new virtual appliances and black boxing of complex stacks including embedded databases.  Organizations will have to work out who is responsible for securing this very fluid environment.  We'll also see unprecedented scalability with Infiniband cores attaching LAN/SAN out to 100's of ESX hypervisors and thousands of VMs.

Organizations will need the capability to shield these complex, fluid environments; because trying to keep track of individual VMs, states, patch levels, locations will make tuning an IPS for polymorphic attacks look like childs play in comparison.   Effective solutions will need to be highly accurate, low latency solutions deployed in correct mode. Gone will be the days of man-to-man blocking and tuning.  Here to stay are the days of zone defense.

4) VMware just purchased Determina and intends to integrate their memory firewall IPS product as an ESX VM plug-in.  Given your early partnership with VMware, are you surprised by this move?  Doesn't this directly compete with the VirtualSheild offering?

I wouldn't read too much into this. Determina hit the wall on sales, primarily because it's original memory wall technology was too intrusive, and fell short of handling new vulnerabilities/exploits.

This necessitated the LiveShield product, which required ongoing updates, destroying the value proposition of not having to touch servers, once installed. So, this is a technology/people acquisition, not a product line/customer-base acquisition.

VMware was smart to get a very bright set of folks, with deep memory/paging/OS, and a core technology that would do well to be integrated into the hypervisor for the purpose of hypervisor hardening, and interVM isolation. I don't see VMware entering the security content business soon (A/V, vulnerabilities, etc.). I see Blue Lane's VirtualShield technology integrated into the virtual networking layer (vSwitch), as a perfect complement to anything that will come out of the Determina acquisition.

5) Citrix just acquired XenSource.  Do you have plans to offer VirtualShield for Xen? 

A smart move on Citrix's part to get back into the game. Temporary market caps don't matter. Virtualization matters. If Citrix can make this a two or three horse race, it will keep the VMware, Citrix, Microsoft triumvirate on their toes, delivering better products, and net good for the customer.

Regarding BlueLane, and Citrix/Xensource, we will continue to pay attention to what customers are buying as they virtualize their data centers. For now, this is a one horse show :-)

Remotely Exploitable Dead Frog with Embedded Web Server - The "Anatomy" of a Zero-Day Threat Surface

You think I make this stuff up, don't you?

Listen, I'm a renaissance man and I look for analogs to the security space anywhere and everywhere I can find them.

I maintain that next to the iPhone, this is the biggest thing to hit the security world since David Maynor found Jesus (in a pool hall, no less.)

I believe InfoSec Sellout already has produced a zero-day for this using real worms.  No Apple products were harmed during the production of this webserver, but I am sad to announce that there is no potential for adding your own apps to the KermitOS...an SDK is available, however.

The frog's dead.  Suspended in a liquid.  In a Jar.  Connected to the network via an Ethernet cable.  You can connect to the embedded webserver wired into its body parts.  When you do this, you control which one of its legs twitch.  pwned!

You can find the pertinent information here.

A Snort signature will be available shortly.

/Hoff

(Image and text below thanks to Boing Boing)

The Experiments in Galvanism frog floats in mineral oil, a webserver installed it its guts, with wires into its muscle groups. You can access the frog over the network and send it galvanic signals that get it to kick its limbs.

Experiments in Galvanism is the culmination of studio and gallery experiments in which a miniature computer is implanted into the dead body of a frog specimen. Akin to Damien Hirst's bodies in formaldehyde, the frog is suspended in clear liquid contained in a glass cube, with a blue ethernet cable leading into its splayed abdomen. The computer stores a website that enables users to trigger physical movement in the corpse: the resulting movement can be seen in gallery, and through a live streaming webcamera.
    - Risa Horowitz

Garnet Hertz has implanted a miniature webserver in the body of a frog specimen, which is suspended in a clear glass container of mineral oil, an inert liquid that does not conduct electricity. The frog is viewable on the Internet, and on the computer monitor across the room, through a webcam placed on the wall of the gallery. Through an Ethernet cable connected to the embedded webserver, remote viewers can trigger movement in either the right or left leg of the frog, thereby updating Luigi Galvani's original 1786 experiment causing the legs of a dead frog to twitch simply by touching muscles and nerves with metal.

Experiments in Galvanism is both a reference to the origins of electricity, one of the earliest new media, and, through Galvani's discovery that bioelectric forces exist within living tissue, a nod to what many theorists and practitioners consider to be the new new media: bio(tech) art.
    - Sarah Cook and Steve Dietz

Fat Albert Marketing and the Monetizing of Vulnerability Research

Over the last couple of years, we've seen the full spectrum of disclosure and "research" portals arrive on scene; examples stem from the Malware Distribution Project to 3Com/TippingPoint's Zero Day Initiative.  Both of these examples illustrate ways of monetizing the output trade of vulnerability research.   

Good, bad or indifferent, one would be blind not to recognize that these services are changing the landscape of vulnerability research and pushing the limits which define "responsible disclosure."

It was only a matter of time until we saw the mainstream commercial emergence of the open vulnerability auction which is just another play on the already contentious marketing efforts blurring the lines between responsible disclosure for purely "altruistic" reasons versus commercial gain.

Enter Wabisabilabi, the eBay of Zero Day vulnerabilities.   

This auction marketplace for vulnerabilities is marketed as a Swiss "...Laboratory & Marketplace Platform for Information Technology Security" which "...helps customers defend their databases, IT infrastructure, network, computers, applications, Internet offerings and access."

Despite a name which sounds like Mushmouth from Fat Albert created it (it's Japanese in origin, according to the website) I am intrigued by this concept and whether or not it will take off.

I am, however, a little unclear on how customers are able to purchase a vulnerability and then become more secure in defending their assets. 

A vulnerability without an exploit, some might suggest, is not a vulnerability at all -- or at least it poses little temporal risk.  This is a fundamental debate of the definition of a Zero-Day vulnerability. 

Further, a vulnerability that has a corresponding exploit but without a countermeasure (patch, signature, etc.) is potentially just as useless to a customer if you have no way of protecting yourself.

If you can't manufacture a countermeasure, even if you hoard the vulnerability and/or exploit, how is that protection?  I suggest it's just delaying the inevitable.

I am wondering how long until we see the corresponding auctioning off of the exploit and/or countermeasure?  Perhaps by the same party that purchased the vulnerability in the first place?

Today in the closed loop subscription services offered by vendors who buy vulnerabilities, the subscribing customer gets the benefit of protection against a threat that they may not even know they have, but for those who can't or won't pony up the money for this sort of subscription (which is usually tied to owning a corresponding piece of hardware to enforce it,) there exists a point in time between when the vulnerability is published and when it this knowledge is made available universally.

Depending upon this delta, these services may be doing more harm than good to the greater populous.

In fact, Dave G. over at Matasano argues quite rightly that by publishing even the basic details of a vulnerability that "researchers" will be able to more efficiently locate the chunks of code wherein the vulnerability exists and release this information publicly -- code that was previously not known to even have a vulnerability.

Each of these example vulnerability service offerings describes how the vulnerabilities are kept away from the "bad guys" by qualifying their intentions based upon the ability to pay for access to the malicious code (we all know that criminals are poor, right?)  Here's what the Malware Distribution Project describes as the gatekeeper function:

Why Pay?

Easy; it keeps most, if not all of the malicious intent, outside the gates. While we understand that it may be frustrating to some people with the right intentions not allowed access to MD:Pro, you have to remember that there are a lot of people out there who want to get access to malware for malicious purposes. You can't be responsible on one hand, and give open access to everybody on the other, knowing that there will be people with expressly malicious intentions in that group.

ZDI suggests that by not reselling the vulnerabilities but rather protecting their customers and ultimately releasing the code to other vendors, they are giving back:

The Zero Day Initiative (ZDI) is unique in how the acquired vulnerability information is used. 3Com does not re-sell the vulnerability details or any exploit code. Instead, upon notifying the affected product vendor, 3Com provides its customers with zero day protection through its intrusion prevention technology. Furthermore, with the altruistic aim of helping to secure a broader user base, 3Com later provides this vulnerability information confidentially to security vendors (including competitors) who have a vulnerability protection or mitigation product.

As if you haven't caught on yet, it's all about the Benjamins. 

We've seen the arguments ensue regarding third party patching.  I think that this segment will heat up because in many cases it's going to be the fastest route to protecting oneself from these rapidly emerging vulnerabilities you didn't know you had.

/Hoff

Take5- Five Questions for Chris Wysopal, CTO Veracode

In this first installment of Take5, I interview Chris Wysopal, the CTO of Veracode about his new company, secure coding, vulnerability research and the recent forays into application security by IBM and HP.

This entire interview was actually piped over a point-to-point TCP/IP connection using command-line redirection through netcat.  No packets were harmed during the making of this interview...

First, a little background on the victim, Chris Wysopal:

Chris Wysopal is co-founder and CTO of Veracode. He has testified on Capitol Hill on the subjects of government computer security and how vulnerabilities are discovered in software. Chris co-authored the password auditing tool L0phtCrack, wrote the windows version of netcat, and was a researcher at the security think tank, L0pht Heavy Industries, which was acquired by @stake. He was VP of R&D at @stake and later director of development at Symantec, where he led a team developing binary static analysis technology.

He was influential in the creation of responsible vulnerability disclosure guidelines and a founder of the Organization for Internet Safety.  Chris wrote "The Art of Software Security Testing: Identifying Security Flaws", published by Addison Wesley and Symantec Press in December 2006. He earned his Bachelor of Science degree in Computer and Systems Engineering from Rensselaer Polytechnic Institute.

1) You’re a founder of Veracode which is described as the industry’s first provider
of automated, on-demand application security solutions.  What sort of application
security services does Veracode provide?  Binary analysis, Web Apps?
 

Veracode currently offers binary static analysis of C/C++ applications for Windows and Solaris and for Java applications.  This allows us to find the classes of vulnerabilities that source code analysis tools can find but on the entire codebase including the libraries which you probably don't have source code for. Our product roadmap includes support for C/C++ on Linux and C# on .Net.  We will also be adding additional analysis techniques to our flagship binary static analysis.

 
2) Is this a SaaS model? How do you charge for your services?  Do you see manufacturers
using your services or enterprises?
 

Yes. Customers upload their binaries to us and we deliver an analysis of their security flaws via our web portal.  We charge by the megabyte of code.  We have both software vendors and enterprises who write or outsource their own custom software using our services.  We also have enterprises who are purchasing software ask the software vendors to submit their binaries to us for a 3rd party analysis.  They use this analysis as a factor in their purchasing decision. It can lead to a "go/no go" decision, a promise by the vendor to remediate the issues found, or a reduction in price to compensate for the cost of additional controls or the cost of incident response that insecure software necessitates.

 
3) I was a Qualys customer — a VA/VM SaaS company.  Qualys had to spend quite
a bit of time convincing customers that allowing for the storage of their VA data was
secure.  How does Veracode address a customer’s security concerns when uploading their
applications?

We are absolutely fanatical about the security of our customers data.  I look back at the days when I was a security consultant where we had vulnerability data on laptops and corporate file shares and I say, "what were we thinking?"  All customer data at Veracode is encrypted in storage and at rest with a unique key per application and customer.  Everyone at Veracode uses 2 factor authentication to log in and 2 factor is the default for customers.  Our data center is a SAS 70 Type II facility. All data access is logged so we know exactly who looked at what and when. As security people we are professionally paranoid and I think it shows through in the system we built.  We also believe in 3rd party verification so we have had a top security boutique do a security review our portal application.

 
4) With IBM’s acquisition of Watchfire and today’s announcement that HP will buy
SPI Dynamics, how does Veracode stand to play in this market of giants who will
be competing to drive service revenues?
 

We have designed our solution from the ground up to have the Web 2.0 ease of use and experience and we have the quality of analysis that I feel is the best in the market today.  An advantage is Veracode is an independent assessment company that customers can trust to not play favorites to other software companies because of partnerships or alliances. Would Moody's or Consumer Reports be trusted as a 3rd party if they were part of a big financial or technology conglomerate? We feel a 3rd party assessment is important in the security world.

 
5) Do you see the latest developments in vulnerability research with the drive for
pay-for-zeroday initiatives pressuring developers to produce secure code out of the box
for fear of exploit or is it driving the activity to companies like yours?
 

I think the real driver for developers to produce secure code and for developers and customers to seek code assessments is the reality that the costs of insecure code goes up everyday and its adding to the operational risk of companies that use software.  People exploiting vulnerabilities are not going away and there is no way to police the internet of vulnerability information.  The only solution is for customers to demand more secure code, and proof of it, and for developers to deliver more secure code in response.

CSI Working Group on Web Security Reseach Law Concludes...Nothing

In May I blogged what I thought was an interesting question regarding the legality and liability of reverse engineering in security vulnerability research.  That discussion focused on the reverse engineering and vulnerability research of hardware and software products that were performed locally.

I continued with a follow-on discussion and extended the topic to include security vulnerability research from the web-based perspective in which I was interested to see how different the opinions on the legality and liability were from many of the top security researchers as it relates to the local versus remote vulnerability research and disclosure perspectives.

As part of the last post, I made reference to a working group organized by CSI whose focus and charter were to discuss web security research law.  This group is made up of some really smart people and I was looking forward to the conclusions reached by them on the topic and what might be done to potentially solve the obvious mounting problems associated with vulnerability research and disclosure.

The first report of this group was published yesterday. 

Unfortunately, the conclusions of the working group is an inditement of the sad state of affairs related to the security space and further underscores the sense of utter hopelessness many in the security community experience.

What the group concluded after 14 extremely interesting and well-written pages was absolutely nothing:

The meeting of minds that took place over the past two months advanced the group's collective knowledge on the issue of Web security research law.  Yet if one assumed that the discussion advanced the group's collective understanding of this issue, one might be mistaken.

Informative though the work was, it raised more questions than answers.  In the pursuit of clarity, we found, instead, turbidity.

Thus it follows, that there are many opportunities for further thought, further discussion, further research and further stirring up of murky depths.  In the short term, the working group has plans to pursue the following endeavors:

• Creating disclosure policy guidelines -- both to help site owners write disclosure policies, and for security researchers to understand them.
• Creating guidelines for creating a "dummy" site.
• Creating a more complete matrix of Web vulnerability research methods, written with the purpose of helping attorneys, lawmakers and law enforcement officers understand the varying degrees of invasiveness

Jeremiah Grossman, a friend and one of the working group members summarized the report and concluded with the following: "...maybe within the next 3-5 years as more incidents like TJX occur, we’ll have both remedies."  Swell.

Please don't misunderstand my cynical tone and disappointment as a reflection on any of the folks who participated in this working group -- many of whom I know and respect.  It is, however, sadly another example of the hamster wheel of pain we're all on when the best and brightest we have can't draw meaningful conclusions against issues such as this.

I was really hoping we'd be further down the path towards getting our arms around the problem so we could present meaningful solutions that would make a dent in the space.  Unfortunately, I think where we are is the collective shoulder shrug shrine of cynicism perched periously on the cliff overlooking the chasm of despair which drops off into the trough of disillusionment.

Gartner needs a magic quadrant for hopelessness. <sigh>  I feel better now, thanks.

/Hoff

Should Vendors Mitigate All Vulnerabilities Immediately?

I read an interesting piece by Roger Grimes @ InfoWorld wherein he described the situation of a vendor who was not willing to patch an unsupported version of software even though it was vulnerable and shown to be (remotely) exploitable.

Rather, the vendor suggested that using some other means (such as blocking the offending access port) was the most appropriate course of action to mitigate the threat.

What's interesting about the article is not that the vendor is refusing to patch older unsupported code, but that ultimately Roger suggests that irrespective of severity, vendors should immediately patch ANY exploitable vulnerability -- with or without public disclosure.

A reader who obviously works for a software vendor commented back with a reply that got Roger thinking and it did for me, also.   The reader suggests that they don't patch lower severity vulnerabilities immediately (they actually "sit on them" until a customer raises a concern) but instead focus on the higher-severity discoveries:

The reader wrote to say that his company often sits on security bugs until they are publicly announced or until at least one customer complaint is made. Before you start disagreeing with this policy, hear out the rest of his argument.

“Our company spends significantly to root out security issues," says the reader. "We train all our programmers in secure coding, and we follow the basic tenets of secure programming design and management. When bugs are reported, we fix them. Any significant security bug that is likely to be high risk or widely used is also immediately fixed. But if we internally find a low- or medium-risk security bug, we often sit on the bug until it is reported publicly. We still research the bug and come up with tentative solutions, but we don’t patch the problem.”

In the best of worlds, I'd agree with Roger -- vendors should patch all vulnerabilities as quickly as possible once discovered, irrespective of whether or not the vulnerability or exploit is made public.  The world would be much better -- assuming of course that the end-user could actually mitigate the vulnerability by applying the patch in the first place.

Let's play devil's advocate for a minute...

Back here on planet Earth, the prioritization of mitigating vulnerabilities and the resource allocation to mitigate the vulnerability is approached by vendors not unlike the way in which the consumers choose to apply patches of the same; most look at the severity of a vulnerability and start from the highest severity and make their way down.  That's just the reality of my observation.   

So, for the bulk of these consumers, is the vendor's response out of line?  It seems in total alignment.

As a counterpoint to my own discussion here, I'd suggest that using prudent risk management best practice, one would protect those assets that matter most.  Sometimes this means that one would mitigate a Sev3 (medium) vulnerability over a Sev5 (highest) based upon risk exposure...this is where solutions like Skybox come in to play.  Vendors can't attach a weight to an asset, all they can do is assess the impact that an exploitable vulnerability might have on their product...

The reader's last comment caps it off neatly with a challenge:

“Industry pundits such as yourself often say that it benefits customers more when a company closes all known security holes, but in my 25 years in the industry, I haven’t seen that to be true. In fact I’ve seen the exact opposite. And before you reply, I haven’t seen an official study that says otherwise. Until you can provide me with a research paper, everything you say in reply is just your opinion. With all this said, once the hole is publicly announced, or becomes high-risk, we close it. And we close it fast because we already knew about it, coded a solution, and tested it.”

I'm not sure I need an official study to respond to this point, but I'd be interested in if there were such a thing.  Gerhard Eschelbeck has been studying vulnerabilities and their half-lives for some time.  I'd be interested to see how this plays.

So, read the gentleman's posts; in some cases his comments are understandable and in others they're hard to swallow...this definitely depends upon which (if not both) side of the fence you stand.  All vendors are ultimately consumers in one form or another...

Thoughts?

/Hoff

Liability of Reverse Engineering Security Vulnerability Research?

(Ed.: Wow, some really great comments came out of this question.  I did a crappy job framing the query but there exists a cohesiveness to both the comments and private emails I have received that shows there is confusion in both terminology and execution of reverse engineering. 

I suppose the entire issue of reverse engineering legality can just be washed away by what appeared to me as logical and I stated in the first place -- there is no implied violation of an EULA or IP if one didn't agree to it in the first place (duh!) but I wanted to make sure that my supposition was correct.]

I have a question that hopefully someone can answer for me in a straightforward manner.  It  popped into my mind yesterday in an unrelated matter and perhaps it's one of those obvious questions, but I'm not convinced I've ever seen an obvious answer.

If I as an individual or as a representative of a company that performs vulnerability research and assurance engages in reverse engineering of a product that is covered by patent/IP protection and/or EULA's that expressly forbids reverse engineering, how would I deflect liability for violating these tenets if I disclose that I have indeed engaged in reverse engineering?

HID and Cisco have both shown that when backed into a corner, they will litigate and the researcher and/or company is forced to either back down or defend (usually the former.) (Ed:. Poor examples as these do not really fall into the same camp as the example I give below.)

Do you folks who do this for a living (or own/manage a company that does) simply count on the understanding that if one can show "purity" of non-malicious motivation that nothing bad will occur?

It's painfully clear that the slippery slope of full-disclosure plays into this, but help me understand how
the principle of the act (finding vulnerability and telling the company/world about it) outweighs the liability involved.

Do people argue that if you don't purchase the equipment you're not covered under the EULA?  I'm trying to rationalize this.  How does one side-step the law in these cases without playing Russian Roulette?

Here's an example of what I mean.  If you watch this video, the researchers that demonstrated the
Cisco NAC attack @ Black Hat clearly articulate the methods they used to reverse engineer Cisco's products.

I'm not looking for a debate on the up/downside of full disclosure, but more specifically the mechanics of the process used to identify that a vulnerability exists in the first place -- especially if reverse engineering is used.

Perhaps this is a naive question or an uncomfortable one to answer, but I'm really interested.

Thanks,

/Hoff

Another Virtualized Solution for VM Security...

I got an email reminder from my buddy Grant Bourzikas today pointing me to another virtualized security solution for servers from Reflex Security called Reflex VSA.  VSA stands for Virtual Security Appliance and the premise appears to be that you deploy this software within each guest VM and it provides what looks a lot like host-based intrusion prevention functionality per VM.

The functionality is defined thusly:

Reflex VSA solves the problem that traditional network security such as IPS and firewall appliances currently can not solve: detecting and preventing attacks within a virtual server. Because Reflex VSA runs as virtualized application inside the virtualized environment, it can detect and mitigate         threats between virtual hosts and networks.

Reflex VSA Features:
        • Access firewall for permission enforcement for intra-host and external network
           communication
        • Intrusion Prevention with inline blocking and filtering for virtualized networks
        • Anomaly, signature, and rate-based threat detection capability        
        • Network Discovery to discover and map all virtual machines and applications
        • Reflex Command Center, providing a centralized configuration and management
           console, comprehensive reporting tools, and real-time event aggregation and
           correlation   

It does not appear to wrap around or plug-in to the HyperVisor natively, so I'm a little confused as to the difference between deploying VSA and whatever HIPS/NIPS agent a customer might already have deployed on "physical" server instantiations.

Blue Lane's product addresses this at the HyperVisor layer and it would be interesting to me to have the pundits/experts argue the pros/cons of each approach. {Ed. This is incorrect.  Blue Lane's product runs as a VM/virtual appliance also.  With the exposure via API of the hypervisor/virtual switches, products like Blue Lane and Reflex would take advantage to be more flexible, effective and higher performing.}

I'm surprised most of the other "security configuration management" folks haven't already re-branded their agents as being "Virtualization Compliant" to attack this nascent marketspace. < :rolleyes here: >

It's good to see that folks are at least owning up to the fact that intra-VM communications via virtual switches are going to drive a spin on risk models, detection and mitigation tools and techniques.  This is what I was getting at in this blog entry here.

I would enjoy speaking to someone from Reflex to understand their positioning and differentiation better, but isn't this just HIPS per VM?  How's that different than firewall, AV, etc. per VM?

/Hoff

Blue Lane VirtualShield for VMWare - Here we go...

Greg Ness from Blue Lane and I have known each other for a while now, and ever since I purchased Blue Lane's first release of products a few years ago (when I was on the "other" side as a *gasp* customer) I have admired and have taken some blog-derived punishment for my position on Blue Lane's technology.

I have zero interest in Blue Lane other than the fact that I dig their technology and products and think it solves some serious business problems elegantly and efficiently with a security efficacy that is worth its weight in gold.

Vulnerability shielding (or patch emulation...) is a provocative subject and I've gone 'round and 'round with many a fine folk online wherein the debate normally dissolves into the intricacies of IPS vs. vulnerability shielding versus the fact that the solutions solve a business problem in a unique way that works and is cost effective.

That's what a security product SHOULD do.  Yet I digress.

So, back to Greg @ Blue Lane...he let me know a few weeks ago about Blue Lane's VirtualShield offering for  VMWare environments.  VirtualShield is the first commercial product that I know of that specifically tackles problems that everyone knows exists in VM environments but have, until now, sat around twirling thumbs at.

In fact, I alluded to some of these issues in this blog entry regarding the perceived "dangers" of virtualization a few weeks ago.

In short, VirtualShield is designed to protect guest VM's running under a VMWare ESX environment in the following manner (and I quote):

• Protects virtualized servers regardless of physical location or patch-level;
• Provides up-to-date protection with no configuration changes and no agent installation on each virtual machine;
• Eliminates remote threats without blocking legitimate application requests or requiring server reboots; and
• Delivers appropriate protection for specific applications without requiring any manual tuning.

VS basically sits on top of the HyperVisor and performs a similar set of functionality as the PatchPoint solution does for non-VM systems.

Specifically, VirtualShield discovers the virtual servers running on a server and profiles the VM's, the application(s), ports and protocols utilized to build and provision the specific OS and application protections (vulnerability shielding) required to protect the VM.

I think the next section is really the key element of VirtualShield:

As traffic flows through VirtualShield inside the hypervisor, individual sessions are decoded and monitored for vulnerable conditions. When necessary, VirtualShield can replicate the function of a software security patch by applying a corrective action directly within the network stream, protecting the downstream virtual server.

As new security patches are released by software application vendors, VirtualShield automatically downloads the appropriate inline patches from Blue Lane. Updates may be applied dynamically without requiring any reboots or reconfigurations of the virtual servers, the hypervisor, or VirtualShield.

While one might suggest that vulnerability shielding is not new and in some cases certain functionality can be parlayed by firewalls, IPS, AV, etc., I maintain that the manner and model in which Blue Lane elegantly executes this compensating control is unique and effective.

If you're running a virtualized server environment under VMWare's ESX architecture, check out VirtualShield...right after you listen to the virtualization podcast with yours truly from RSA.

/Hoff

My Take on the future of Vulnerability Management

I've followed Alan Shimel's musings on the furture of vulnerability assessment (VA) and found myself nodding along for the most part about where Alan sees the VA market and technology heading:

    "Over the past year, many have asked what is next for   
    VA.  I think we are seeing the answer.  The answer is VA
    is morphing into security configuration management."

Alan preceded this conclusion by illustrating the progression VA has taken over the lifecycle of offerings wherein pure "scanning"  VA toolsets  evolved with integration into vulnerability management (VM) suites that included reporting, remediation and integration and ultimately into a compliance measurement mechanism.

So Alan's alluded that ultimately VA/VM was really a risk management play and I wholeheartedly agree with this.  However, I am confused as to how broadly the definition of "security configuration management (SCM)" spreads its arms under the fold of the definition of "risk management;" it seems to me that SCM is a subset of an overall risk management framework and not vice versa. Perhaps this is already clear enough to folks reading his post, but it wasn't to me.

So, to the punchline:

My vision for where VA is going aligns with Alan's except it doesn't end (or even next-step to) configuration management.  It leapfrogs directly to security risk management (SRM.)  It's also already available in products such as Skybox and RedSeal.  (Disclosure: I am on Skybox's Customer Advisory Board.)

Before you dismiss this as an ad for Skybox, please realize that I've purchased and implemented this solution in conjunction with the other tools I describe.  It represents an incredible tool and methodology that provided a level of transparency and accuracy that allowed me to communicate and make decisions that were totally aligned to the most important elements within my business which is exactly what security should do.

Skybox is the best-kept secret in the risk manager's arsenal.  It's an amazing product that solves some very difficult business problems that stymie security professionals due to their inability to truly communicate (in real time) the risk posture of their organization.  What do I focus on first?

SRM is defined thusly (pasted from Skybox's website because it's the perfect definition that I couldn't improve upon):

IT Security Risk Management is the complete process of understanding threats, prioritizing vulnerabilities, limiting damage from potential attacks, and understanding the impact of proposed changes or patches on the target systems.
  - IT SRM Solution for Vulnerability Management, Gartner, 2005

Security Risk Management collects network infrastructure and security configurations, evaluates vulnerability scan results, maps dependencies among security devices and incorporates the business value of critical assets. SRM calculates all possible access paths, and highlights vulnerabilities that can be exploited by internal and external attackers as well as malicious worms.

By using Security Risk Management, the information overload associated with thousands of network security policies, control devices and vulnerability scans can be demystified and automated. This is accomplished by prioritizing tens of thousands of vulnerabilities into just the few that should be mitigated in order to prevent cyber attacks. The benefit is a more secure network, higher operational efficiency and reduced IT workload.

That being said, starting some 3 years ago I saw where VA/VM was headed and it was down a rathole that provided very little actionable intelligence in terms of managing risk because the VA/VM tools knew nothing of the definition or value of the assets against which the VA was performed. 

We got 600 page reports of vulnerability dumps with massive amounts of false positives.  While the technology has improved, the underlying "evolution" of VA is occurring only because the information it conveyed is not valuable if you want to make an informed decision.    If you manage purely threats and vulnerabilities, you'll be patching forever.

Qualys, Foundstone (nee McAfee) and nCircle (for example) all started to evolve their products by attaching qualitative or quantitative weighting to IT assets (or groups of them) which certainly allowed folks to dashboard the relative impact a vulnerability would have should it be exploited.

The problem is that these impact or "risk" statements (and the ensuing compliance reporting) were still disconnected from the linked dependencies and cascading failure modalities that occur when these assets are interconnected via a complex network.   These tools measure compliance and impact one vulnerability at a time and within a vulnerability diameter of a single host.  They don't have the context of the network, actors, skill sets or hierarchical infrastructure dependencies.  Throw in dynamic routing and numerous controls and network components in between them and these models proved unrealistic and unreliable.

These tools also assume that somehow you're able to apply the results of a risk assessment (RA) and be able to translate the groups of assets to a singular "asset" against which impact can be measured based upon the existence of a vulnerability but not necessarily the potential for exploit -- VA tools have no concept of whether a  control is in place that mitigates the risk.  You also have to understand an map the sub-components of impact against known elements such as confidentiality, integrity and availability.

That's exactly where Skybox and SRM comes in. [From their materials]

The four step process of SRM is a continuous, consistent, automated and repeatable framework:

• Model the IT environment
• Simulate access scenarios and attack paths
• Analyze network connectivity, business risk and regulatory compliance
• Plan optimal mitigation strategies and safe network changes

What this means is that you can take all that nifty VA/VM data, understand and model the network and the steady-state risk posture of your organization, perform "what-if's" and ultimately understand what a change will do to your environment from not only a pure threat/vulnerability perspective, but also a risk/impact one. The accuracy of your data depends on how good you risk input  data is and how up-to-date the  network and  vulnerability results are.  Automate this and it's as close as you can get to "real-time."  Let's call it "near-time."

You can communicate these results at any level up and down the stack of your organization and truly show compliance as it matters not only to the spirit of a regulation or law, but also to your business (and sometimes those are different things.)  It slots into configuration management/security configuration management programs and interfaces with CMM models and quality and governance management frameworks such as CobiT and ITIL.

This, in my opinion, is where VA is headed -- as a vital component of an intelligent risk management portfolio that is called Security Risk Management.

/Hoff

Does the word 'Matasano' mean 'allergic to innovation' in Lithuanian?

(On the advice of someone MUCH smarter than Ptacek or me [my wife] I removed the use of the F/S-bombs in this post.]

Holy crap.  Thomas Ptacek kicked me square in the nuts with his post here in regards to my commentary about Blue Lane's PatchPoint.

I'm really at a loss for words.  I don't really care to start a blog war of words with someone like Thomas Ptacek who is eleventy-billion times smarter than I'll ever hope to be, but I have to admit, his post is the most stupid frigging illustration of derivate label-focused stubborness I have ever witnessed.  For chrissakes, he's challenging tech with marketing slides?  He's starting to sound like Marcus Ranum.

Thomas, your assertions about Patch Point (a product you've never seen in person) are innaccurate.  Your side-swipe bitch-slap commentary about my motivation is offensive.  Your obvious dislike for IPS is noted -- and misdirected.  This is boring.  You assail a product and THEN invite the vendor to respond?  Dude, you're a vendor, too.  Challenging a technology approach is one thing, but calling into question my integrity and motivation?  Back the hell up.

I just got back from an awesome gathering @ BeanSec!2 and Bourbon6 -- so despite the fact that I'm going to hate myself (and this post) in the morning, I have to tell you that 4 of the people that read your post asked "what the hell?"  Did I piss in your corn flakes inadvertenly?

Let me just cut to the chase:

1) I worked with Blue Lane as a customer @ my last job while they were still in stealth.  That's why the "start date" is befor the "live date"
2) When they went live, I bought their product.  The first, in fact.  It worked aces for me.
3) Call it an IPS.  Call it a salad dressing.  I could care less.  It works.  It solves a business problem.
4) I have ZERO interest in their company other than I think it solves said BUSINESS problem.
5) This *is* third party patching because they apply a "patch" which mitigates the exploit related to the vulnerability.  They "patch" the defect.
6) Your comment answers your own question:

You see what they did there? The box takes in shellcode, and then, by “emulating the functionality of a patch”, spits out valid traffic. Wow. That’s amazing. Now, somebody please tell me why that’s any improvement over taking in shellcode, and then, by “emulating the functionality of an attack signature”, spitting out nothing?

...ummm, hello!  An IPS BLOCKS traffic as you illustrate...That's all. 

What if the dumb IPS today kills a valid $50M wire transaction because someone typed 10 more bytes than they should have in a comment field?  Should we truncate they extra 10 bytes or dump the entire transaction? 

IPS's would dump the entire transaction because of an arbitrary and inexact instantiation of a flawed and rigid "policy" that is inaccurate.  That's diametrically opposed to what security SHOULD do.

[Note: I recognize that is a poor example because it doesn't really align with what a 'patch' would do -- perhaps this comment invites the IPS comparison because of it's signature-like action?  I'll come up with a better example and post it in another entry]

Blue Lane does what a security product should; allow good traffic through and make specifically-identified bad traffic good enough.  IPS's don't do that.  They are stupid, deny-driven technology.  They illustrate all that is wrong with how security is deployed today.  If we agree on that, great!  You seem to hate IPS.  So do I.  Blue Lane is not an IPS.  You illustrated that yourself.

Blue Lane is not an IPS because PatchPoint does exactly what a patched system would do if it received a malicious packet...it doesn't toss the entire thing; it takes the good and weeds the bad but allows the request to be processed.  For example, if MS-06-10000 is a patch that mitigates a buffer overflow of a particular application/port such that anything over 1024 bytes can cause the execution or arbitrary code from executing by truncating/removing anything over 1024 bytes, why is this a bad thing to do @ the network layer?

This *IS* a third party patch because within 12 hours (based upon an SLA) they provide a "patch" that mitigates the exploit of a vulnerability and protects the servers behind the applicance WITHOUT touching the host. 

When the vendor issues the real patch, Blue Lane will allow you to flexibly continue to "network patch" with their solution or apply the vendor's.  It gives you time to defend against a potential attack without destroying your critical machines by prematurely deploying patches on the host without the benefit of a controlled regression test.

You're a smart guy.  Don't assail the product in theory without trying it.  Your technical comparisons to the IPS model are flawed from a business and operational perspective and I think that it sucks that you've taken such a narrow-minded perspective on this matter.

Look,  I purchased their product  whilst at my last job.  I'd do it again today.  I have ZERO personal interest in this company or its products other than to say it really is a great solution in the security arsenal today.  That said, I'm going to approach them to get their app. on my platform because it is a fantastic solution to a big problem.

The VC that called me about this today seems to think so, too.

Sorry dude, but I really don't think you get it this time.  You're still eleventy-billion times smarter than I am, but you're also wrong.  Also, until you actually meet me, don't ever call into question my honor, integrity or motivation...I'd never do that to you (or anyone else) so have at least a modicum of respect, eh?

You're still going to advertise BeanSec! 3, right?

Hoff

People Positing Pooh-Poohing Pre-emptive Patching Practices Please Provide Practical Proof…

I was reading Rothman’s latest post on Security Incite regarding patching and I am left a little confused about his position. Despite his estimation of a high score on the “boredometer scale” as it relates to the media’s handling of the patching frenzy ( I *do* agree with that,) I think he’s a little sideways on the issue. At least now we can say that we don’t always agree.

Mike writes:

I hate Patch Tuesday. It's become more of a media circus that anything useful nowadays. So instead of focusing on what needs to be done, most security administrators need to focus on what needs to be patched. Or not. And that takes up more time because in reality, existing defenses reduce (if not eliminate) the impact of many of the vulnerabilities being patched. Maybe it's just my ADD showing, in that these discussions are just not interesting anymore. If you do the right stuff, then there shouldn't be this crazy urgency to patch - you are protected via other defenses. But the lemmings need something to write about, so there you have it.

One lemming, reporting for duty, sir!

Specifically, Mike’s opinion seems to suggest that basically people who “…do the right stuff” don’t need to patch because “…in reality, existing defenses reduce (if not eliminate) the impact of many of the vulnerabilities being patched.”

Since Mike’s always the champion of the little people, I’ll refer him to the fact that perhaps not everyone has all the “…existing defenses” to rely upon – or better yet, keep them up to date (you know, sort of like patching – but for security appliances!)  In fact, I’m going to argue that despite everyone’s best efforts, currently stealthy little zero-day Trojan buggery does a damn good job of getting through these defenses, despite the vendor hype to the contrary.

Emerging technology will make these sorts of vulnerabilities less susceptible to exploit, but that’s going to mean a whole lot of evolution on the part of both the network and the host layer security solutions; there are a LOT of solutions out there now and not ONE of them actually works well in the real world.

I still maintain that relying on the hosts (the things you are protecting – and worried about) to auto-ameliorate is a dumb idea.  It’s akin to why I think we’re going to have to spend just as much time defending the “self-defending network” than we do today with our poorly-defended ones.

I’m going to tippytoe out on the ledge here because I have a feeling that my response to Mike’s enormous generalization will leave him with just as huge of a hole to bury me in, but so be it.  I think he was in a hurry to go on vacation, so please cut him some slack! ;)

Specifically, many of the latest critical vulnerabilities were released to counter exploits targeted at generic desktop applications such as Excel, Powerpoint and Internet Explorer; things that users rely on everyday to perform their job duties at work. 

You don’t have to click on links or open attachments for these beauties to blow up, you just open a document from “your” IT department over the "trusted" network drive map that was infected by a rogue scanning worm which deposited Trojans across your enterprise and BOOM! No such thing as “trust but verify” in the real world, I’m afraid. 

By the way, this little beauty came into your network through a USB drive that someone used to bring their work from home back to the office...sound familiar?

Yep, we can close that hole down with more layers of security software -- or better yet, epoxy the USB slots closed! ;)

OK, OK, I'm generalizing, too.  I know it, but everyone else does it ...

I don’t know what the “right stuff” is, but if it includes using the Internet, Word, Powerpoint or Excel, short of additional layers of host-based security, it’s going to be difficult to defend against those sorts of vulnerabilities without some form of patching (in combination with reasonable amounts of security -- driven by RISK.)

Suggesting that people will do the right thing is noble – laughable, but noble. 

I’ve heard the CTO’s from several security companies during talks at computer security tradeshows brag that they don’t use AV on their desktop computers, always “do the right thing(s),” and have never been compromised.

I think that’s a swell idea – a little contradictory and stupid if you sell AV software – but swell nonetheless.  I wish I was as attentive as these guys, but sometimes doing the right thing means you actually have to know the difference between “right” and “wrong” as it relates to the inner workings of rootkit installations.   If these experts don't do the "right thing" based upon what we hear every day (patch your systems, keep your AV up to date, run anti-spyware, etc...) what makes you think Aunty Em is going to listen?

I’ll admit, I know a thing or two about computers and security.  I try to do the “right thing” and I’ve been lucky in that I have never had any desktop machine I’ve owned compromised.  But it takes lots of technology, work, diligence, discipline, knowledge and common sense.  That’s a lot of layers. Rot Roh.

Changing gears a little…

It gets even more interesting when we see statistics that uncover that fact that 1 out of 4 Microsoft flaws are discovered by vulnerability bounty hunters – professionals paid to discover flaws! That means we’re going to see more and more of these vulnerabilities discovered because it’s good for business. Then will come the immediate exploits and the immediate patches.

Speaking of which, now that Microsoft is at the “Forefront” of the security space with their desktop security offerings, they will get to charge you for a product that protects against vulnerabilities in the operating system that you purchased – from them! Sweet! That is one bad-ass business model.

We're going to have to keep patching.  Get over it.

/Chris