Rational Survivability

Fiction Versus Function: Three Unspoken Annoynaces of Cisco & VMware's Virtualization "Partnership"

I spend a good amount of time thinking about how multiple technology strategies from market leaders coalesce into a reasonably homogenized version of reality in the networking and security space in order to decide where to place bets; it's akin to reading a tell and analyzing a player's betting strategy at a poker table.

I look at Cisco and VMware and can't help to chuckle at the moves being made in the name of "partnership" on the virtualization front and there's an awful lot of twitching going on that doesn't require Phil Hellmuth to decode. 

Partnerships are nothing new, but usually they are couched with certain modicum of suspicion and cynicism.  However, speaking with folks at VMworld, either folks were high from the Oxygen Bar at the airport, or they were adding naiveté syrup to the drinks because I seemed alone in my concerns...

I've put together a couple of summary points on the matter -- more for my own personal enjoyment and note taking than anything else -- and framed them in terms of what I find to be really annoyingly obvious examples of these two strange bedfellows' behaviors:

1)  The purported cohesion of Cisco's and VMware's virtualization strategies is a simply a matter of converged parallelism and forced perspective.

You've seen diagrams that demonstrate the notion of converged parallel lines, right?  If you haven't here's an example:

You'll notice that in this diagram there exists a series of parallel lines which seem to converge at a "vanishing point" on the horizon.

This in fact is not the case.  The lines don't actually ever converge, they just look like they do.  It's all a matter of perspective.  Imagine these lines as Cisco's and VMware's virtualization strategies.

Similarly, the notion of forced perspective is a method by which the manipulation of perspective employs an optical illusion to make something appear closer, father away, larger or smaller than it actually is (see the title image above*.)

The announcements from Cisco and VMware are very much like these examples. Whilst they offer excellent opportunities for improving the management and security of virtual infrastructure, it's very much Machiavellian marketing -- the end is going to justify the means.

Speaking to either Cisco or VMware you're asked to suspend disbelief and accept that these two companies share a common blueprint for a datacenter OS, but they don't.  In fact, they're quite different, and the balance of who needs whom more is also very lopsided.

Despite the close technical partnership needed to pull off the integration of the Nexus 1000v as the first third party virtual switch (which we've been talking about for almost two years,) Cisco and VMware really are on parallel trajectories in terms of their visions regarding the datacenter OS; how it's designed, provisioned, deployed, managed and governed...and by whom.

Cisco is approaching this primarily as an infrastructure transformation play as a way of clawing back what they lost when the network access layer become absorbed into the virtual hosts while VMware is busy distancing itself from the infrastructure and elevating the discussion to that of the cloud in an effort to stave off Microsoft and Citrix.

Each want to own your datacenter, and while they play nice on the surface, there's really a nasty game of tug of war going on here.  This is a marriage borne of convenience, nothing more.

You try and unify Cisco's DC 3.0 vision with VMware's Virtual Datacenter OS blueprint and tell me how they mesh.

2)  Dear Virtual SysAdmin: You're fired as the network admin.  You're cool with that, right?

It's funny how both Cisco and VMware's marketing folk in the sessions discussing the release of the Nexus 1000v vSwitch, both snarkily (and rhetorically) posited "How many of you Virtual SysAdmins have coordination and communication issues between your virtualization and network teams?"

Leading the witness further, the next question was  "Don't you just hate having to fight to get the network teams to give you a trunk port on an access switch?" 

They followed that up with "Your prayers are answered!  The 1000v will allow you to give the network provisioning back to the network  teams and let them control the networking and connectivity.  Isn't that great?" 

While most nodded away in the affirmative to the first and second questions, I didn't see one audience member who answered positively on the latter.  What makes anyone think the vSysAdmins *want* to give up the control of the virtual networking layer and be at the mercy of the networking teams again?

Interesting battle ground for sure.  Now, please don't misinterpret my commentary as a suggestion that this is a bad thing, but we're already in the middle of a "West Side Story" turf war over organizational fiefdoms.  This will, depending upon what sort of contention exists already, make a really tenuous issue even more so.

3) Software Sucks.  Hardware Rules.   I hope you like ping pong.

I hinted at this point in my post titled (The Network is the Computer...)  The reality is that much like point #1, Cisco could care less in the long term about the Nexus 1000v as a software switch running in someone else's backyard operating environment, but rather introduces it to enable the landscape clawback it gets to enjoy in the short term and make relevant once again it's big network iron in the longer timeframe.

A telling slide was the announcement of what's coming AFTER the Nexus 1000v in one of the sessions that I have not seen presented in detail elsewhere -- that is Cisco's goal to extract networking out of the host completely.

The plan as discussed is to utilize what Cisco calls an "initiator" to replace the 1000v and force traffic, after specialized tagging which denotes affinity of flows to specific VM ID's, and ship them straight back out the network interfaces to a waiting Cisco 5000/7000 switch for processing.  Hence the ping-pong mention above.

Sorry for the quality of the picture as I took it sitting behind somebody, but here's a slide denoting just this very thing:
 
The notion of a third party switching capability is really just a way for Cisco to push the access layer back to where they think it rightfully belongs -- in the physical switch.

Cisco claims that VMware and they have submitted this tagging specification to the IEEE for review/ratification.   I find that very interesting.

I wrote about the need for such a technology at both the virtualization layer and more importantly the application/data level in June of 2007. 

Check out my post which described how I suggested Crossbeam do the exact same thing by way of something I called ADAPT (Applied Data and Application Policy Tagging) which describes this very thing.  What's next, they're going to announce vNAC? ;)

--

All in all, the Cisco/VMware relationship is about as natural looking as the Microsoft/Citrix version -- it's sort of like a midget dating a six foot supermodel...someone's getting the better end of the footrub in that relationship, too.

So, how about it?  Am I stating the obvious again -- and does it need to be stated?

/Hoff

*image from "The Eye of Brad" flickrstream

IDS: Vitamins Or Prophylactic?

Ravi Char commented on Alan Shimel's blog titled "IDS - The Beast That Just Won't Die."

Ravi makes a number of interesting comments in his blog titled "IDS/IPS - is it Vitamins?"  I'd like to address them because they offer what I maintain is a disturbing perspective on the state and implementation of IDS today by referencing deployment models and technology baselines from about 2001 that don't reflect reality based on my personal experience.

Ravi doesn't allow comments for this blog, so I thought I'd respond here.

Firstly, I'm not what I would consider and IDS apologist, but I do see the value in being able to monitor and detect things of note as they traverse my network.  In order to "prevent" you first must be able to "detect" so the notion one can have one without the other doesn't sound very realistic to me.

Honestly, I'd like to understand what commercial stand-alone IDS-only solutions Ravi is referring to.  Most IDS functions are built into larger suites of IDP products and include technology such as correlation engines, vulnerability assessment, behavioral anomaly, etc., so calling out IDS as a failure when in reality it represents the basis of many products today is nonsensical.

If a customer were to deploy an IPS in-line or out-of-band in "IDS" mode and not turn on automated blocking or all of the detection or prevention capabilities, is it fair to simply generalize that an entire suite of solutions is useless because of how someone chooses to deploy it?

I've personally deployed Snort, Sourcefire (with RNA,) ISS, Dragon, TippingPoint, TopLayer and numerous other UTM and IDP toolsets and the detection, visibility, reporting, correlation, alerts, forensic and (gasp!) preventative capabilities these systems offered were in-line with my expectations for deploying them in the first place.

IDS can capture tons of intrusion events, there is so much of don't care events it is difficult to single out event such as zero day event in the midst of such noise. 

Yes, IDS can capture tons of events.  You'll notice I didn't say "intrusion events" because in order to quantify an event as an "intrusion," one would obviously have already defined it as such in the IDS, thus proving the efficacy of the product in the first place.  This statement is contradictory on face value.

The operator's decision to not appropriately "tune" the things he or she is interested in viewing or worse yet not investing in a product that allows one to do so is a "trouble between the headsets" issue and not a generic solution-space one.  Trust me when I tell you there are plenty of competent IDS/IPS systems available that make this sort of decision making palatable and easy.

To take a note from a friend of mine, the notion of "false positives" in IDS systems seems a little silly.  If you're notified that traffic has generated an alert based upon a rule/signature/trigger you defined as noteworthy, how is that a false positive?  The product has done just what you asked it to do!  Tuning IDS/IPS systems requires context and an understanding of what you're trying to protect, from where, from whom, and why.

People looking for self-regulating solutions are going to get exactly what they deserve.

Secondarily, if an attacker is actually using an exploit against a zero-day vulnerability, how would *any* system be able to detect it based on the very description of a zero-day?

It requires tremendous effort to sift through the log and derive meaningful actions out of the log entries.

I again remind the reader that manually sifting through "logs" and "log entries" is a rather outdated concept and sounds more like forensics and log analysis than it does network IDS, especially given today's solutions replete with dashboards, graphical visualization tools, etc.

IDS needs a dedicated administrator to manage. An administrator who won't get bored of looking at all the packets and patterns, a truly boring job for a security engineer. Probably this job would interest a geekier person and geeks tend to their own interesting research!

I suppose that this sort of activity is not for everyone, but again, this isn't Intrusion Detection, Shadow Style (which I took at SANS 1999 with Northcutt) where we're reading through TCPDUMP traces line by line...

There are companies that do without IDS, and they do just fine. I agree with Alan's assessment that IDS is like a Checkbox in most cases.  Business can run without IDS just fine, why invest in such a technology?

I am convinced that there are many companies that "do without IDS."  By that token, there are many companies that can do without firewalls, IPS, UTM, AV, DLP, NAC, etc., until the day they realize they can't or realize that to be even minimally compliant with any sort of regulation and/or "best practice" (a concept I don't personally care for,) they cannot.

I'm really, really interested in understanding how it is that these companies "...do just fine."  Against what baseline and using what metrics does Ravi establish this claim.  If they don't have IDS and don't detect potential "intrusions," then how exactly can it be determined that they are better off (or just as good) without IDS?

Firewalls and other devices have built in features of IDS, so why invest in a separate product.

So I misunderstood the point here?  It's not that IDS is useless, it's just that standalone IDS deployments from 2001 are useless but if they're "good enough" functions bundled with firewalls or "other devices," they have some worth because it costs less?

I can't figure out whether the chief complaint here is an efficacy or ill-fated ROI exercise.

IDS is like Vitamins, nice to have, not having won't kill you in most cases. Customers are willing to pay for Pain Killers because they have to address their pain right away. For Vitamins, they can wait. Stop and think for moment, without Anti-virus product, businesses can't run for few days. But, without IDS, most businesses can run just fine and I base it out of my own experience.

...and it's interesting to note the difference between chronic and acute pain.  In many cases, customers don't notice chronic pain -- it's always there and they adjust their tolerance thresholds accordingly.  It may not kill you suddenly, but over time the same might not be true.  If you don't ingest vitamins in some form, you will die.

When an acute pain strikes, people obviously notice and seek immediate amelioration, but the pain killer option is really just a stop-gap, it doesn't "cure" it simply masks the pain. 

By this definition it's already too late for "prevention" because the illness has already set in and detection is self-evident -- you're ill.  Take your vitamins, however, and perhaps you won't get sick in the first place...

The investment strategy in IDS is different than that of AV.  You're addressing different problems, so comparing them equliaterally is both unfair and confusing.  In the long term, if you don't know what "normal" looks like -- a metric or trending you can certainly gain from IDS/IPS/IDP systems -- how will you determine what "abnormal" looks like?

So, sure, just like the vitamin example -- not having IDS may not "kill" you in the short term, but it can certainly grind you down over the long haul.

Probably, I would have offended folks from the IDS camp. I have a good friend who is a founder of an IDS company, I am sure he will react differently if he reads my narratives about IDS.  Once businesses start realizing that IDS is a Checkbox, they will scale down their investments in this area. In the current economic climate, financial institutions are not doing well. Financial institutions are big customers in terms of security products, with the current scenario of financial meltdown, they would scale down heavily on their spending on Vitamins.

Again, if you're suggesting that stand-alone IDS systems are not a smart investment and that the functionality will commoditize into larger suites over time but the FUNCTION is useful an necessary, then we agree.

If you're suggesting that IDS is simply not worthwhile, then we're in violent disagreement.

Running IDS software on VMware sounds fancy.  Technology does not matter unless you can address real world pain and prove the utilitarian value of such a technology. I am really surprised that IDS continues to exist. Proof of existence does not forebode great future. Running IDS on VMware does not make it any more utilitarian. I see a bleak future for IDS.

Running IDS in a virtual machine (whether it's on top of Xen, VMware or Hyper-V) isn't all that "fancy" but the need for visibility into virtual environments is great when the virtual networking obfuscates what can be seen from tradiational network and host-based IDS/IPS systems. 

You see a bleak future for IDS?  I see just as many opportunites for the benefits it offers -- it will just be packaged differently as it evolves -- just like it has over the last 5+ years.

/Hoff

VMWare's VirtSec Vision...Virtual Validation?

Rich Miller from one of my favorite blogs, Telematique, wrapped up his VMworld 2008 coverage in a post titled "What Happens In Vegas..."

Rich made a number of very interesting observations, one of which caught my eye:

The theme I noted most at VMworld 2007 a year ago was "security."  This year, it seemed noticeably absent.  My sense is that the industry has yet to catch up and capitalize on VMsafe. Because all of the "next generation" of offerings from VMware and the independent providers are still in development, no one made too much of security issues.

I'd agree with Rich to a point, except for the part about how "no one made too much of security issues..."
There was at least one schmuck waving his arms and I'm wearing his underwear ;)

As I alluded to in my post titled "VMWorld 2008: Forecast For VMware?  Cloudy...Weep For Security?" the messaging associated with this years re-branding certainly had the word "security" front and center under the Application vServices pillar:

...but as Rich alluded to and my post made mention, the bulk of VMware's security efforts would appear to focus primarily on what's coming from VMsafe and the ecosystem partners supporting it:

I think, however, this would be a rather short-sighted perspective.

While the tremendous amount of re-branding and messaging certainly "clouded" the horizon in some cases, it was clear that the underlying technology roadmap that was shown (and demonstrated as being real in the labs I participated in) lays down some pretty significant clues as to what's in store from VMware in regards to security.  More on that in a minute...

You might recall my debate with Citrix's CTO, Simon Crosby, on what, where and how virtualization platform providers should invest in terms of security.  Our divergent views were quite passionate, but I maintain that my perspective is that echoed by the majority of the folks I've spoken to over the last 3 years.

My position, presented in this post, is best summarized thusly:

"...that it is the responsibility of virtualization platform providers to ensure that their [virtualized] data center operating system platforms of the future" don't become the next generation of insecure infrastructure. 
...

It's important to understand that I'm not suggesting that virtualization platform providers should secure the actual guest operating systems but they should enable an easier and more effective way of doing so when virtualized.

I mean that the virtualization platform providers should ensure the security of the instantiation of those guests as "hosted" by the virtualization platform.  In some cases this means leveraging technology present in the virtualization platform to do things that non-virtualized instances cannot. That's more than just securing the hypervisor."

What did I mean by that?

Obviously since I was using VMware as the model for my position, I was referring to VMsafe as the ecosystem enabler, but I was also referring to VMware's prior security acquisition in 2007 of Determina as a fundamental building block for the virtualization platform's security efforts.

Pair that with the root of VM/VMM introspection that Mendel and Tal Garfinkel envisioned back in the day, and things start to make sense...

So back to the original point of this post, which is where VMware's efforts are focused in terms of security.  I think that this year's show concreted a two definites in my book:

1. VMware will make additional acquitisions in the security space.  Yes, I know this sounds heretical given the delicate balance most "platform" providers keep with their ecosystem partners, but VMware have already shown that they are ready to buy as well as build and ally with prior acquisitions and security will continue to be a key differentiator for them.  They've done it once already with Determina, they'll do it again.
    
2. The level of commitment and investment shown by both Cisco and VMware shows there will be a continuing deeply-concreted technology integration between the two beyond the Nexus 1000v.  The software instantiation of the Nexus functionality is a fantastic story when combined with the larger vNetwork/distributed virtual switch approach, but I maintain it's a stop-gap. 
   
   The real picture (for Cisco) is clearly a thinner software layer within ESX and a further dependence upon switching hardware external to the host (the Nexus 5000/7000) with of course more security capabilities being provided by the external iron.  Why?  Read my Four Horsemen presentation: virtual switching and virtual security appliances -- both in software -- ultimately don't scale, don't perform and don't play well with the rest of the stack.
   
   What does that have to do with VMware's strategy?  Well, to me it's an indication that while virtual appliances are great for many things, security isn't necessarily one of them and that the platform has to be bolstered with as much security huevos as makes sense, allowing the commodity functionality to be shipped off to ISV's software.  I mean, that's why the stuff demo'd by the ISV's looks just like existing solutions -- with VirtualCenter integration.  A firewall's still a firewall, AV's still AV...the model changes a bit, so back we go to the network.
   
   Remember this post?: Security Will Not End Up In the Network... It sounds a little contradictory to the point I just made, but read the post.  For the sake of this one, it simply illustrates that cyclically, we're swinging the pendulum back to the "network."  However, where the "network" is becomes a little nebulous.
   
   More on this concept in a separate post soon.

What this ultimately means to me is that within the next 24 months with the delivery of VI4, a mature VMsafe API and shipping ISV code, we'll see some of the natural market consolidation activity occur and VMware will lock and load, snap up one or more of the emerging security players in the VirtSec space and bolster their platform's security capabilities.

Meanwhile Cisco will help secure VMware further in the enterprise with their integrated play and the remaining security ecosystem players will begrudgingly fight to stay on the good side of the fence...while they hedge their bets by supporting Microsoft and Hyper-V.

I think we've just seen the beginning of what VMware has to offer from a security perspective.  They (and us) have a tremendously long way to go, but every cloud's got a silver lining...

/Hoff

The Network Is the Computer...(Is the Network, Is the Computer...)

If there's one motif emerging from VMworld this year, it's very much the maxim "be careful what you ask for, because you might just get it."

The precipitate convergence of virtualized compute, network and storage is really beginning to take significant form; after five hard years of hungering for the materialization of the technology, enterprise architecture, and market/business readiness, the notion of a virtualized datacenter OS with a value proposition larger than just " cost-optimized infrastructure" has now become a deliciously palpable reality.  

The overlap and collision of many leading technology providers' "next generation" datacenter (OS) blueprints is something I have written about before.  In many cases there's reasonable alignment between the overall messaging and promised end result, but the proof is in the implementation pudding. I'm not going to rehash this here because I instead want to pick on something I've been talking about for quite some time.

From a network and security perspective, things are about to (again) fundamentally and profoundly change in terms of how we operationally design, provision, orchestrate, manage, govern and secure our infrastructure, applications and information.  It's important to realize that this goes way beyond just  adding a 'v' to the name of a product. 

What's incredibly interesting is the definition and context of where and what makes up the "network" that transports all our bits and how the resources and transports interact to deliver them securely.

It should be clear that even in a homogenous platform deployment, there exists an overwhelming complex conglomerate of mechanisms that make up the machinery enabling virtualization today.  I think it's fair to say that we're having a difficult time dealing with the non-virtualized model we have today.   Virtualization?  She's a cruel mistress bent on reminding us we've yet to take out the trash as promised.

I'm going to use this post to highlight just how "complexly simple" virtual networking and security have become using as an example the last two days' worth of  announcements, initiatives and demonstrations of technology and solutions from Intel, VMware, Cisco and the dozens of security ISV's we know and love.

Each of the bumps in these virtual wires deserves its own post, which they are going to get, especially VMware's vNetwork/VMsafe, distributed network switch, and Cisco's Nexus 1000v virtual switch announcements.  I'm going to break each of these elemental functions down in much more detail later as they are simply amazing.

Now that networking is abstracted across almost every layer of this model and in many cases managed by separate organizational siloes and technologies, how on earth are we going to instantiate a security policy that is consistent across all strata?  We're used to this problem today in physical appliances, but the isolation and well-definable goesinta/goesouta perimeterized boundaries allows us to easily draw lines around where these

policy differentials intersect. 

 It's used to be the devil you knew.  Now it's eleven different devils in disguise.

As you visualize the model below and examine how it applies to your experience, I challenge you to tell me where the "network" lives in this stack and how,  at a minimum, you think you're going to secure it.   This is where all those vendor roadmaps that are colliding and intersecting start to look like a hodgepodge:

In the example model I show here, any one of these elements -- potentially present in a single VMware ESX host -- can directly or indirectly instantiate a set of networking or security functions and policies that are discrete from one another's purview but ultimately interrelated or even dependent in ways and using  methods we've not enjoyed before.  

In many cases, these layered components are abstracted from one another and managed by separate groups.  We're seeing the re-emergence of  network-centricity, it's just that the  network is camouflaged in all its cloudy goodness.  This isn't a story where we talk about clearly demarcated hosts  that plug into "THE" network, regardless of whether there's a hypervisor in the picture.

 

Here's where it gets fun...

In this model you have agents in the Guest OS interacting with security/networking virtual appliances on the ESX host either inline or via vnetworking APIs (switching or security) which in turn uses a fastpath networking kernel driver  connected to VMware's vSwitch while another VA/VM is connected to a Cisco Nexus 1000v vSwitch implemented as a second distributed virtual network switching fabric which are all running atop an Intel CPU utilizing SR-IOV via VT-d in the chipset which in turn allows VM's to direct attach (bypassing the VMM) to NIC cards with embedded switching connected to your network/storage fabrics...

Mass hysteria, cats and dogs living together...

So I'll ask you again: "Where's the network in that picture?"  Or, more precisely, "where isn't it?"  

This so hugely profound, but that may because I've been exposed to each of the bubbles in this diagram and see how each of them relate or do not.  When you step back and look at how, as an example, Cisco and VMware are both going through strategic sea changes in how they are thinking about networking and security, it's truly 

amazing but I think the notion of network intelligence is a little less cut and dry as some might have us believe.

Is this as mind-blowing to you as it is to me?  If not, wait until I rip open the whole vNetworking and Nexus 1000v stuff.  Very, very cool.

/Hoff

VMWorld 2008: Forecast For VMware? Cloudy...Weep For Security?

This post was written prior to the opening of the Partner Day/Technology Exchange, based solely upon information that is publicly available.  No NDA's were harmed during the making of this blog...

So now that I can talk about it outside of the embargo, VMware is announcing extensions to its product roadmap and product marketing to deliver what it calls its "virtual datacenter OS:"

VMware's comprehensive roadmap of groundbreaking new products expand its flagship VMware Infrastructure suite into a virtual datacenter OS. The virtual datacenter OS addresses customers’ needs for flexibility, speed, resiliency and efficiency by transforming the datacenter into an “internal cloud” – an elastic, shared, self- managing and self-healing utility that can federate with external clouds of computing capacity freeing IT from the constraints of static hardware-mapped applications. The virtual datacenter OS guarantees appropriate levels of availability, security and scalability to all applications independent of hardware and location. 

The components that make up the VMware's virtual datacenter OS are:

• Application vServices guarantee the appropriate levels of availability, security and scalability to all applications independent of hardware and location.
• Infrastructure vServices subtract, aggregate and allocate on-premise servers, storage and network for maximum infrastructure efficiency.
• Cloud vServices federate the on-premise infrastructure with third party cloud infrastructure.
• Management vServices allow you to proactively manage the virtual datacenter OS and the applications running on it.

Each of these components have service/product definitions below them.  

While it's exciting to see VMware's strategy around its version of the datacenter OS, it's going to be a bumpy ride as we continue to see how Microsoft, Cisco and VMware all interact and how these roadmaps align -- or don't. 

Remember, despite how they play nice, each has their own bottom line to watch and it's every man for himself.

It's quite clear we're going to have some very interesting security challenges bubbling up to the surface; we barely have our arms around what we might call virtualization v1.0 -- we've a lack of maturity in solutions, operations, visibility and security and we're pulling the trigger on what's sure to be a very contentious security model...or lack thereof. 

In the vApplication services, there is a direct call-out titled "Security" in which VMware's ESX 3i's size is touted as it's current security feature (rolleyes) and in 2009 we see the following:

• VMware VMsafe provides x-ray visibility into virtual machine resources from the vantage point of the hypervisor, making it possible to monitor every aspect of the execution of the system and stop previously undetectable viruses, rootkits and malware before they can infect a system

• Checkpoint, IBM, McAfee, Radware, TrendMicro and are announcing their plans to deliver VMSafe –integrated products in 2009 that provide superior protection to virtual machines than possible with physical machines or other virtualization solutions

There's nothing new here, except the dependence upon VMsafe, ISV's and virtual appliances...I think you know how I feel about that.

In line with my posts regarding the Cisco vSwitch for ESX (what I'm calling the cSwitch,) the "Infrastructure vServices" component hints at the development of three major investment points: vCompute, vNetwork and vStorage.  

In vNetwork, you'll notice the 2009 arrival of the following three elements which are very interesting, indeed:

• Distributed Switch simplifies the setup and change of virtual machine networking
• Network VMotion enables network statistics and history to travel with a virtual machine as it moves from host to host for better monitoring and security
• Third party virtual switches plug into virtual networks and deliver value added network monitoring, security and QoS

I'll be interested to see what distributed networking actually means -- there's a session today on that, but coupled with the cSwitch, I wonder if it means more than just plugging into virtualcenter/VFrame for management.

Let's not forget how some of the elements in vCompute will effect networking and security such as VMDirect which provides "intelligent" VMM bypass and allow direct access from the VM's...all in the name of performance.  I wrote about that here a couple of days ago.

It looks as though we might see some policy extensions to afford affinity such that policies travel with the VM!?

The notion of vCloud is being desrbied as the notion of portability, mobility and supportability of applications that can be developed and deployed inside an enterprises' "internal cloud" and then handed off to an "external cloud" providers service offerings.  It's really the "infrastructureless infrastructure" play. 

One thing that immediately comes to mind when I hear words "federation" -- as I assume it might to any security professionals ears -- is the issues surrounding exposure of AAA (authentication, authorization and accounting) between internal and external credential stores and how this intersects with SOA environments.

As more details come to light, I'll be adding my thoughts about where (if at all) security really plays into this evolving strategy.

Gotta shower and get to the con.

/Hoff

VMWorld 2008: "Introducing Cisco's Virtual Switch for VMware ESX"...

 

Update below.

It's the night before VMworld 2008 and the Technology Exchange/Partner day begins and I'm pawing through the stuff in my bag, separating the "keep it" from the "toss it" schwag.

There's an innocuous little flyer stuffed in the bag on Cisco letterhead titled "Introducing Cisco's Virtual Switch for VMware ESX."  Fantastic.  Let's call it the 'cSwitch' ;)

A year and a month ago in August of 2007, I blogged about this very thing in a post titled: "VMware to Open Development of ESX Virtual Switches to Third Parties...Any Guess Who's First?" based on a hint from virtualization.info.

Given that VMworld 2007 came and went without this announcement, I'm very excited that we're actually going to get a look at what Cisco will offer; I think this is huge news and ultimately offers some profound game-changing (for good and bad) implications on the network and security fronts.

I have dozens of questions like: I wonder how much of the Nexus (7000 series)/NX-OS code cross-pollinates over (if any) to this solution and if we'll see capabilities such as STP/PVST+/Private VLANs, HSRP, Multicast, etc. make their way into Cisco's vSwitch and how this virtual switch with integrate/interoperate with the vkernel.

Further, as Ed Haletky and I unofficially bet over drinks this evening, I wonder if it will be a direct replacement for VMware's at-boot loadable module or it will co-exist?  I bet the former. ;)

In addition to the "cSwitch," there are a couple of sessions I am very, very interested in attending given my exposure to VFrame and some Cisco engineers/architects at last year's show:

Simplify VMotion with Virtual Machine–Aware Network and Storage Services
See how network and storage services can be linked to a virtual machine so they move with VMotion events.

ESX Server in a Unified Fabric Environment
See how ESX Server works in a unified fabric environment with ESX 3.5 U2, Emulex Converged Network Adaptors, and the Cisco Nexus 5000.

VFrame: Enriching ESX Deployment with End-to-End Orchestration
Cisco’s VFrame DC 1.2 provides an easy-to-use template-based provisioning approach for rapid, repeatable, and compliant provisioning of ESX Servers. Through a rich set of networking and storage orchestration capabilities, it reduces the time required to bring up ESX clusters while providing operational scalability to manage large clusters effectively.

See the second topic above?  Remember when I mentioned in prior posts about virtualizing applications directly within the Nexus?

Should be a very interesting couple of days.

/Hoff

Update: So there was no direct news/mention specifically of Cisco today in any of the distributed virtual networking (DVN) sessions -- there's a lot of messaging collisions because the re-branded 'v-everything' strategy has things being renamed.  Hopefully we'll see/hear more from Cisco tomorrow.

Many of the underlying functions that will enable 3rd party virtual switches as well as any network interface to the vkernel via API were discussed today under the capabilities described by vNetwork (this includes the vNetwork Appliance API's and what you've known as VMsafe.)  You can see more about vNetwork here in this post.

All I can say is that I got a lot of my suspicions confirmed, questions answered and conclusions affirmed in today's sessions.  Some good, some bad.  It's going to be a bumpy ride, kids.

The Four Horsemen live! ;)

Travel: Off to VMworld 2008 in Vegas

 

This coming week I'm off to VMworld in Las Vegas for a week of virtual immersion.

I'm looking forward to meeting with fellow practitioners, analysts, folks from VMware, Microsoft, Citrix and vendors as well as attending many of the sessions and labs.

There are several meet-ups including community get-togethers, so if you're in town, ping me and let's get together and exchange ideas.

There's a bunch of anticipated high-profile announcements and I hope some of them pan out.  I'll be live-blogging/tweeting (beaker) as much as possible from the show.

See you there.

/Hoff

Direct Attach Hardware/VMM Bypass Technologies Are a Security Anathema...

I'm going to clean this post up later, but I just gave a presentation that was an extension of my Four Horsemen talk.

One of the new topics that I'm speaking about is how technologies in chipset/CPU technologies that allow for VMM bypass, direct VM to hardware attachment as well as cramming virtual switching back down into the CPU/NIC are the devil...from a security perspective.

If you haven't seen technology examples such as SR-IOV (single root IO virtualization) -- Intel VT-d extensions --and CPU extensions in upcoming processors such as Nehalem that allow specific VM's to bypass the VMM and directly attach/access hardware for the sake of performance, get ready for some seriously nasty security and networking implications.

Technology like SR-IOV and VMDirectPath are great for performance and scalability (if you're not already using paravirtualization) but provide the archnitectural antithesis from the perspective of mobility and security visibility.  If you bypass the VMM, you lose mobility and the virtual security appliances/security capabilities that might be present in the VMM disappear...

Further, given the recent exploration of abuse of DMA as an attack vector, it's a little disturbing.

I'll expand more later, but here's a slide that sums up the introduction to this issue:

 

Virtualization (In)Security Through Economic Obfuscation...

Here's something that's bothered me for some time...

To date, the bulk of security research focused on the many elements of virtualization are clearly focused on open-source based virtualization platforms/hypervisors such as Xen or other widely-available hosted hypervisor architectures and not on closed-source, commercially-available offerings.

The reason for this is two-fold: cost and availability of source code.*

It's clear that it costs nothing and is easier to find vulnerabilities in free and freely-available source code or free/low-cost hosted platforms.

So, assuming that VMware enjoys 50%+ marketshare in the (server) virtualization space, it's frightening that perhaps the reason we don't see a lot of research and/or publically-available review of security in VMware comes down to the first stumbling block that puchasing ESX is simply not economically viable for many researchers.  So they don't.

This comment has been made by several top-rung researchers over the last 3 days in a VirtSec conference I'm attending.

Security through economic obfuscation.

Ugh.

It will be interesting to see what happens as the cost of commercial hypervisors approach zero even if they are closed sourced.  As ESX and Hyper-V are now basially free, I look forward to seeing work done by the researchers in this space as they turn their attention to it now that it's affordable.

/Hoff

*Oh, yeah...there's that little issue of NDA/legal, too...if you happen to directly engage with said vendors who can therefore limit disclosure.

Big Iron is Dead...Long Live Big Iron?

Over the last 5 years or so with the pronounced emergence of inexpensive multi-core COTS compute platforms and the rabid evangelism of virtualization, I have debated many times with folks who continue to suggest that "big iron" -- high performance equipment in the compute, network, security and storage domains -- is now "extinct" and that nobody buys bespoke equipment any more.

Many of them argued "All you need is a COTS PC, package up OS/Applications and voila!  Instant appliance!  Ready for the edge, ready for the core."

Not surprisingly, many of the networking/security/application delivery companies that these folks worked for ultimately introduced custom-engineered hardware solutions, melding their software with custom hardware and COTS elements to produce a piece of big iron of their own...

About a year ago, I wrote a blog on this topic highlighted by a post titled "All your COTS multicore CPU's with non-optimized security software are belong to us," in which I extrapolated some very interesting points regarding Moore's law and the hubris surrounding the gluttony of compute power offered by higher density chipsets without the corresponding software architecture to take advantage of it.

Update: I forgot that I wrote a really good post (this March!) on this same topic titled "I Love the Smell Of Big Iron In the Morning..."

This is a little tickler to that post.

I come from the land of servicing large enterprises, service providers, municipalities and nation states  and not the SME. 

While there are certainly exceptions to the "rule," and it's reasonable to suggest that my perspective is skewed, I've always been careful to ensure I framed my discussions this way, so debating/contrasting the architectural slants of an SME with a Fortune 10 doesn't really move the discussion along any.

So, sticking with the large enterprise theme, there are two interesting divergent themes emerging: the centralization of compute and storage with the distributed nature of connectivity and information.

Without muddying the water too much about how these scenarios are not all that mutually exclusive, let's stick with the "centralization" theme for a moment.

The mainstream adoption of virtualization as an enabler brings us full-circle back around to the centralized mainframe model* of compute, networking and storage.  Now that reasonably reliable, high speed and low latency connectivity is available, centralization of resources makes sense since people can generally get access to the assets they require and the performance to get from point A to point B is for the most part acceptable (and getting more so.)

Once again, the natural consolidation of functionality and platform architecture follows suit -- we're back to using big iron to deliver the balance of cost, resilience, performance and simplified management that goes along with squeezing more from less.

To wit, let's examine some recent virtualization-inspired, big iron plays:

Compute:

HP introduced [recently] the Proliant BL495c G5, the first blade designed to be a virtual machine-intensive host in a data center rack, along with other virtualization products to enhance the offerings of VMware and Citrix Systems.

As system administrators achieve savings by consolidating 8-10 virtual machines per server, HP is saying its new blade will host up to 32 virtual machines, based on each virtual machine needing a minimum of four Gigabytes of memory. When 16 of the blades are stacked in an HP C7000 enclosure, a total of 512 virtual machines can be run from a single rack, said Jim Ganthier, director of HP BladeSystem, its blade server division.

Network & Storage:

The Cisco Nexus 7000 Series is the flagship member of the Cisco Nexus Family, the first in a new data center class of switching products. The Nexus 7000 is a highly scalable modular platform that delivers up to 15 terabits per second of switching capacity in a single chassis, supporting up to 512 10-gigabits-per-second (Gbps) Ethernet and future delivery of 40- and 100-Gbps Ethernet. Its unified fabric architecture combines Ethernet and storage capabilities into a single platform, designed to provide all servers with access to all network and storage resources. This enables data center consolidation and virtualization. Key components of the unified fabric architecture include unified I/O interfaces and Fibre Channel over Ethernet support to be delivered in the future.

Security:

The Crossbeam X-Series is a carrier-class modular security services switch.  The X80 platform is Crossbeam's flagship high-end security services platform for complete network, security.  The X80 provides up to 40 Gigabit Ethernet ports and 8 x 10 Gigabit Ethernet ports or up to 64 Fast Ethernet ports and up to 40 Gbps of full duplex firewall throughput.  Each network processor module features an integrated 16-core MIPS-64 security processor, a high speed network processor and a custom-designed switch fabric FPGA.  The X80 supports up to 10 application processor modules which are based on Intel technology running a hardened version of Linux supporting best-in-breed security software from leading vendors in highly resilient, load-balanced configurations.

These are just a few examples.

Each of these solutions delivers the benefits of hefty amounts of virtualized service, but what you'll notice is that they do so in massive scale, offering consolidated services in high-density "big iron" configurations for scale, resilience, manageability and performance.

In this period of the technology maturity cycle, we'll see this trend continue until we hit a plateau or an inflection point -- whether it is triggered due to throughput, power, heat, latency, density or whatnot.  Then we'll start anew and squeeze the balloon once more, perhaps given what I hinted at above with clusters of clouds that define an amporphous hive of virtualized big iron.

But for now, until service levels, reliability, coherence and governance are sorted, I reckon we'll see more big iron flexing it's muscle in the data center.

What about you?  Are you seeing the return of big iron in your large enterprise.  Perhaps it never left?

I for one welcome my new blinking dark overlord...

/Hoff

* There's even a resurgence of the mainframe itself lately.  See IBM's z/10 and Unisys' ClearPath for example.

Upcoming Presentation: The Frogs Who Desired A King: A Virtualization Security Fable Set To Interpretive Dance

The sequel to the "Four Horsemen of the Virtualization Security Apocalypse," is my next presentation entitled "The Frogs Who Desired A King: A Virtualization Security Fable Set To Interpretive Dance."

It goes something like this:

Aesop wrote a little ditty about some discontented frogs who lived in a pond.  They asked Jupiter for a King.  They got one.  They didn't like it.  They got a replacement. It ate them.  The moral of this story is "be careful what you wish for."

The corresponding analog is that of the future state of security in a virtualized world.  It's coming, but it's not going to look much like what security looks like today and it's certainly not what people are expecting.  In fact,it may consume us all because we're actually unprepared for what we're asking for.

You'll laugh, you'll cry.  You'll want to know what I used to make my slides... ;)

Coming soon to a disturbed audience near you (seriously.)

/Hoff

The Most Overused Term In Security Product Management/Marketing...

Next Generation <anything>

Sick of it.  Sucks monkey balls.  Is about as relevant and non-sensical to me as "kosher ham."

I've been really annoyed by this term since I ashamedly added it to my lexicon of "roll-off-the-tip-of-my-tongue" buzzwords years ago for reasons I can't rightly remember.  Too much TV.

I suppose temporally, anything not shipping, regardless of how (r)evolutionary it may or may not be, is technically "next generation," but it's today overly (ab)used to imply some quantum leap in capability, functionality, or saleability.  Oh, and one usually has to pay more for it.

The truth is -- and as I pointed out in my disruptive innovation presentations -- there just aren't that many "big bangs" that deserve to have this moniker hung upon the mantle, but rather a series of dampened oscillations due to punctuated equilibrium until everything settles down and looks pretty much the same.

Then version 1.17 ships and BAM!  Next generation, baby!

To all you product managers and marketers, "next generation" is so over-played at this point that the populous at large simply regards it like the features lists plastered on the trunk lids of automobiles advertising the niftiest new (but abundantly standard) set of features purchased on the luxo-barge meandering about in the lane ahead.

Whilst I am happy to know that Bob got the GLX, limited edition, R-Series with ABS, sunroof, intercooled turbo with XM radio and AWD, the suggestion that his "seats 8 but still makes him look like a dork" mini-van is a "next generation" platform doesn't really say much about Bob, now does it?

--

On the flip side, I'm just thrilled to learn via press release today that "Secure Computing [is] to acquire Securify to drive [its] next generation firewalls" which oddly enough includes a list of features that are aimed squarely at competing with folks like Palo Alto Networks'* "next generation" firewalls which were released sometime ago. 

Further, someone at PAN and Secure Computing will undoubtedly be shocked to learn that Crossbeam, Fortinet, and Cisco all have "next generation firewalls" too.  Crap!  What comes after "next generation?" 

I suppose whatever it is would have to be made of pure unobtanium...

I knew I should have trademarked that...

/Hoff

* Speaking of Palo Alto Networks, you may have missed that a couple of weeks ago, PAN secured a C-Round of $27M.  That ought to be good for a couple more 'next generations' of something...they also finally got a new CEO back in July (Lane Bess from Trend Micro.)

Google's Chrome: We Got {Secure?} Browsing Bling, Yo.

From the Department of "Oops, I did it again..."

Back in June/July of 2007, I went on a little rant across several blog posts about how Google was directly entering the "security" business and would eventually begin to offer more than just "secure" search functions, but instead the functional equivalent of "clean pipes" or what has now become popularized as safe "cloud computing."

I called it S^2aaS (Secure Software as a Service) ;)  OK, so I'm not in marketing.

Besides the numerous initiatives by Google focused on adding more "security" to their primary business (search) the acquisition of GreenBorder really piqued my interest.   Then came the Postini buyout.

To be honest, I just thought this was common sense and fit what I understood was the longer term business model of Google.  To me it was writing on the wall.  To others, it was just me rambling.

So in my post from last year titled "Tell Me Again How Google Isn't Entering the Security Market?  GooglePOPs will Bring Clean Pipes..." I suggested the following:

In fact, I reckon that in the long term we'll see the evolution of the Google Toolbar morph into a much more intelligent and rich client-side security application proxy service whereby Google actually utilizes client-side security of the Toolbar paired with the GreenBorder browsing environment and tunnel/proxy all outgoing requests to GooglePOPs.
...
Google will, in fact, become a monster ASP.  Note that I said ASP and not ISP.  ISP is a commoditized function.  Serving applications and content as close to the user as possible is fantastic.  So pair all the client side goodness with security functions AND add GoogleApps and you've got what amounts to a thin client version of the Internet.

Now we see what Google's been up to with their announcement of Chrome (great writeup here,) which is their foray into the Browser market with an open source model with heaps of claimed security and privacy functions built in.  But it's the bigger picture that's really telling.

Hullo!  This isn't about the browser market!  It's about the transition of how we're going to experience accessing our information; from where, what and how.  Chrome is simply an illustration of a means to an end.

Take what I said above and pair it with what they say below...I don't think we're that far off, folks...

From Google's Blog explaining Chrome:

...we began seriously thinking about what kind of browser could exist if we started from scratch and built on the best elements out there. We realized that the web had evolved from mainly simple text pages to rich, interactive applications and that we needed to completely rethink the browser. What we really needed was not just a browser, but also a modern platform for web pages and applications, and that's what we set out to build.

...

Under the hood, we were able to build the foundation of a browser that runs today's complex web applications much better. By keeping each tab in an isolated "sandbox", we were able to prevent one tab from crashing another and provide improved protection from rogue sites. We improved speed and responsiveness across the board. We also built a more powerful JavaScript engine, V8, to power the next generation of web applications that aren't even possible in today's browsers.

Here come the GooglePipes being fed by the GooglePOPs, being... ;)

/Hoff