Rational Survivability

Process Control Systems (SCADA and the like) & Virtualization

Just when you get out, they pull you back in...

Jason Holcomb over at the Digital Bond blog posted something that attracted my attention.  He popped up an innocuous entry titled "Virtualization in the SCADA World: Part 1" which intrigued me for reasons that should be obvious to anyone who reads my steaming pile of blogginess with any sort of regularity.

It would be easy to knee-jerk and simply roll my eyes suggesting that adding virtualization to the "security by obscurity" approach we've seen being argued recently is just inviting disaster (literally,) but I'm trying to be rational about this.  I want to understand the other camp's position and learn from it, hopefully contributing to break down a wall or two...

Jason sets it up:

A few years back, the traditional IT world was debating the merits of virtualization. There were concerns about performance, security, vendor support, and a host of other issues. Fast-forward to today, however, and you’ll find virtual machines in use in nearly every data center.

I think it's fair to say that while most folks would be hard pressed to dispute the merits of virtualization, the concerns regarding "...performance, security, vendor support, and a host of other issues" are hardly resolved.  In fact, they are escalating.

<snip>

So what are the implications of this in the SCADA world? I think it’s just a matter of time before we see more widespread acceptance of VMware and other virtualization platforms in production control systems. The benefit here may be less about cost savings, though, and more about increased functionality. The ability to snapshot and clone machines for backup and testing, for example, is very attractive.

I think the paragraph above is extremely telling because it's really focused on debating the value proposition which is really a foregone conclusion for all the reasons Jason mentions.   The real meat will hopefully be discussed in the follow-on's:

We’re going to examine this subject over a series of blog posts. Hopefully we’ll cover all the major topics – security, reliability, performance, serial communication issues, vendor support, and adoption rate, to name a few.

I look forward to your comments and opinions.

In my first comment to Jason's posting, I alluded to a whole host of virtualization-related issues which are grounded in practice and not hype and asked that since SCADA security is billed as being SO much different than "IT security" what this intersection will bring and how one might assess risk (and against what.)

Further, given various C&A standards, I'm interested in how one might approach (depending upon industry) holding these systems up to a C&A process once virtualization is added to the mix. 

It will be an interesting discussion, methinks.

/Hoff

 

Mommy, Why Is There a Server In the House?

Hat tip to Scott Lowe...

This is an honest-to-[insert diety here] book.  You can check it out on Amazon.  You can also read the online version here.

Unfortunately this book hits a little too close to home.  Literally.

You see, there are currently two rackmount appliances, several switches and some laptops whirling away in my my wife's sun room.  Last week they were accompanied by a couple of network security appliances, also.  I work out of my house lab, so I need stuff to hook to my 20Mb/s FIOS line to justify the expense (besides the UFC pay-per-views.)

Each of these global warmers has what must be several hundred cooling fans, various buzzing thingys and 40 power supplies amongst them.  They're so neat to look at in the dark, casting eerie LED reflections onto the snow outside on my deck.  Yet I digress.

But what are they doing here, you ask?  Why are they in the Sun room?

That is exactly the question asked by my four year old.  Daddy calmly answered "Well because they sound like the combined output of a swarm of angry bees and a Sikorsky dual-rotor helicopter and I sure as hell don't want them in my office."

Puzzled, she toddled off to watch Dora the Explorer downstairs in the family room where the only thing resembling a computer is the Verizon FIOS STB with DVR.  Ingrate.

I shall print this handy guide to edumacating my child-spores so that no longer shall I have to endure their petty little questions regarding the 20-node Beowulf cluster I'm building in the kitchen.

Anyone have the ISO for the latest DivorceOS?

/Hoff

I/O Virtualization: The Battle for the Datacenter OS and What This Means to Security

One of the very profound impacts virtualization will have on security is the resultant collateral damage caused by what I call the "battle for the datacenter OS" framed by vendors who would ordinarily not be thought of as "OS vendors." 

I call the main players in this space the "Three Kings:" Cisco, VMware and EMC. Microsoft is in there also, but that's a topic for another post as I bifurcate operating system vendors in the classical sense from datacenter infrastructure platforms.  Google deserves a nod, too.

The "Datacenter OS" I am speaking of is the abstracted amalgam of virtualization and converged networking/storage that delivers the connected and pooled resource equivalent of the utility power grid.   Nick Carr reflects in his book "The Big Switch":

A hundred years ago, companies stopped producing their own power with steam engines and generators and plugged into the newly built electric grid.”

The "datacenter" and its underlying "operating system," in whatever abstracted form they will manifest themselves, will become this service layer delivery "grid" to which all things will connect; services will be merely combinations of resources and capabilities which are provisioned dynamically.

We see this starting to take form with the innovation driven by virtualization, the driving forces of convergence, the re-emergence of grid computing, the architectures afforded by mash-ups and the movements and investments of the Three Kings in all of these areas.

It's pretty clear that that these three vendors are actively responsible for shaping the future of computing as we know it.  However, It's not at all clear to me how much of the strategic overlap between them is accidental or planned, but they're all approaching the definition of how our virtualized computing experience will unfold in very similar ways, albeit from slightly different perspectives.

One of the really interesting examples of this is how virtualization and convergence are colliding to produce the new model of the datacenter which blurs the lines between computing, networking and storage.

Specifically, the industry -- as driven by customers -- is trending toward the following:

• Upgrading from servers to blades
• Moving from hosts and switches to clusters and fabrics
• Evolving from hardware/software affinity to grid/utility computing
• Transitioning from infrastructure to service layers in “the cloud”

The topic of this post is really about the second bullet, moving from the notion of the classical hosts/servers plugging into separate network and storage switches to instead clusters of resources connecting to fabric(s) such that what we end up with are pools of resources to be provisioned, allocated and dispatched where, when and how needed. 

This is where I/O virtualization enters the picture.  I/O virtualization at the macro level of the datacenter describes the technology which enables the transition from the discrete and directly-connected model of storage versus networking to a converged virtualized model wherein network and storage resources are aggregated into a single connection to the "fabric."

Instead of having separate Ethernet, fiber channel and Infiniband connections, you'd have a single pipe connected to a "virtual connectivity switch" that provides on-demand, dynamic and virtualized allocation of resources to anything connected to the fabric.  The notion of physical affinity from the server/host's perspective goes away.

Andy Dornan from Information Week just did a nice write-up titled "Cisco pitches virtual switches for next-gen data centers." 

It's obviously focused on Cisco's Nexus 7000 Series switch, but also gives some coverage of Brocade's DCX Backbone, Xsigo's Director and 3Leaf's v-8000 products.

Check out what Andy had to say about Cisco's strategy:

Cisco's vision is one in which big companies off-load an increasing number of server tasks to network switches, with servers ultimately becoming little more than virtual machines inside a switch.*

The Nexus doesn't deliver that, but it makes a start, aiming to virtualize the network interface cards, host bus adapters, and cables that connect servers to networks and remote storage. At present, those require dedicated local area networks and storage area networks, with each using a separate network interface card and host bus adapter for every virtual server. The Nexus aims to consolidate them all into one (or two, for redundancy), with virtual servers connecting through virtual NICs.

This stuff isn't vaporware anymore.  These products are real...from numerous entities.  These companies -- and especially Cisco -- are on a mission to re-write the datacenter blueprint and security along with it.  VMware's leading the virtualization charge and Cisco's investing for the long run.  When you look at their investment in VMware, the I/O virtualization play and what they're doing with vFrame, it's impressive -- and scary at the same time.

Them's a lot of eggs in one basket, and it's perfectly clear that there is a huge sucking sound coming from the traditional security realm as we look out over the horizon.  How do you apply a static security sensibility grounded in the approaches of 20 years ago to an amorphous, fluid, distributed and entirely dynamic pooled set of resources and information?

Cisco has thrown their hat in the ring to address the convergence of role-based admission and access control with the announcement of TrustSec which will be available in the Nexus as it is in the higher-end Catalyst switches.  Other vendors such as HP, Extreme and now Juniper as well as up-starts like Nevis and Consentry have their perspectives.  What each of these infrastructure networking vendors have in store for how their solutions will play in the world of virtualized and distributed computing is still to unfold.

How might this emerging phase of technology, architecture, provisioning, management, deployment and virtualization of resources impact security especially since we've barely even started to embrace the impact server virtualization has?  One word:

Completely.

More on this topic shortly...

/Hoff

*Update: A colleague of mine from Unisys, Michael Salsburg, prompted me via discussion to clarify a point.  I think that  for at least the short term, the "server tasks" that will be offloaded to I/O virtualization solutions such as Cisco's will be fairly narrow in scope and logically defined.  However, given that NX-OS is Linux based, one might expect to see a Hypervisor-like capability within the switch itself, enabling VM's and applications to be run directly within it.

Certainly we can expect and intermediary technology derivation which would include Cisco developing their own virtual switch that complements/replaces the vSwitch present in the VMM today; at this point given the heft/performance of the Nexus, one could potentially see it existing "outside" the vHost and using a high-speed 10Gb/s connection, redirect all virtual network functions to the external switch...

What a Shocker, Stiennon & I Disagree: Arbor + Ellacoya Make Total Sense...

"Common sense has nothing to do with it. When I say he's wrong, he's wrong." -- Ethel Mertz, I Love Lucy.

What a surprise, I disagree totally with Richard Stiennon on his assessment of the value proposition regarding the acquisition of Ellacoya by Arbor Networks. 

Specifically, I find it hysterical that Richard claims that Arbor is "abandoning the security space."  Just the opposite, I believe Arbor -- given what they do -- is pursuing a course of action that will allow them to not only continue to cement their value proposition in the security space, but extend it further, both in the carrier and enterprise space.

I think that it comes down to what Richard defines as "security" -- a term I obviously despise for reasons just like this.

Here's we we diverge:

I was actually in Ann Arbor last week when news broke that Arbor Networks had acquired Ellacoya a so called “deep packet inspection” technology vendor. I was perplexed. That’s not security.

"That's not security." Funny.  See below.

First let me clear up some terminology.   “Deep Packet Inspection” was the term some Gartner analyst popularized to describe what content filtering gateways do. They inspect content for worms, attacks, and viruses. Somewhere along the line the traffic shaping industry(Ellacoya, Allot, Sandvine) co-opted the term to describe what their devices do: look at the packet header to determine what protocol is being transported and throttle the throughput based on protocol. In other words Quality of Service for network traffic. These devices do not look at payloads at all except in some rare instances when you have to determine if Skype-like programs are spoofing different protocols.

Firstly, Richard conveniently trivialized DPI.  DPI is certainly about inspecting the packet (beyond the header, by the way) and determining what protocol and application that is being used with precision and fidelity.   In a carrier network, that's used for provisioning, network allocation, bandwidth management and service level management.

These are terms every enterprise of worth is used to hearing and managing to!

Certainly a disposition once the packets are profiled could be to apply QoS which is often what one might do in DoS/DDoS situations, but there are multiple benefits of being able to apply policies and enact dispositions which are dependent on the use or misuse of a specific application or protocol.

In fact, if you don't think that this is "security" why do we see QoS/Rate limiting in almost every firewall platform today -- it may not show up in the GUI, but this is a fundamental way of dealing with attack.

Oh, by the way Richard, perhaps you ought to read your own product manuals as Fortinet provides QoS as a "security" function...perhaps not as robustly as Ellacoya...and soon Arbor:

FortiGate Traffic Shaping Technical Note

The FortiGate Traffic Shaping Technical Note, available on the Technical Documentation Web Site, discusses Quality of Service (QoS) and traffic shaping, describes FortiGate traffic shaping using the token bucket filter mechanism, and provides general procedures and tips on how to configure traffic shaping on FortiGate firewalls.

FortiOS v3.0 MR1introduced inbound traffic shaping per interface. For any FortiGate interface you can use the following command to configure inbound traffic shaping for that interface. Inbound traffic shaping limits the bandwidth accepted by the interface.

config system interface
   edit port2
      set inbandwidth 50
   end
end

This command limits the inbound traffic that the port2 interface accepts to 50 Kb/sec. You can set inbound traffic shaping for any FortiGate interface and for more than one FortiGate interface. Setting inbandwidth to 0 (the default) means unlimited bandwidth or no traffic shaping.

Inbound traffic shaping limits the amount of traffic accepted by the interface. This limiting occurs before the traffic is processed by the FortiGate unit. Limiting inbound traffic takes precedence over traffic shaping applied using firewall policies. This means that traffic shaping applied by firewall policies is applied to traffic already limited by inbound traffic shaping.

Lot of uses of the word "firewall" in the context of "traffic shaping" in that description...Here's a link to your knowledge base, just in case you don't have it ;)

Secondly, since availability is often a function of security, as an administrator I'd want to be able to craft a "security policy" that allows me to make sure that the stuff that matters to me most gets through and is prioritized as such and the rest fight for scraps.  Doing this with precision and fidelity is incredibly important whether you're a carrier or an enterprise.  Oh, wait, here's some more Fortinet documentation that seems to contradict the "that's not security" sentiment:

Traffic shaping which is applied to a Firewall Policy, is enforced for traffic which may flow in either direction. Therefore a session which may be setup by an internal host to an external one, via a Internal->External policy, will have Traffic shaping applied even if the data stream is then coming from external to internal. For example, an FTP ‘get’ or a SMTP server connecting to an external one, in order to retrieve email.

Remember CHKP's FloodGate?  Never particularly worked out from an integration perspective, but a good idea, nonetheless.  Cisco's got it.  Juniper's got it...

Further, there's a new company -- you may have heard about it -- that takes application specificity and applies granular policies on traffic that does just this sort of thing: Palo Alto Networks.  They call it their "next generation firewall."  Love or hate the title (I don't particularly care for it) I call it common sense.  Are you going to tell me this isn't security, either? 

The next, next-generation of security devices will extend this decision-making criteria from ports/protocols through application "conduits" and start making decisions on content in context.  This is the natural extension of DPI.

I won't argue with the rest of Richard's points about M&A risk and market expansion because he's right in many of his examples, but that wasn't the title of his post or the real sentiment.

I think that this deal enhances both the capabilities and applicability of Arbor's solutions which have been largely stovepiped and pigeonholed in the DDoS category based upon what they do today.  I hope they can execute on the integration play.

As to the notion of ignoring the enterprise and "doubling down on the carrier market," Arbor has a great DDoS product for both markets; this allows them now to take advantage of the cresting consolidation activity in both and start diversifying their SECURITY offerings in a way that is intelligently roadmapped.

Who knows.  Perhaps they'll re-market the combined products as a "multiservices security gateway" just like Fortinet does with their carrier products (here.)

I think your marketing slip is showing, Rich.

/Hoff

Pushing Reset On the IT vs. SCADA Security Debate....

I think that perhaps I have chosen a poor approach in trying to raise awareness for process control and SCADA (in)security.  You can find recent SCADA posts here, including the "awareness campaign" Mogull and I launched a couple of weeks back that got a ton of eyeballs ...

I believe I reacted poorly to the premise that some of those who assert expertise in this area tend to dismiss anyone who has a background only in what they define as "IT Security" as being unable to approach understanding -- let alone securing -- this technology.

Let me take a step back for a moment.

I'd like to get to the bottom of something regarding the alleged great divide between what is being described as diametrically opposed aptitude and experience required to secure "IT" infrastructure versus process control systems such as SCADA. 

I notice a similar divergence and statements being made between those who specialize in web application security (WebAppSec) versus information or network security (InfoSec/NetSec.)

For example, WebAppSec is a discipline and specialty that some suggest requires a level of experience and expertise that goes beyond that of traditional "information security" or "network security" practitioners.  It is suggested that in order to truly secure web applications, one generally requires programming experience, understanding complex data structures, databases, distributed application architecture, etc., at a very detailed level.

I think these statements are reasonable, but does it preclude an InfoSec/NetSec practitioner from contributing to effectively manage risk in a WebAppSec environment?

A network security practitioner can deploy a web application firewall and generally configure the solution, but the antagonists suggest that in order to provide a level of protection commensurate with the complexity and dynamics of the code which they are attempting to "secure," it cannot be done without an in-depth understanding of the application, it's workflow and behavior.

Again, in reflection, I'd say that's not an unreasonable assessment.  However, WebAppSec and NetSec/InfoSec guys in mature organizations generally should know what they don't know and work together to implement a holistic solution across layers.  It doesn't always work out that way, but in order to secure the WebApps, we can't ignore the underpinnings of the network or information security foundations, either. 

It really should be a discussion, then, on how to unify complimentary approaches at various levels with an overall focus on managing risk. However, what I find is a downright civil war on the "IT" vs "SCADA" security front.  I have to ask why?

Here's an excerpt from a post I found on Dale Peterson's excellent Digital Bond blog.  It was a review of a SCADA security presentation at a CCC event in Italy regarding an introduction (of sorts) to SCADA security.  The premise isn't really important, but I think that this does a good job of explaining some of the issues and sentiments that I am referring to:

Now here’s the good news: Asset owners, you don’t need to worry about hackers. When they talk about “owning critical infrastructure”, they’re just sharing their wildest dreams. In reality, they have nothing in their hands. Zero. Nada. Niente. It will take several more years until the hacker community has learned to master various flavours of PLCs with their different protocols and vulnerabilities. It will take further years until they get to things like OPC and furnish advanced attack methods against it. And by the time they come up with decent exploits for the various SCADA applications that we use today, most CxOs will already be retired. We have heard over and over again that the IT folks aren’t particularly good at securing SCADA environments. Guess what, they aren’t good at attacking them either. However our hackers do think nobody will notice because the stuff is all so complex. That’s what I call “insecurity by obscurity”.

This whole notion of "it's so complex and so few people know anything about it so we have nothing to fear" seems to be the point of divide.

There's another really telling post on Dale's site (authored by him) titled "Firewalls are easy, control systems are hard" wherein the following inaccurate premise is painted which reduces the scope of the entire infosec/netsec profession down to a five-tupule in a basic packet filtering firewall:

One of the common refrains heard again at ISA Expo is that IT firewalls are too difficult to configure and deploy. Several presenters, especially those promoting field security appliances, mentioned this, and it seemed to be generally accepted. While I’m all for simplicity and credit the vendors for trying to ease deployment, firewalls are simple compared to the deploying PLC’s, defining points in the SCADA database, developing displays, control loops, and the myriad of other detailed configuration required to make a control system work.

A firewall ruleset is as simple as defining rules by source IP, destination IP and port. Since communication in control systems is limited as compared to the corporate network, the ruleset is usually very small.

How simple is that compared to monitoring and controlling a complex process distributed over a plant or large part of the country with 5000 points or 100,000 points? I was introduced to control systems in 2000 and have worked on a large number of SCADA and DCS in a variety of industry sectors and I still marvel at the effectiveness and attention to detail in these systems. There is nothing in firewall or any other IT security system configuration that comes close to the complexity in configuring and deploying control systems

That maybe true in an SME/SMB network, but the reality is now that firewalls in a large enterprise (which is a much more reasonable comparison) are just a small piece of the puzzle.  Endpoints numbering in the thousands (if not tens of thousands) which run hundreds of application combinations aren't exactly chopped liver to secure.  Add in Web Application Firewalls, Database Monitoring, Encryption (at rest, in motion,) IDS, IPS, Proxies, A/V, URL Filtering, Anti-spam, NBAD, SIEM, etc. and it just gets more complex from there.

If life were as simple as deploying a firewall and firing off a five-tuple ruleset, we wouldn't be in this pickle.

Regardless of whether a NetSec/InfoSec practitioner knows in-depth details regarding implementing PLC's/RTU's or the inner-workings of the IEC61131-3 block programming language is neither here nor there because it's only one piece of the puzzle. 

Many InfoSec/NetSec practitioners don't have expertise in SQl/pSQL, but they work with the DBA's to secure databases, right?

Once these systems are interconnected to an IP-enabled network, it requires cooperation up and down the stack.  InfoSec/NetSec pro's need to become SCADA-aware and SCADA pro's need to stop suggesting that this technology is just so complex and overwhelming that it's beyond our ability to effectively collaborate and that "firewall jockeys" just can't understand.

The reality is that the bad guys look for the weakest link in the chain.  Will they attack complex protocol stacks and programming languages first?  No, they'll go after the low-hanging fruit like poorly-configured/secured end-nodes, bad perimeter controls and general user-driven crap like we see in the rest of the world.  They won't need to even spell PLC.

We need the same level of information sharing and respective skill set cross-pollinization in this regard instead of squaring off like it's a battle between us versus them. 

/Hoff

Client Virtualization and NAC: The Fratto Strikes Back...

Attention NAC vendors who continue to barrage me via email/blog postings claiming I don't understand NAC:  You're missing the point of this post which basically confirms my point; you're not paying attention and are being myopic.I included NAC with IPS in (the original) post here to illustrate two things:

(1) Current NAC solutions aren't particularly relevant when you have centralized and virtualized client infrastructure and

(2) If you understand the issues with offline VM's in the server world and what it means to compliance and admission control on spin-up or when VMotioned, you could add a lot of value by adapting your products (if you're software based) to do offline VM conformance/remediation and help prevent VM sprawl and inadvertent non-compliant VM spin-up...

But you go ahead and continue with your strategy...you're doing swell so far convincing the market of your relevance.

Now back to our regular programming...

-- ORIGINAL POST --

I sense a disturbance in the force...

Mike Fratto's blog over at the NWC NAC Immersion Center doesn't provide a method of commenting, so I thought I'd respond to his post here regarding my latest rant on how virtualization will ultimately and profoundly impact the IPS and NAC appliance markets titled "How the hypervisor is death by a thousand cuts to the network IPS/NAC appliance vendors."

I think Mike took a bit of a left turn when analyzing my comments because he missed my point.  Assuming I'm wrong, I'll respond the best I can.

A couple of things really stood out in Mike's comments and I'm going to address them in reverse order.  I think most of Mike's comments strike an odd chord to me because my post was about what is going to happen to the IPS/NAC markets given virtualization's impact and not necessarily what these products look like today.

Even though the focus of my post was not client virtualization, let's take this one first:

Maybe I am missing something, but client virtualization just doesn’t seem to be in the cards today. Even if I am wrong, and I very well could be, I don’t think mixing client VM’s with server VM in the same hypervisor would be a good idea if for no other reason than the fact that a client VM could take down the hypervisor or suck up resources.

I don't say this to be disrespectful, but it doesn't appear like Mike understands how virtualization technology works.  I can't understand what he means when he speaks of "...mixing client VM's with server VM in the same hypervisor."  VM's sit atop of the hypervisor, not *in* it.   Perhaps he's suggesting that despite isolation and the entire operating premise of virtualization that it's a bad idea to have a virtualized client instance colocated in the same physical host as a VM next to a VM running a server instance?  Why?

Further, beyond theoretical hand wringing, I'd very much like to see a demo today of how a "...client VM could take down the hypervisor."

I won't argue that client virtualization is still not as popular as server virtualization today, but according to folks like Gartner, it's on the uptake, especially when it goes toward dealing with endpoint management and the consumerization of IT.  With entire product lines from folks like Citrix (Desktop Server, Presentation Server products, XenDesktop) and VMware (VDI) it's sort of a hard bite to swallow.

This is exactly the topic of my post here (Thin Clients: Does this laptop make my assets look fat?), underscored with a quick example by Thomas Bittman from Gartner:

Virtualization on the PC has even more potential than server virtualization to improve the management of IT infrastructure, according to Mr Bittman.“Virtualization on the client is perhaps two years behind, but it is going to be much bigger. On the PC, it is about isolation and creating a managed environment that the user can’t touch. This will help change the paradigm of desktop computer management in organizations. It will make the trend towards employee-owned notebooks more manageable, flexible and secure.”

Today, I totally get that NAC is about edge deployment (access layer,) keeping the inadvertent client polluter from bringing something nasty onto the network, making sure endpoints are compliant to network policy, and in some cases, controlling access to network resources:

NAC is, by definition, targeting hosts at the edge. The idea is to keep control access of untrusted or untrustworthy hosts to the network based on some number of conditions like authentication, host configuration, software, patch level, activity, etc. NAC is client facing regardless of whether you’re controlling access at the client edge or the data center edge.

I understand that today's version of NAC isn't about servers, but the distinction between clients and servers blurs heavily due to virtualization and NAC -- much like IPS -- is going to have to change to address this.  In fact, some might argue it already has.  Further, some of the functionality being discussed when using the TPM is very much NAC-like.  Remember, given the dynamic nature of VMs (and technology like VMotion) the reality is that a VM could turn up anywhere on a network.  In fact, I can run (I do today, actually) a Windows "server" in a VM on my laptop:

You could deploy NAC to access by servers to the network, but I don’t think that is a particularly useful or effective strategy mainly because I would hope that your servers are better maintained and better managed than desktops. Certainly, you aren’t going to have arbitrary users accessing the server desktop and installing software, launching applications, etc. The main threat to server is if they come under the control of an attacker so you really need to make sure your apps and app servers are hardened.

Within a virtualized environment (client and server) you won't need a bunch of physical appliances or "NAC switches," as this functionality will be provided by a virtual appliance within a host or as a function of the trusted security subsystem embedded within the virtualization provider's platform.

I think it's a natural by-product of the productization of what we see as NAC platforms today, anyhow.  Most of the NAC solutions today used to be IPS products yesterday.  That's why I grouped them together in this example.

This next paragraph almost makes my point entirely:

Client virtualization is better served with products like Citrix MetaFrame or Microsoft’s Terminal Services where the desktop configuration is dictated and controlled by IT and thus doesn’t t suffer from the same problems that physical desktop do. Namely, in a centrally managed remote client situation, the administrator can more easily and effectively control the actions of a user and their interactions on the remote desktop. Drivers that are being pushed by NAC vendors and analysts, as well as responses to our own reader polls, relating the host condition like patch level, running applications, configuration, etc are more easily managed and should lead to a more controlled environment.

Exactly!  Despite perhaps his choice of products, if the client environment is centralized and virtualized, why would I need NAC (as it exists today) in this environment!?  I wouldn't.  That was the point of the post!

Perhaps I did a crappy job of explaining my point, or maybe if I hadn't included NAC alongside IPS, Mike wouldn't have made that left turn, but I maintain that IPS and NAC both face major changes in having to deal with the impact virtualization will bring.

/Hoff

 

A Shout Out to My Boy Grant Bourzikas...It's How We Roll...

I was reading Jeremiah Grossman's review of Fortify's film "The New Face of Cybercrime" (watch the trailer here) and noted this little passage in his review:

Then in a bold move, Roger Thorton (CTO of Fortify) and director Fredric Golding (with the 3 other panelists), opened things up to the audience to comment and ask questions. Right when they did that I was thinking to myself, OMG, these guys are crazy asking an infosec what they thought! To their credit they were very patient and professional in dealing with the many inane “constructive” criticisms voiced.

The stand out of the panelists was Grant Bourzikas, CISO of Scottrade, who was able to answer pointed question masterfully from “business” interest perspective. Clearly he has been around the block once or twice when it comes to web application security in the real world.

I was thrilled that Jeremiah pointed Grant out.  See, G. was one of my biggest enterprise customers at Crossbeam and I can tell you that he and the rest of the Scottrade security team know their stuff.  They have an incredible service architecture with one of the most robust security strategies you've seen in a business that lives and dies by the uptime SLAs they keep; availability is a function of security and Grant and his team do a phenomenal job maintaining both.

I can personally attest to the fact that he's been around the block more than a couple of times ;)  It's very, very cool to see someone like Jeremiah recognize someone like Grant -- since I know both of them it's a double-whammy for me because of how much respect I have for each of them.

Wow.  This got a little mushy, huh?  I guess I just miss him and his bobble-head doll (inside joke, sorry Evan.)

My only question is how did Grant manage to escape St. Louis?

/Hoff

CIA: Hackers to Blame for Power Outages ('nuff said)

I'm sorry, did someone say we have nothing to worry about when it comes to SCADA and control systems security?  I must have missed the memo:

CIA: Hackers to Blame for Power Outages

By  TED BRIDIS  –  3 hours ago

WASHINGTON (AP) — Hackers literally turned out the lights in multiple cities after breaking into electrical utilities and demanding extortion payments before disrupting the power, a senior CIA analyst told utility engineers at a trade conference.

All the break-ins occurred outside the United States, said senior CIA analyst Tom Donahue. The U.S. government believes some of the hackers had inside knowledge to cause the outages. Donahue did not specify what countries were affected, when the outages occurred or how long the outages lasted. He said they happened in "several regions outside the United States."

"In at least one case, the disruption caused a power outage affecting multiple cities," Donahue said in a statement. "We do not know who executed these attacks or why, but all involved intrusions through the Internet."

A CIA spokesman Friday declined to provide additional details.

"The information that could be shared in a public setting was shared," said spokesman George Little. "These comments were simply designed to highlight to the audience the challenges posed by potential cyber intrusions."

Donahue spoke earlier this week at the Process Control Security Summit in New Orleans, a gathering of engineers and security managers for energy and water utilities.

The Bush administration is increasingly worried about the little-understood risks from hackers to the specialized electronic equipment that operates power, water and chemical plants.

In a test last year, the Homeland Security Department produced a video showing commands quietly triggered by simulated hackers having such a violent reaction that an enormous generator shudders as it flies apart and belches black-and-white smoke.

The recorded demonstration, called the "Aurora Generator Test," was conducted in March by government researchers investigating a dangerous vulnerability in computers at U.S. utility companies known as supervisory control and data acquisition systems. The programming flaw was fixed, and equipment makers urged utilities to take protective measures.

Now, this article says these attacks were outside the U.S. (since it came from the CIA, you can imagine why.)  Also, it does NOT directly say that SCADA systems were attacked.  However, these statements were made at a SCADA "Process Control" Security conference, so I'm going to take the liberty of bridging that assumption.  Either way, it highlights the problem at hand (see the 787 Dreamliner story and the Polish Tram derailment...)

Do y ou really think it's that much of a reach to suggest it's not happening on our shores?

If anyone gives me any more crap about being concerned regarding the possibility/potential for disruption...look at the boldfaced section.  The compromise was conducted over the Internet.  Don't forget, this sort of thing is supposed to be impossible given some comments from my "awareness campaign":

Oh gosh, where do I begin Chris? 

What do the first letters of SCADA stand for?  Supervisory Control. 

A real SCADA system doesn't issue direct controls. It issues Supervisory Controls. There should be no time critical control loops in SCADA. In other words, we have vulnerabilities. But they won't destroy anything right away. We engineers know better than to trust complex software.

Most good design practice is based upon graceful degradation. In other words, we don't send a command to open a valve. We send commands to change the pressure differential setpoint. A local controller takes care of the rest. There are sanity checks in the local controller.

You could send commands to the field that would screw things up. But most people would notice and we'd take action. Keep in mind, that while our operation is very careful and deliberate, the distribution system was built for some wild extremes including pipe breaks, extreme weather, communcation outages, and vandalism. A successful attack would require intimate knowledge of where the real vulnerabilities are.

Are you an expert at water utilities too? 

Posted by: Jake Brodsky | December 15, 2007 at 11:47 AM

No, Jake.  I'm not a water utilities expert, just a concerned observer & citizen. 

Hat tip to Stiennon for the source.

/Hoff

UPDATED: How the Hypervisor is Death By a Thousand Cuts to the Network IPS/NAC Appliance Vendors

Attention NAC vendors who continue to barrage me via email/blog postings claiming I don't understand NAC:  You're missing the point of this post which basically confirms my point; you're not paying attention and are being myopic.

I included NAC with IPS in this post to illustrate two things:

(1) Current NAC solutions aren't particularly relevant when you have centralized and virtualized client infrastructure and

(2) If you understand the issues with offline VM's in the server world and what it means to compliance and admission control on spin-up or when VMotioned, you could add a lot of value by adapting your products (if you're software based) to do offline VM conformance/remediation and help prevent VM sprawl and inadvertent non-compliant VM spin-up...

But you go ahead and continue with your strategy...you're doing swell so far convincing the market of your relevance.

Now back to our regular programming...

-- ORIGINAL POST --

From the "Out Of the Loop" Department...

Virtualization is causing IPS and NAC appliance vendors some real pain in the strategic planning department.  I've spoken to several product managers of IPS and NAC companies that are having to make some really tough bets regarding just what to do about the impact virtualization is having on their business.

They hmm and haw initially about how it's not really an issue, but 2 beers later, we're speaking the same language...

Trying to align architecture, technology and roadmaps to the emerging tidal wave of consolidation that virtualization brings can be really hard.  It's hard to differentiate where the host starts and the network ends...

In reality, firewall vendors are in exactly the same spot.  Check out this Network World article titled "Options seen lacking in firewall virtual server protection."  In today's world, it's almost impossible to distinguish a "firewall" from an "IPS" from a "NAC" device to a new-fangled "highly adaptive access control" solution (thanks, Vernier Autonomic Networks...)

It's especially hard for vendors whose IPS/NAC software is tied to specialty hardware, unless of course all you care about is enforcing at the "edge" -- wherever that is, and that's the point.  The demarcation of those security domain diameters has now shrunk.  Significantly, and not just for servers, either.  With the resurgence of thin clients and new VDI initiatives, where exactly is the client/server boundary?

Prior to virtualization, network-based IPS/NAC vendors would pick arterial network junctions and either use a tap/SPAN port in an out-of-band deployment or slap a box inline between the "trusted" and "untrusted" sides of the links and that was that.  You'd be able to protect assets based on port, VLAN or IP address.

You obviously only see what traverses those pipes.  If you look at the problem I described here back in August of last year, where much of the communication takes place as intra-VM sessions on the same physical host that never actually touch the externally "physical" network, you've lost precious visibility for detection let alone prevention.

I think by now everyone recognizes how server virtualization impacts network and security architecture and basically provides four methods (potentially in combination) today for deploying security solutions:

1. Keep all your host based protections intact and continue to circle the wagons around the now virtualized endpoint by installing software in the actual VMs
2. When available, take a security solution provider's virtual appliance version of their product (if they have one) and install in on a host as a VM and configure the virtual networking within the vSwitch to provide the appropriate connectivity.
3. Continue to deploy physical appliances between the hosts and the network
4. Utilize a combination of host-based software and physical IPS/NAC hardware to provide off-load "switched" or "cut-through" policy enforcement between the two.

Each of these options has its pros and cons for both the vendor and the customer; trade-offs in manageability, cost, performance, coverage, scalability and resilience can be ugly.  Those that have both endpoint and network-based solutions are in a far more flexible place than those that do not.

Many vendors who have only physical appliance offerings are basically stuck adding 10Gb/s Ethernet connections to their boxes as they wait impatiently for options 5, 6 and 7 so they can "plug back in":

5.  Virtualization vendors will natively embed more security functionality within the hypervisor and continue integrating with trusted platform models

6.  Virtualization vendors will allow third parties to substitute their own vSwitches as a function of the hypervisor

7. Virtualization vendors will allow security vendors to utilize a "plug-in" methodology and interact directly with the VMM via API

These options would allow both endpoint software installed in the virtual machines as well as external devices to interact directly with the hypervisor with full purview of inter and intra-VM flows and not merely exist as a "bolted-on" function that lacks visibility and best-of-breed functionality.

While we're on the topic of adding 10Gb/s connectivity, it's important to note that having 10Gb/s appliances isn't always about how many Gb/s of IPS traffic you can handle, but also about consolidating what would otherwise be potentially dozens of trunked LACP 1Gb/s Ethernet and FC connections pouring out of each host to manage both the aggregate bandwidth but also the issues driven by a segmented network.

So to get the coverage across a segmented network today, vendors are shipping their appliances with tons of ports -- not necessarily because they want to replace access switches, but rather to enable coverage and penetration.

On the other hand, most of the pure-play software vendors today who say they are "virtualization enabled" really mean that their product installs as a virtual appliance on a VM on a host.  The exposure these solutions have to traffic is entirely dependent upon how the vSwitches are configured.

...and it's going to get even more hairy as the battle for the architecture of the DatacenterOS also rages.  The uptake of 10Gb/s Ethernet is also contributing to the mix as we see customers:

• Upgrading from servers to blades
• Moving from hosts and switches to clusters and fabrics
• Evolving from hardware/software affinity to gird/utility computing
• Transitioning from infrastructure to service layers in “the cloud”

Have you asked your IPS and NAC vendors who are hardware-bound how they plan to deal with this Tsunami on their roadmaps within the next 12 months.  If not, grab a lifejacket.

/Hoff

UPDATE:  It appears nobody uses trackbacks anymore, so I'm resorting to activity logs, Google alerts and stubbornness to tell when someone's referencing my posts.  Here are some interesting references to this post:

• Mike Fratto - NWC NAC and Virtualization Immersion Center
• Dana Hendrickson - SecureAccessCentral

...also, this is right on the money:

• Chris Silva - Forrester

I think I'll respond to them on my blog with a comment on theirs pointing back over...

Come to Boston's Own (New) Security Conference in March 2008 - Source Boston

Besides the monthly BeanSec! gatherings, New England really needs a security conference to call its own.  Now we have one.

You can find a ton of detail about the show here, but if you're impatient because you're pahking the cah in the yahd and can't get to your browsa, here's the skinny:

The security convention is called SOURCE: Boston 2008, and it's held from March 12-14th, the W-F before St. Patrick's Day weekend. The place it's being held is the Hyatt Regency Cambridge, right on MIT's campus.

It's the big step-shaped hotel with the neon framing right on the water. We have negotiated low room rates and are sporting quite a line-up of speakers and keynotes, including keynotes from Dan Geer of MIT Athena/Kerberos fame, Richard Clarke, and Steven Levy.

We will also have a panel with the members of the L0pht - speaking together for the first time in 10 years. We have some great evening activities such as a VIP reception and a Thirsty Thursday Pub Crawl.

The three tracks are application security, business and security, and new security technologies. It's a professional conference and we're having several CEOs speak, as well as other chief officers. However, it's combining that professionalism and business component with the edginess and fun of some of the hacker conferences.

Rich Mogull and I are appearing on stage together as Click and Clack (or is it Wallace & Grommet?)  That ought to be worth the price of admission right there.

See you there.

/Hoff

Ginko Financial Collapse Ultimately Yields Real Virtual Risk (Huh?)

I'm feeling old lately.  First it was my visceral reaction to the paranormal super-poking goings-on on Facebook and now it's this news regarding Linden Lab's Second Life that has my head spinning. 

It seems that the intersection between the virtual and physical worlds is continuing to inch ever closer.

In fact, it's hitting people where it really counts, their (virtual) wallets. 

We first saw something like this bubble up with in-world gambling issues and now Linden announced in their blog today that any virtual "in-world banks" must be registered with real-world financial/banking regulatory agencies:

As of January 22, 2008, it will be prohibited to offer interest or any direct return on an investment (whether in L$ or other currency) from any object, such as an ATM, located in Second Life, without proof of an applicable government registration statement or financial institution charter. We’re implementing this policy after reviewing Resident complaints, banking activities, and the law, and we’re doing it to protect our Residents and the integrity of our economy.

Why?  It seems there's more bad juju brewin'.  A virtual bank shuts down and defaults.  What's next?  A virtual sub-prime loan scandal?

Since the collapse of Ginko Financial in August 2007, Linden Lab has received complaints about several in-world “banks” defaulting on their promises. These banks often promise unusually high rates of L$ return, reaching 20, 40, or even 60 percent annualized.

Usually, we don’t step in the middle of Resident-to-Resident conduct – letting Residents decide how to act, live, or play in Second Life.

But these “banks” have brought unique and substantial risks to Second Life, and we feel it’s our duty to step in. Offering unsustainably high interest rates, they are in most cases doomed to collapse – leaving upset “depositors” with nothing to show for their investments. As these activities grow, they become more likely to lead to destabilization of the virtual economy. At least as important, the legal and regulatory framework of these non-chartered, unregistered banks is unclear, i.e., what their duties are when they offer “interest” or “investments.”

There is no workable alternative. The so-called banks are not operated, overseen or insured by Linden Lab, nor can we predict which will fail or when. And Linden Lab isn’t, and can’t start acting as, a banking regulator.

Some may argue that Residents who deposit L$ with these “banks” must know they’re assuming a big risk – the high interest rates promised aren’t guaranteed, and the banks aren’t overseen by Linden Lab or anyone else. That may be true. But for all of the other reasons we’ve set out above, we can’t let this activity continue.

Thus, as we did in the past with gambling, as of January 22, 2008 we will begin removing any virtual ATMs or other objects that facilitate the operation or facilitation of in-world “banking,” i.e., the offering of interest or a rate of return on L$ invested or deposited. We ask that between now and then, those who operate these “banks” settle up on any promises they have made to other Residents and, of course, honor valid withdrawals. After that date, we may sanction those who continue to offer these services with suspension, termination of accounts, and loss of land.

Wow.  Loss of land!  I thought overdraft fees were harsh!?

Ed Felten from Freedom to Tinker summed it up nicely:

This was inevitable, given the ever-growing connections between the virtual economy of Second Life and the real-world economy. In-world Linden Dollars are exchangeable for real-world dollars, so financial crime in Second Life can make you rich in the real world. Linden doesn’t have the processes in place to license “banks” or investigate problems. Nor does it have the enforcement muscle to put bad guys in jail.

Expect this trend to continue. As virtual world “games” are played for higher and higher stakes, the regulatory power of national governments will look more and more necessary.

So far I've stayed away from Second Life; I've got enough to manage in my First one.  Perhaps it's time to take a peek and see what all the fuss is about?

/Hoff

On Patch Tuesdays for Virtualization Platforms...

In December I made note of an interesting post on the virtualization.info blog titled "Patch Tuesday for VMware."  This issue popped up today in conversation with a customer and I thought to bubble it back up for discussion.

The post focused on some work done by Ronald Oglesby and Dan Pianfetti from GlassHouse Technologies regarding the volume, frequency and distribution of patches across VMware's ESX platform. 

When you combine Ronald and Dan's data with Kris Lamb's from ISS that I wrote about a few months ago, it's quite interesting.

The assertion that Ronald/Dan are making in their post is that platforms like VMware's ESX have to date required just as much care and feeding from a patching/vulnerability management perspective as a common operating system such as a Windows Server:

So why make this chart and look at the time between patches? Let's take a hypothetical server built on July 2nd of 2007, 5 months ago almost exactly. Since being built on that day and put into production that server would have been put into maintenance mode and patched/updated eight times. That's right eight (8) times in 5 months. How did this happen? Let's look at the following timeline:

Maybe it's time to slow down and look at this as a QA issue? Maybe it's time to stop thinking about these platforms as rock solid, few moving parts systems? Maybe it's better for us not to draw attention to it, and instead let it play out and the markets decide whether all this patching is a good thing or not. Obviously patching is a necessary evil, and maybe because we are so used to it in the Windows world, we have ignored this so far. But a patch every 18.75 days for our "hypothetical" server is a bit much, don't you think?

I think this may come as a shock to some who have long held the belief that bare-metal, Type 1 virtualization platforms require little or no patching and that because of this, the "security" and availability of virtualized hosts was greater than that of their non-virtualized counterparts.

The reality of the situation and the effort and potential downtime (despite tools that help) have led to unexpected service level deviance, hidden costs and latent insecurity in deployed virtualized environments.  I think Ronald/Dan said it best:

If a client is buying into the idea of server virtualization as a piece of infrastructure (like a SAN or a switch) only to see the types of patching we see in Windows, they are going to get smacked in the face with the reality that these are SERVERS. The reality that the vendors are sticking so much into the OS that patches are going to happen just as often as with Windows Servers? Or, if the client believes the stability/rock solidness and skips a majority of general patches, they wind up with goofy time issues or other problems with iSCSI, until they catch up.

As a counterpoint to this argument I had hoped to convince Kris Lamb to extend his patch analysis of VMware's releases and see if he could tell how many patched vulnerabilities existed in the service console (the big ol' fat Linux OS globbed onto the side of ESX) versus the actual VMM implementation itself.  For some reason, he's busy with his day job. ;) This is really an important data point.  I guess I'll have to do that myself ;(

The reason why this is important is exactly the reason that you're seeing VMware and other industry virtualization players moving to embedded hypervisors; skinnying down the VMM's to yield less code, less attack surface and hopefully less vulnerabilities.  So, to be fair, the evolution of the virtualization platforms is really on-par with what one ought to expect with a technology that's still fairly nascent.

In fact, that's exactly Nand Mulchandani, VMware's Sr. Director of Security Product Management & Marketing in response to Ronald/Dan's post:

As the article points out, "patching is a necessary evil" - and that the existence of ESX patches should not come as a shock to anyone. So let’s talk about the sinister plan behind the increase in ESX patches. Fortunately, the answer is in the article itself. Our patches contain a lot of different things, from hardware compatibility updates, feature enhancements, security fixes, etc.

...

We also want customers to view ESX as an appliance - or more accurately, as a product that has appliance-like characteristics.

...

Speaking of appliances, another thing to consider is that we are now offering ESX in a number of different form-factors, including the brand new ESX Server 3i. 3i will have a significantly different patch characteristics - it does not have a Console OS and has a different patching mechanism than ESX that will be very attractive to customers.

I see this as a reasonable and rational response to the issue, but it does point out that whether you use VMware or any other vendor's virtualization platform, you should make sure to recognize that patching and vulnerability management of the underlying virtualization platforms is another -- if not really critical -- issue that will require operational attention and potential cost allocation.

/Hoff

P.S. Mike D. does a great job of stacking up other vendors against Microsoft in this vein such as Microsoft, Virtual Iron, and SWSoft.

BeanSec! Wednesday, January 16, 2008 - 6PM to ?

 

Yo!  BeanSec! is once again upon us.  Wednesday, January 16th, 2008.

BeanSec! is an informal meetup of information security professionals, researchers and academics in the Greater Boston area that meets the third Wednesday of each month. 

I say again, BeanSec! is hosted the third Wednesday of every month.  Add it to your calendar.

Come get your grub on.  Lots of good people show up.  Really.

Unlike other meetings, you will not be expected to pay dues, “join up”, present a zero-day exploit, or defend your dissertation to attend. Map to the Enormous Room in Cambridge.

Enormous Room: 567 Mass Ave, Cambridge 02139.  Look for the Elephant on the left door next to the Central Kitchen entrance.  Come upstairs. We sit on the left hand side...

Don't worry about being "late" because most people just show up when they can.  6:30 is a good time to aim for.  We'll try and save you a seat.  There is a parking garage across the street and 1 block down or you can try the streets (or take the T)

In case you're wondering, we're getting about 30-40 people on average per BeanSec!  Weld, 0Day and I have been at this for just over a year and without actually *doing* anything, it's turned out swell.

We've had some really interesting people of note attend lately (I'm not going to tell you who...you'll just have to come and find out.)  At around 9:00pm or so, the DJ shows up...as do the rather nice looking people from the Cambridge area, so if that's your scene, you can geek out first and then get your thang on.

The food selection is basically high-end finger-food appetizers and the drinks are really good; an attentive staff and eclectic clientèle make the joint fun for people watching.  I'll generally annoy you into participating somehow, even if it's just fetching napkins. ;)

See you there.

/Hoff  

Kid's Software & The Evolution of SaaS

Something interesting just occurred to me as my (now) four year old and a couple of her friends were huddled around one of the computers in my office playing a Dragon Tales coloring game while my seven and eleven year old were in their respective rooms playing with the virtual pets in their virtual worlds of Webkinz.

Why is the fact that my kids are huddled around their Mac Mini's on a cold day playing computer games in any way remarkable?  Because all of them were playing games which are hosted and presented via the Internet; no software except a browser required on this end.

So? 

Well, the really interesting thing was that my wife reminded me that we haven't purchased ANY children's software titles in the last two years because all of the games, learning applications and reference materials are all on-line.  Software as a, well, service.  And mostly free to boot!  The same usually can't be said in the, um, "adult" realm.

This sounds like the kiddy version of GoogleApps.  How long until you don't even have to buy a DVD for your XBox, PS3 or Wii...as high speed feeds continue to rise, it's not that large of a stretch...

I think it's a very interesting perspective on the more "mainstream" adoption of SaaS and a rather interesting way of monetizing it.  In the case of Webkinz, while the virtual world -- replete with currency, domiciles, and social services -- is free, one has to purchase a physical doll which has attached to it a "secret code" that one uses to register your pet in Webkinz World.  You then give it a name and choose a sex and start building rooms.

It's basically second life for kids -- without the job fairs (yet.)  My wife has now informed me that she's going to get her own doll to play with the kids online.  I wonder if we'll actually ever speak to one another in person any more!? 

Scratch that, she's at this moment registering a bulldog named Waldo.

I'm hoping they start making 5'11 blonde Swedish nurse Webkinz soon...

...which reminds me, given the fact that since all this kid's software is now on-line, how long until someone compromises it for unseemly reasons?   An interesting attack vector, no doubt.  I think I'll call it SaaD - Software as a Disservice?

/Hoff

Horrific Facebook Breach!

I don't know what's happening behind the scenes at Facebook.  Perhaps it's the manifest evils of Beacon victimizing humanity coming back to haunt them, but there's been a horrific breach over at Facebook.

I just don't feel safe there any longer.

I joined FB as a number of groups to which I belong and enjoy monitoring decided to leverage the mass pandemonium that is social netgawking and make their information available only on this populist portal.

So I log on today, fully expecting to check my hatching eggs, play a round of scrabulous and explain to yet another aging filipino male "philanthropist" that I'm really not a 16 year old Catholic school girl trolling for a good time when I discovered the harrowing news:

I'd been SuperPoked!

That's right. 

I was eFingered right after being attacked with a drive-by Vampire tea bagging knee-bar and an inverted trout slap...and some "friend" decided to curse me with a Thunder Pinch and send me a Pink Sock as a gift.

Seriously, what the hell!  When did we start talking like this?  I just mastered Snoop Dogg's schizzle, yo!  I can't keep up.

Most of the people that I am FB "friends" with are 30+ years old.  What possible reason would any self-respecting "old person" have for superpoking, trout-slapping or thunder-pinching me?  I've tried talking my wife into this stuff and it doesn't work unless I liquor her up.  How do you think this is going to work on me?

Me!?

If you can't figure out how to revert to good old-fashioned email or IM, and speak 'murican, I don't want to talk to you.

No more hatching Jack-a-lopes.  No more Roman Candles.  No more Romanian Turtle Wedgies.

Screw this Web2.0 crap.

I'm going to play bowling on my Wii now.

/Hoff

How To Say "Whoops! We Need To Rethink Our Train System's Control Plane" In Polish...

My dear friend Murray (sorry if that expression of warmth comes as a surprise, Murr...) sent me this story from the Register:

Polish teen derails tram after hacking train network

A Polish teenager allegedly turned the tram system in the city of Lodz into his own personal train set, triggering chaos and derailing four vehicles in the process. Twelve people were injured in one of the incidents.

The 14-year-old modified a TV remote control so that it could be used to change track points, The Telegraph reports. Local police said the youngster trespassed in tram depots to gather information needed to build the device. The teenager told police that he modified track setting for a prank.

"He studied the trams and the tracks for a long time and then built a device that looked like a TV remote control and used it to manoeuvre the trams and the tracks," said Miroslaw Micor, a spokesman for Lodz police.

"He had converted the television control into a device capable of controlling all the junctions on the line and wrote in the pages of a school exercise book where the best junctions were to move trams around and what signals to change.

"He treated it like any other schoolboy might a giant train set, but it was lucky nobody was killed. Four trams were derailed, and others had to make emergency stops that left passengers hurt. He clearly did not think about the consequences of his actions," Micor added.

Transport command and control systems are commonly designed by engineers with little exposure or knowledge about security using commodity electronics and a little native wit. The apparent ease with which Lodz's tram network was hacked, even by these low standards, is still a bit of an eye opener.

Problems with the signalling system on Lodz's tram network became apparent on Tuesday when a driver attempting to steer his vehicle to the right was involuntarily taken to the left. As a result the rear wagon of the train jumped the rails and collided with another passing tram. Transport staff immediately suspected outside interference.

The youth, described by his teachers as an electronics buff and exemplary student, faces charges at a special juvenile court of endangering public safety. ®

Yes, yes.  I know, it's not a SCADA system...as fun as that would be to bring up again, I don't need any death threats, so I won't mention it...directly.  But if you read about the recent security design debacle of the Boeing 787 Dreamliner and then look at this, it doesn't take much of a logic jump to see why we should be worried about how command/control systems are implemented.

My next piece of chicanery is to steal one of Mogull's Wii Guitar Hero controllers, hack it, and cause it to electrocute his cat every time he hits C# on Stairway to Heaven...

/Hoff

Thin Clients: Does This Laptop Make My Ass(ets) Look Fat?

Juicy Fat Assets, Ripe For the Picking...

So here's an interesting spin on de/re-perimeterization...if people think we cannot achieve and cannot afford to wait for secure operating systems, secure protocols and self-defending information-centric environments but need to "secure" their environments today, I have a simple question supported by a simple equation for illustration:

For the majority of mobile and internal users in a typical corporation who use the basic set of applications:

1. Assume a company that:
   ...fits within the 90% of those who still have data centers, isn't completely outsourced/off-shored for IT and supports a remote workforce that uses Microsoft OS and the usual suspect applications and doesn't plan on utilizing distributed grid computing and widespread third-party SaaS
2. Take the following:
   Data Breaches.  Lost Laptops.  Non-sanitized corporate hard drives on eBay.  Malware.  Non-compliant asset configurations.  Patching woes.  Hardware failures.  Device Failure.  Remote Backup issues.  Endpoint Security Software Sprawl.  Skyrocketing security/compliance costs.  Lost Customer Confidence.  Fines.  Lost Revenue.  Reduced budget.
   
3. Combine With:
   Cheap Bandwidth.  Lots of types of bandwidth/access modalities.  Centralized Applications and Data. Any Web-enabled Computing Platform.  SSL VPN.  Virtualization.  Centralized Encryption at Rest.  IAM.  DLP/CMP.  Lots of choices to provide thin-client/streaming desktop capability.  Offline-capable Web Apps.
4. Shake Well, Re-allocate Funding, Streamline Operations and "Security"...
5. You Get:
   Less Risk.  Less Cost.  Better Control Over Data.  More "Secure" Operations.  Better Resilience.  Assurance of Information.  Simplified Operations. Easier Backup.  One Version of the Truth (data.)

---

I really just don't get why we continue to deploy and are forced to support remote platforms we can't protect, allow our data to inhabit islands we can't control and at the same time admit the inevitability of disaster while continuing to spend our money on solutions that can't possibly solve the problems.

If we're going to be information centric, we should take the first rational and reasonable steps toward doing so. Until the operating systems are more secure, the data can self-describe and cause the compute and network stacks to "self-defend," why do we continue to focus on the endpoint which is a waste of time.

If we can isolate and reduce the number of avenues of access to data and leverage dumb presentation platforms to do it, why aren't we?

...I mean besides the fact that an entire industry has been leeching off this mess for decades...

---

I'll Gladly Pay You Tuesday For A Secure Solution Today...

The technology exists TODAY to centralize the bulk of our most important assets and allow our workforce to accomplish their goals and the business to function just as well (perhaps better) without the need for data to actually "leave" the data centers in whose security we have already invested so much money.

Many people are doing that with the servers already with the adoption of virtualization.  Now they need to do with their clients.

The only reason we're now going absolutely stupid and spending money on securing endpoints in their current state is because we're CAUSING (not just allowing) data to leave our enclaves.  In fact with all this blabla2.0 hype, we've convinced ourselves we must.

Hogwash.  I've posted on the consumerization of IT where companies are allowing their employees to use their own compute platforms.  How do you think many of them do this?

Relax, Dude...Keep Your Firewalls...

In the case of centralized computing and streamed desktops to dumb/thin clients, the "perimeter" still includes our data centers and security castles/moats, but also encapsulates a streamed, virtualized, encrypted, and authenticated thin-client session bubble.  Instead of worrying about the endpoint, it's nothing more than a flickering display with a keyboard/mouse.

Let your kid use Limewire.  Let Uncle Bob surf pr0n.  Let wifey download spyware.  If my data and applications don't live on the machine and all the clicks/mouseys are just screen updates, what do I care?

Yup, you can still use a screen scraper or a camera phone to use data inappropriately, but this is where balancing risk comes into play.  Let's keep the discussion within the 80% of reasonable factored arguments.  We'll never eliminate 100% and we don't have to in order to be successful.

Sure, there are exceptions and corner cases where data *does* need to leave our embrace, but we can eliminate an entire class of problem if we take advantage of what we have today and stop this endpoint madness.

This goes for internal corporate users who are chained to their desks and not just mobile users.

What's preventing you from doing this today?

/Hoff

Are Virtualization Laws That Are Immutable, Disputable?

A few months ago, Pete Lindstrom shot me over the draft of a Burton paper on virtualization security.  We sputtered back and forth at one another, I called him names, and then we had beer later.

The title of the paper was the "Five Immutable Laws of Virtualization Security."

I must admit, I reacted to what he sent me in a combinational fit of puzzlement and apathy.   I really couldn't put my finger on why.  Was it the "not invented here syndrome?"  I didn't think so.  So what was it that made me react the way I did?

I think that over time I've come to the conclusion that to me, these aren't so much "immutable laws" but more so derivative abstractions of common sense that left me wondering what all the fuss was about.

Pete posted the five laws on his blog today.  A more detailed set of explanations can be found on the Burton blog here. 

I dare you to read through these without having to re-read each of them multiple times and then re-read them in cascading sequence since (hint) they are recursive:

Law 1: Attacks against the OS and applications of a physical system have the exact same damage potential against a duplicate virtual system.

Law 2: A VM has higher risk than its counterpart physical system that is running the exact same OS and applications and is configured identically.

Law 3: VMs can be more secure than related physical systems providing the same functional service to an organization when they separate functionality and content that are combined on a physical system.

Law 4: A set of VMs aggregated on the same physical system can only be made more secure than its physical, separate counterparts by modifying the configurations of the VMs to offset the increased risk introduced by the hypervisor.

Law 5: A system containing a “trusted” VM on an “untrusted” host has a higher risk level than a system containing a “trusted” host with an “untrusted” VM.

Ultimately, I'd suggest that for the most part, these "observations" are correct, if not oversimplified in a couple of spots.  But again, I'm left with the overall reaction of "so what?" 

Pete even mentions the various reactions he's been getting:

I have been getting interesting reactions to these. Some say they are wrong. Some say they are common sense. Some just don't like the word "immutable." I think they serve to clarify some of the confusion that comes up when discussing virtualization by applying fairly straightforward risk management principles.

I want to believe that somehow these "laws" will enable some sort sort of actionable epiphany that will magically allow me to make my virtualized systems more secure, but I'm left scratching my head regarding who the audience for this was?

I don't think it clarifies any "confusion" regarding risk and virtualization and I'm puzzled that Burton suggests that these "laws" will enlighten anyone and dispel any confusion relating to whether or not deploying virtualization is more or less risky than not deploying virtualization:

In reality, we can apply traditional security practices to virtualization to determine whether risk increases or decreases with new virtualization architectures. It shouldn’t be surprising that the increase or decrease in risk is predicated on the current architecture. Here are five laws to live by when evaluating your virtualization architectures.

...

When combining the standard risk principles with an understanding of the use cases of virtualization, a set of immutable laws can be derived to assist in securing virtual environments

So, I'm with the "common sense" crowd since most of these "laws" have been discussed -- and some practical advice to go along with them -- for quite some time before the "Burton Tablets" came down from the mountain.

So I don't disagree, but I'm reminded of a couple of good lines from a bad movie wherein the nasty knight says to the good knight "you've been weighed, measured and found wanting..."

So, there we are.  My $0.02.  I think I'll add a slide or two about this at the virtualization forum next month...

/Hoff

Complimentary Admission to the InfoWord Virtualization Executive Forum, February 4th, San Francisco...

If any of my readers are interested in attending the InfoWorld Virtualization Executive Forum in San Francisco on February 4th, 2008, I have complimentary passes available (a $795 value)

You might remember that when InfoWorld first ran this forum in New York, I was grumpy because there were no topics/speakers focused on security.  I contacted them and expressed my concern.  Alan Shimel doubted anything would come of my kvetching, but lo and behold, the organizers invited me to come and present at this show in February.

So, if you believe in something strongly enough, good things *do* happen ;)

I am speaking (the last speaker of the day -- all that stands between the audience and beer...is me!) so come on down and heckle me:

  Addressing Security Concerns in Virtual Environments
Easy to create and easy to move, introducing a new software layer between hardware and operating system, and operating over virtual networks as well as physical ones, virtual servers present a new order of security risks and challenges. This session will explore the impact of virtualization on network and host security, how security solutions providers are beginning to address them, and the best practices that are emerging for securing virtual environments.

If you're interested in attending free of charge, ping me via email and I'll give you the details in order to register.

Hope to see you there!

/Hoff

{Edited for proper Yiddish...Thanks, Alan!}

Grab the Popcorn: It's the First 2008 "Ethical Security Marketing" (Oxymoron) Dust-Up...

Robert Hansen (RSnake / ha.ckers.org / SecTheory) created a little challenge (pun intended) a couple of days ago titled "The Diminutive XSS worm replication contest":

The diminutive XSS worm replication contest is a week long contest to get some good samples of the smallest amount of code necessary for XSS worm propagation. I’m not interested in payloads for this contest, but rather, the actual methods of propagation themselves. We’ve seen the live worm code and all of it is muddied by obfuscation, individual site issues, and the payload itself. I’d rather think cleanly about the most efficient method for propagation where every character matters.

Kurt Wismer (anti-virus rants blog) thinks this is a lousy idea:

yes, folks... robert hansen (aka rsnake), the founder and ceo of sectheory, felt it would be a good idea to hold a contest to see who could create the smallest xss worm... ok, so there's no money changing hands this time, but that doesn't mean the winner isn't getting rewarded - there are absolutely rewards to be had for the winner of a contest like this and that's a big problem because lots of people want rewards and this kind of contest will make people think about and create xss worms when they wouldn't have before...

Here's where Kurt diverges from simply highlighting nominal arguments of the potential for misuse of the contest derivatives.  He suggests that RSnake is being unethical and is encouraging this contest not for academic purposes, but rather to reap personal gain from it:

would you trust your security to a person who makes or made malware? how about a person or company that intentionally motivates others to do so? why do you suppose the anti-virus industry works so hard to fight the conspiracy theories that suggest they are the cause of the viruses? at the very least mr. hansen is playing fast and loose with the publics trust and ultimately harming security in the process, but there's a more insidious angle too...

while the worms he's soliciting from others are supposed to be merely proof of concept, the fact of the matter is that proof of concept worms can still cause problems (the recent orkut worm was a proof of concept)... moreover, although the winner of the contest doesn't get any money, at the end of the day there will almost certainly be a windfall for mr. hansen - after all, what do you suppose happens when you're one of the few experts on some relatively obscure type of threat and that threat is artificially made more popular? well, demand for your services goes up of course... this is precisely the type of shady marketing model i described before where the people who stand to gain the most out of a problem becoming worse directly contribute to that problem becoming worse... it made greg hoglund and jamie butler household names in security circles, and it made john mcafee (pariah though he may be) a millionaire...

I think the following exchange in the comments section of the contest forum offers an interesting position from RSnake's perspective:                   

Re: Diminutive XSS Worm Replication Contest

            

Posted by: Gareth Heyes (IP Logged)

      

Date: January 04, 2008 04:56PM

      

@rsnake

This contest is just asking for trouble :)

Are there any legal issues for creating such a worm in the uk?

------------------------------------------------------------------------------------------------------------

 

Re: Diminutive XSS Worm Replication Contest

 

            

Posted by: rsnake (IP Logged)

      

Date: January 04, 2008 05:11PM

      

@Gareth Heyes - perhaps, but trouble is my middle name. So is danger. Actually I have like 40 middle names it turns out. ;) No, I'm not worried, this is academic - it won't work anywhere without modification of variables, and has no payload. The goal is to understand worm propagation and get to the underlying important pieces of code.

I'm not in the UK and am not a lawyer so I can't comment on the laws. I'm not suggesting anyone should try to weaponize the code (they could already do that with the existing worm code if they wanted anyway).

So, we've got Wismer's perspective and (indirectly) RSnake's. 

What's yours?  Do you think holding a contest to build a POC for a worm a good idea?  Do the benefits of research and understanding the potential attacks so one can defend against them outweigh the potential for malicious use?  Do you think there are, or will be, legal ramifications from these sorts of activities?

/Hoff