Security In The Virtual World

Leveraging VMWare for Firewall Consolidation - MSSPs

Virtualization has become a powerful way to reduce IT spend as it relates to servers as we all know.  It allows data centers to conserve rack space, power consumption, server cost and a number of other things.

Everyone that has looked at virtualization has looked at it for user applications and has only thought about security as it relates to securing those user applications within the virtual environment.  So this got me thinking about a year ago.  I began to think about the other benefits and use cases for virtualization and security, and one thing that popped into my mind was that security is an application like a Web Server, Database server, etc.  So why couldn't one just virtualize a firewall for the purpose of reducing the number of firewalls within their infrastructure?  Wouldn't this be the same as reducing the number of physical servers within your infrastructure?

Ah, but surely this has been done before??  Cosine had a Virtual Firewall years ago where you could run multiple instances of a Firewall within a single hardware platform.  NetScreen had VSYS (Virtual Systems) that could allow for the delivery of separately managed firewalls within a single platform.  Fortinet has something called VDOM's (Virtual Domains) that does the same thing.  I remember reviewing the Cosine patents when we acquired them at Fortinet.  So, no need to use VMWare to do this if its already been done right?

Well, I'm not so sure of that...

The other day I was speaking with a Telco that is building out an MSSP offering and they were
interested in virtualizing Firewalls so that each customer could have their own firewall in the cloud service.  They could easily pick up the phone and call Fortinet, NetScreen and others but one concern they had was the sharing of resources and the potential for one shared customers traffic to consume a bunch of CPU cycles and effect the performance of the other customers on the shared platform.

I immediately thought of VMWare.  You see, VMWare's Hypervisor based scheduling algorithms allow CPU resources to be partitioned, as well as memory and disk.  One could essentially set up a Virtual Machine and set parameters that say the the VM can never exceed more than 1 gHZ of computing power.  Sort of like this.  I have a 3 gHz CPU and I want to reserve a maximum of 1 gHZ for VM 1, 2 and 3.  Each of the 3 VM's could only peak to 1 gHZ.  Furthermore one can set specific maximums, such that if VM 2 and VM 3 were idle, VM 1 could burst up to 3 gHZ and take advantage of those idle cycles.

The difference between this method and VDOMs or VSYS is that we know have true hardware isolation vs. just a separation in the management  of policies for firewalls.  Its truly a separate firewall just like a physically separated firewall. 

Take a look at the graphic bellow for a better understanding and click comment to give me your opinion of this concept.  I'd love to flush out how useful or not this is.

Thanks!!

JP

High Availability Security In Your Virtual Environment

How many times have security products been the blame for network outages?  Many right? 

If something goes down and the network team gets a call, they immediately point their finger at the Firewall.  If a user can't access something on the network, its the Firewall.  If something is running slow on the network, guess what! 

Its the firewall.

And with Intrusion Prevention products, because they were very unstable during the early years and would crash or generate false positives a lot, customers started demanding that these devices had some failure mechanisms in them.  Customers demanded "Fail Open".  Fail Open to a security guy doesn't make a whole lot of sense because it basically says, if there is a problem with the metal detector at the airport, it should just "Fail Open" and let everyone into the gate area to board airplanes!

I'd rather block all traffic until I know it was secure, but I live in a world where most people don't think like me.  So.... Why the heck am I blogging about this in a virtualization blog?

Well, I know that Virtual Networks function much like Physical Networks and since network engineers don't always trust security devices I understand that the same set of requirements placed on physical security products will be placed on virtual security products.

Why wouldn't the networking guys demand that virtual security products have either "Fail Open" or what I feel is a better solution "Fail Over".

"Fail Open" is not really possible with virtual security products because true fail open means that you have some sort of physical relay or in the case of optical networks, mirrors that short circuit software to allow bits to bypass and flow around the software application.

"Fail Over" however is possible and customers are going to ask for the same things I believe when it comes to uptime on a virtual network as they do a physical network.

Take a look at the attached picture.  It depicts a software solution that has two firewall type products running in Active / Passive. 

CLICK PIC TO ENLARGE

So, as you are looking at security solutions for your virtual environment, you should ask the question of whether or not they provide any high availability and if so, what level of high availability.  Active / Active, Active / Passive, Statefull, Stateless, and everything you've asked of your physical vendors.

My guess is that if you ask and they don't have it, they will start developing it and marketing its ability.  Its a battle that cant be won completely.  Customers will always want high availability be it virtual or physical.

Until the next post...

JP

Network World Focus on Security in 3/17/08 issue

It looks like virtual security is getting some attention this week as seen on the front page of Network World.  There are multiple articles in this issue that talk about the security challenges in the virtual environment.  I suggest everyone interested in the topic take a read.

After reading the articles, I did want to put out a short blog today to bring clarity to some of the vendor hype and mis-information that has been floating around lately.  I've heard many people say that Reflex, Blue Lane and Catbird provide security between virtual machines.  This isn't true.  What these vendors do is provide "monitoring" between virtual machines as stated on page 48 of Network World's article on virtual security.  What monitoring gives you in the case of Reflex is IDS (detection) and while useful it does not provide what many THINK it does.  Many think it provides prevention. 

<-- Click to enlarge

The way they provide monitoring is by taking a port on the virtual switch and enabling "promiscuous mode" and hanging a virtual security appliance off of that port.  Its in a sense like setting up a span port on a physical switch and mirroring all traffic out that span port.

This is definitely helpful from a visibility perspective  but does not give  you  VM to VM isolation or VM to VM intrusion prevention.  Take a look at the attached graphic from Reflex.  They displayed this graphic today on a webinar about PCI compliance.  You'll notice the VM's on the right can all talk to each other and the VM's on the left can all talk to each other.

<--Click to Enlarge

Now from the picture you can see that it does provide prevention from the GROUP of VM's on the left if they try to talk to the GROUP of VM's on the right.  I've yet to see anyone deploy their VM's like this however it does make sense to put your VM's in trust Zones. 

I am of the opinion however to put every server on their own trust zones and set up policy between those zones.

-JP

Virtual Environments will be more secure than their physical counter parts by 2010

Montego Networks Prediction:

Virtual Environments will be more secure than their physical counter parts by 2010.

Neil McDonald of Gartner reported in 2007 that throughout 2009, 60% of virtual environment deployments would be less secure than their physical counter parts.

Although I tend to believe Neil’s prediction I’m a bit optimistic about the markets awareness of the security concerns within virtualized environments and feel companies will start to address those concerns by 2009. I also believe that by the end of 2009 the majority of companies virtualizing will have built virtualized environments that are more secure than their physical counter parts.

Now, you may be thinking I’m either crazy or that I’m just one of these guys that just states the opposite of what someone else says!

Well, not at all. I’ve been studying the virtual security market for some time now and after talking with many companies that are deploying virtualization I’m starting to get the sense that people get it (security). It’s pretty evident that when people are made aware of what seems to be the obvious (security), that something clicks and they get it right away. In fact, many times the light bulbs start turning on and people start thinking about more creative ways to secure severs by taking advantage of virtualization which enables them to do things they’ve never been able to do before. 

So, although I agree that there has been this issue of security being once again forgotten and that 60% of virtual environments will be less secure up until 2009, I’m not so sure I’m going to underestimate the market and think that this pattern will continue much longer after that.

Take a look at the following graphic and it depicts the various layers in a network. History has proven itself time and time again that a new network layer is built first and security always comes along afterwards.

 

Well, one of the challenges we’ve seen with these physical networks is that it’s pretty costly, time consuming and a burden to purchase, install and administer security. Then once it’s in place and being run, you have to fork lift upgrade certain parts of your security infrastructure due to bandwidth demands and changes in application security concerns.

What virtualization brings to the table is not only cost savings for server consolidation, power consumption and datacenter space but the ability to do all of those things for parts of your security infrastructure as well.

Imagine instead of having to deploy engineers to install 20 firewalls across your datacenter, you could sit from a single workstation with a couple of guys and install 20 firewalls in hours vs. days. The reason this is possible is because now firewalls have just went virtual! You can roll them out as software images or virtual appliances without leaving the comfort of your cubical. 

Imagine being able to “virtual-lift upgrade” vs. “fork-lift upgrade” a new firewall, UTM appliance, IPS or whatever by simply powering off a Firewall Virtual Machine and powering on a new one.  Imagine being able to improve your performance by taking advantage of the multi-core processing and blade server computing trends vs. waiting for the next super fast security ASIC chip.

In the past it’s been difficult to get security as close as possible to the servers and desktops without having to deploy host based solutions. The reason for this is because we have been constrained by the physical limitations of our hardware purchases from the likes of Cisco, Extreme and Foundry. Then for vendors that have thought about putting security in a switch there has always been the price per port debate. Also, many don't want to take the risk and replace Cisco for a new startup building a new switch (ie. Force 10's Switch + IPS product).  Typically switching ports are cheap and security is more expensive and when trying to combine the two, you end up with a switch that costs a lot of money. So imagine having a 200+ port switch with a Firewall built in for $300 bucks. How could this be so? Because its virtual, and because its 100% software.

Did he just elude to a firewall for every port?  Does each Server or Desktop have firewalling between every other Server & Desktop on the same switch?  Absolutely! all because of virtualization!

Software makes it easier to bring the price per port down. When things are in software you can deploy multiple copies of them to scale your network capacity without breaking the bank. Virtualization also allows you to do things like “Freeze” and “Thaw” servers and desktops automatically when vulnerability is detected. If a denial of service is occurring against a Virtual Server you can always VMotion that server to a network with more capacity without an administrator having to lift a finger. Imagine an attack happening on a machine and instead of it being quarantined it makes a snapshot image of the infected machine and freezes it in its current bad state so you can go back and analyze how someone broke in. As you can see, there are lots of new capabilities brought to the security round table.

Virtualization will make security solutions even more powerful and increase the adoption rate of security in general due to the massive cost savings that can be appreciated through virtualization. For these reasons I see the market quickly leveraging virtualization to make Virtual Environments more Secure than their counter parts. Virtualization will enable the innovations in security that has been since UTM and Reputation based Anti-Spam.

VMWare, Virtual Iron, Citrix and others, thanks from the security industry for the innovation!

John Peterson, Montego Networks, Co-Founder & CTO

Montego Networks spotted on radar

 

Montego Networks has been flying under radar for the past year and this week increased its elevation just enough to be seen on the virtualization industries radar detector. Montego Network’s announcement of securing virtual network communications between VM’s has everyone buzzing but what has caught most people’s attention is Montego Network’s technology that enables 3^rd party security vendors to do the same thing (VM to VM). Now, I’m the CTO of Montego Networks, so my comments here are a bit biased but also first hand. So, when I tell you that it’s been a great announcement, I truelly feel it has. Everyone I have spoken with in the analyst and press community thus far has embraced the idea of security vendors working together to provide a solid solution vs. every vendor trying to be all things to everybody.

So, what does this really mean and how does it work?

Let’s say you have VM1 (Virtual Machine) and VM2 (Virtual Machine) and they need to be able to transfer data between each other but only once or twice a week. This means you can’t have them 100% isolated. Because you have a communication need between them, it probably makes sense to only open up the channels (TCP/UDP Ports) that they need to communicate on vs. opening up all channels. This helps mitigate exposure. So, let’s say you open up port 6667 and only port 6667 for them to communicate with each other. Well, this is now a bit more secure than the other option of leaving all ports open but let’s say this is a very very critical server and you want deep packet inspection done on all of its traffic. The reason you want to do this is because there is the potential that worms and BOTnet communication could occur over this port 6667 but the only way to determine that is to do deep packet inspection.  I am using port 6667 as the example because I spoke with someone that had a real live case where one of their Linux VM's got infected with this BOTnet:  http://www.energymech.net/ on port 6667

Now, I could put some sort of virtual IPS product inline and look at Physical to Virtual communication for all of the VM’s (VM1, VM2, VM3, VM4, etc.) but I don’t care to take that kind of performance hit and I also already have a physical IPS handling Physical to Virtual. What I really needs is IPS between the VM’s which I haven’t been able to find from any vendor yet and even if I did find such a solution on the market I don’t care to take the performance hit of doing IPS between ALL VM’s.

So, now that you understand the challenge, how can Montego help and what’s this HyperVSecurity thing they talked about in their press release that allows other vendors to interoperate with them. Well, with Montego’s Policy Based Switching technology you, the administrator can control what types of VM to VM traffic you would like to have inspected by a 3^rd party security solution. I would simply set up a policy that says VM1 to VM2 on port 6667 will have its traffic sent to a StillSecure virtual IPS product and once a week when that traffic starts to flow it will be sent over to the IPS product for further inspection. Or if traffic starts to flow outside that once a week norm, it will still be sent for inspection. This way if some attacker tries to get in on that port he will have to make sure he can get past the IPS that now is able to VM to VM IPS.

Pretty cool huh? I think so.

 Now, back to Montego coming out of stealth mode…

You’ll start to hear and see a lot more innovation coming out of Montego Networks now that we’ve popped slightly above radar and the industry knows we are here but is scrambling trying to figure out what exactly we do, how sustainable will this new startup be and if we really have what we say we have. I’m certain competing companies will throw FUD and make all sorts of comments about what we do, how it performs, etc. etc. and all I can say is to just keep an eye on the after burners because we are starting to get lift off.

-JP

NetFlow and Visibility in the Virtual Environment

With so much talk about securing communications within the virtual environment and potential hypervisor based attacks, we sometimes forget about the visibility problem within the virtual environment.

Today's blog is about just that. Visibility!

We've all probably heard the saying, its hard to secure what you can't see and that understanding your environment is the first step to security.  Well, with virtualization, understanding whats going on in your virtual environment is even a challenge.  Because virtual switches are not as feature rich as physical switches we are left unable to do many of the things we've done in the physical world that enables visibility.  One of the features that exists in physical switches that is commonly used as a security and visibility tool is Netflow.

Over the past week or so I've begun speaking with VMWare customers and Netflow enabled vendors like Mazu Networks (who has an awesome product) and they both have been struggling to figure out an elegant way of gaining visibility into the VM to VM communication within the virtual infrastructure.  You see, in the physical world people turn on Netflow on their switches so that they can do reporting and behavioral analysis but in the virtual world there is no Netflow enabled virtual switch (at least not until now - I'll get to that in a moment). 

So for companies like Mazu Networks and Lancope and for their customer base that is migrating parts of their network to virtual networks, there exists a significant challenge to the business of behavioral based analysis.  Investment in tools that use Netflow enabled switches now starts to become obsolete for parts of the network that is now virtual. 

We've heard vendors to date talk about Virtual Patch Management, Virtual Firewall, Virtual IPS but these talks leave customers confused on what they really need and doesn't necessarily solve all of the security and visibility challenges they thought they had already addressed.  Hmm.. Maybe whats needed is the ability to enable all of these things.  What about Virtual Behavioral Analysis!  Wow, another Virtual Security product that we haven't thought about!  Maybe someone could just virtualize a Behavioral Analysis product and run it inside VMWare,  put the world "Virtual" in front of the name of the technology and call it a day?  Hmmm.. Thats probably not a good idea due to the performance impacts you could encounter.  One of the biggest challenges with security is how to do all of the things we've done in the physical world in the virtual world without impacting performance.

So, back to visibility... Netflow is a technology originally invented by Cisco that sends flow records to a listening device that does some data crunching on those flow records to give you a visual picture of the data in the network.  With this data you can determine abnormalities in traffic patterns, see who the top talkers are in a network as well as home in on what network applications are running in the environment.  With this information you are now better equipped with the right level of knowledge of the environment to start putting security controls in place.  The problem is that it doesnt exist in the virtual switch provided by VMWare, Citrix, etc..

So, how can we do Netflow in the virtual environment so that we can have "Virtual Behavioral Based Analysis"?  Well after looking into this problem and talking with Netflow experts at Mazu Networks, Montego Networks has now enabled Netflow in its Virtual Security Switch. 

Heres how it works:

VM1 is sending traffic to VM2 and VM3 is sending traffic to VM9 and VM5 is sending traffic to the physical network.  Well, for the VM to VM communication, any physical Mazu or Lancope boxes will have either no visibility or have to get creative and put a solution in place thats not optimal or practical.  Vendors in this space are also probably concerned about shrinking revenue if more of the physical network starts to erode away as virtual networks take off and customers are probably concerned about investment in products that are no longer able to provide maximum value.

So as traffic enters Montego Network's Virtual Security Switch we will send a Flow record to a Mazu Networks or a like listening device on the physical network.  Since we see VM to VM communication we can extend this capability to 3rd parties by simply sending them a Netflow record for them to analyze and tada!  You have Behavioral Analysis for your virtual environment.  Notice the Netflow text on the bellow graphic.  It depicts collecting data from the virtual servers and sending a Netflow record somewhere.