Security In The Virtual World

Security Through Visibility - Montego, Lancope and NetFlow

We've probably all heard that you can't secure what you can't see and that statement is even more profound when it comes to virtual environments.  This is because it is extremely challenging to see what is going on at a micro vs. macro level within a virtual environments network.  The virtualization vendors such as VMWare and Citrix have provided embedded tools into their management consoles that show a macro level of visibility but its not enough to identify security events in the environment.  Take a look at the attached picture.  It simply shows VMWare's ability to monitor virtual network performance statistics from a bits per second perspective.

<-Click To Enlarge

With only this level of detail how can one determine which network applications are causing spikes.  Is it FTP traffic that is occuring at a high volume at an unuseal time of day?  If that were occuring, could that be indicative of either a breach or some sort of problem? What if FTP isn't even an authorized service in the virtual environment but there is a high volume of it?  Did someone install a rouge FTP service so they could steal information from the server at will?

These types of questions can't really be answered without a micro level of detail into the packets flowing in, out and within the virtual environment.  Now, what I am highlighting is not security in the traditional sense of prevention but using visibility as a means to first identify, then pin point the source of an issue so that it can properly be mitigated.  Having constant visibility can also ensure that other security products in the environment are performing as expected.  What if a Montego HyperSwitch with firewalling enabled is configured with many policies but someone forgot to create an FTP block policy.  One could think they are protected from rouge FTP services transmiting data out of the network, but without constant visibility monitoring, can you be certain?

Some vendors, namely Reflex Security will get you to believe that their IPS / IDS solution that is inline and running in the virtual environment is the right and only approach.  Or they will tell you to hang a virtual IDS off a span port in the virtual environment and you will at least have visibility into the attacks that are taking place.  Well, sure... You now have attack visibility but at the performance cost of your virtual environment.  Signature matching technologies are great, I'm a huge believer; however they don't scale very well in shared computing environments such as virtual ones.  IDS systems also don't typically track protocol and network service (FTP, HTTP, etc.) utilizations; which is another important part of visibility.

So, what do we do to gain visibility without the performance headache?  Well, for starters its probably best to put your IDS/IPS solutions in the physical environment where performance will be less of a concern.  In fact, you can span a virtual switch's traffic out to a physical NIC as easy as you can to a virtual one.  So why do it virtual and have to pay a 60% CPU utilization tax?  Another solution is to IDS inspect only the things you care about.  Why IDS inspect SSL traffic if you know your solution can't unencrypt SSL.  Its just a waste of compute cycles isnt it?  Policy based switching helps you with directing only the things you care about to an IDS (attack visualization product).  Montego's HyperSwitch also can help you with the traffic redirection of only the things you care about.

Another method of visibility which I tend to be a fan of is one of packet analysis (aka NetFlow).  NetFlow was invented by Cisco some time ago and has gained popularity in the physical world and definately has a use in the virtual world.  NetFlow is lightweight.  Let me say that again, its light weight!  It only sends a summation of packet detail to an analytical engine which can do some number crunching, packet comparison, etc. etc. to make some sense out of whats going on.  Lancope, an Atlanta based visibility company that provides Network Visibility, Security Visibility and User Visibility has this tool on their website that is a Netflow Bandwidth calculator.  You'll see from playing with this ( http://www.lancope.com/netflowcalculator.aspx ) calculator that it doesn't consume a lot of network bandwidth to transmit these network accounting records.  It also doesn't cause a lot of CPU overhead to send these records to an analytical engine sitting somewhere in the network.

Lancope's analytical engines have the ability to do the following for you within your virtual environment:

 

1. •Monitor and Alert network behavior of VMs
2. •Track Vmotion movement of VMs accross physical servers
3. •Monitor and Alert on communication between VMs
4. •Identify users accessing VMs
5. •Identify unauthorized or rouge VMs
6. •Monitor and Alert when VM’s go online or offline
7. •Identify network services running on VMs
8. •Monitor Network / Application performance of VMs
   Display active hosts accessing VMs

...and probably a slew of other things I'm not aware of.  A screen shot of their product is bellow:

<- Click to enlarge

You'll notice from the screenshot that you are able to visualize who is talking to who, how much traffic they have sent and received and something called a concern index (not seen on this screenshot).

Now, a concern index is a number that increases as Lancopes analytical engines monitor suspicious activity on a session.  A high counter can be indicative of a security problem.  Its another way of identifying (visualizing) compromised hosts (virtual machines) without having to do signature matching like a heavy weight IPS engine.  Example:  Lets say you have a VM that has a BOT on it and is "owned".  The Lancope product is monitoring this long life session.  Let's say that session is established for several hours or maybe even days or months.  Lets also say that the conversation appears to be mostly unidirectional from a public ip address not belonging to your enterprise.  Lancope would increase a the concern index on this since this server hasn't typically had this type of behavior.  Once the concern index reached a certain level it could then fire off an email, send you a text message or something saying:  Warning, Warning, Danger, Danger Will Robinson!!! You're virtual server may be infected with a BOT, please investigate immediately!!!

This example is VISIBILITY which helps you with SECURITY.  There are a number of other things you can do with NetFlow and Lancope products that have less to do with security and more to do with operational efficiencies.  Things like, helping you answer questions of:  How do I know what network applications are taking up the most bandwidth?  When should I move those applications over to a server with more horsepower?  When did these VM's vmotion over here and was there a traffic condition / CPU condition that caused that to occur?  I could go on and on but thats a topic for another blog entry.

So, my suggestion is to take a look at what NetFlow has to offer.  Montego Networks supports NetFlow transmission and Lancope supports NetFlow analytics and with both you can regain what was lost visibility.

I hope this was helpful to you all!

-John Peterson

Virtual Security NIC - Concept

Virtual Environment performance has been a widely discussed topic when it comes to running security within virtual environments and there is this concept that I have had in my head for a while now that I thought I'd share with the public to get feedback on.  Its called the Virtual Security Nic and is intended to move security out of the shared computing layer (virtual environment) and into the physical layer with dedicated processors.  By doing this the performance challenge goes away and you are able to get security as close as possible to the VM's.  All traffic going from VM to VM will have to traverse the bus and be inspected by this security NIC before it is delivered to its final destination.

Take a look at the picture bellow and feel free to comment either on this blog or email me at:  [email protected]

Security Between Virtual Machines?

Is there security needed between virtual machines?  Some say no, some say yes.  I've been out talking to a number of virtualization users and non users on this topic and I'm finding that some say no and some say yes.  The users of virtualization technology tend to say yes while others looking at virtualization from the outside tend to say no.  Why is this?

Well, I thought I'd blog on my thoughts on this!

You see, in the physical datacenter there is no firewalling between servers plugged into the same switch and because of this some people think, well if its not done in the physical world why should it be done in the virtual world.  I believe that its not done in the physical world today because there are no solutions today that embed security into datacenter switches.  Should it be done in the physical world?  I think so!  It never hurts to get security as close as possible to the things you are trying to protect and what better place than the switch port in which the critical asset are connected to.  This is why people have HOST BASED FW/IPS ON SERVERS!  To get security as close as possible!  Is that needed? 

So my first response to those that say, security between virtual machines is not needed because its not done in the physical world is:  Well, just because people have done things one way for many years doesn't mean there isn't a better way.

Would environments be more secure if there was security between servers?  I tend to think so.  You see, many of the attacks that are taking place these days are not attacks for fame but attacks for fortune and gone are the days where people just hacked to spread nasty viruses.  Its all about the data these days (ie. credit cards, social security numbers, etc).  We've all heard about the TJ Max security breach where customer data was compromised and many others like banks that have had credit cards compromised. 

How and the heck do you think most of these things happened?  Attackers are targeting the datacenter these days.  Physical or Virtual.  Their gateway into these environments are the Web Front End Servers.  Let me say that again.  The Web Front End Servers!  Hackers get to the data from the web front end server that talks to the database backend server.  This useually occurs by something called "Cross-Site Scripting" or "SQL Injection" breaches. 

Here is a trival way of how this happens:

A hacker finds a vulnerable web site.  He sometimes does this by something called Google Hacking.  He uses Google to search for sites that has vulnerabilities on it.  Say a web site has some content on one of the pages that says "Powered by Drupal 4.1".  If a hacker knows that Drupal 4.1 software has a vulnerability in it, he can now target all the search results related to this.  Click Here for more detail.

Now lets say Drupal 4.1 on a web site has a SQL-Injection vulnerability because the developer of the Drupal software didn't do Form Field Validation properly.  A Form field is something you fill out on a web page like a form that asks for the user name and password.  User names and passwords to log into the web site are stored on whats called a Database Server.  Hmmm... So this means the web server needs to talk to the database server right?  Yes!  Keep this in the back of our head for now.  The hacker enters in "Admin" for the user ID and "password doesn't matter 'or 1=1--" for the password.  And presto!  He is logged in to the server as Admin.

The reason he was able to log in is because the web site sends a SQL Database command to the Database server and because the developer of the Drupal software didn't do "Form Field Validation" properly (method of checking for invalid characters like the ' (single quote)  symbol), the user was able to bypass the password.  Notice the 'OR 1=1 command appended to the password.  One does equal one so therefore it will return a TRUE result to the password checker and the OR says use the password typed in (password doesnt matter) OR check to see if one is equal to one.  If its true then the password is valid for this user which is Admin.

Now that the user is on the web server, he probably has the ability to connect to the database server or other servers in the network.  Why?  Because there is connectivity from the web front end to all of the backend servers.  He essently can backdoor his way throughout the network.

Another method is for him to append some SQL statement to another SQL statement.  Lets say their is a FORM FIELD on the website that collects some information from the database to display it to web site users.  It could be entering in the Zip code to find store locations in your area.  Instead of putting in the zip code you could put in "95123 'UNION SELECT * FROM credit_card_table--".  The hacker is injecting via the UNION command (which means join one SQL statement with another one) a command that says grab all (via the asterisk) information out the credit card table.

Lastly, the hacker can use the UNION command to write text of his desire to a text file on the database server.  He may write some nasty code, tell the database to write the code to a file and then tell the server to execute that file.  The code could be used to do a denial of service attack to the other virtual machines or whatever.  The possibilities are endless!!

Anyway, these are high level examples.  I think you get the point.

The Web Front End Virtual Machine has a need to talk to the Web Back End Virtual Machine and security such as Firewalling, Intrusion Prevention definately needs to be in place to have a higher level of security.

Another reason to have security between virtual machines is because servers are now mobile in the virtual world.  They move between trust domains to take advantage of computing resources that may be available on a given piece of hardware.  Lets say one PHYSICAL server was hosting database VM's and another PHYSICAL server was hosting file server VM's.  The file server VM could VMOTION to the same environment as the database VM's.   Now where is your isolation between trust domains or unlike resources?

People should think about this problem in greater detail.  I'd love to hear everyones comments as to whether or not they think security between VM's is needed.

John Peterson
Montego Networks

Lancope and Montego Networks Does VM2VM Visibility with Netflow

I've blogged on this topic of Netflow enabling visibility within virtual environments in the past but thought I'd discuss this topic once again as I feel visibility within virtual networks is VERY important.

One of the big problems that comes along with virtualization is the inability to see "hidden" traffic flows within virtual networks created by VMWare, Citrix and others.  There are a number of companies off building "agents" or visibility products that can drop inside VMWare to bring back that visibility that was once had in the physical network and I think this is great!  But! These solutions although needed are yet another tool that needs to be purchased, managed and maintained.  These new solutions also only work within the virtual environment.

Wouldn't it be great if you could leverage either existing tools that give you physical network visibility or being able to use a tool that could give you both physical and virtual visibility?  It would be one less thing to manage right?  It could also probably correlate information for your entire network vs. just a subset of it.

Well, look no further.  With the enablement of a feature called Netflow within virtual switches from Montego Networks and an experimental version that exists in VMWare ESX 3.5 you can now export Netflow records to physical network monitoring solutions from the likes of Lancope, Plixer International, Mazu Networks, Arbor Networks and others.

What triggered my blog on this topic today was a webinar I listened in on this morning from CTO Adam Powers of Lancope.  He did an excellent job explaining the how VM 2 VM communications are hidden and how you can bring back that visibility by leveraging Netflow and Lancope. 

I would suggest everyone interested in this topic CLICK HERE to register for the next Webinar by Lancope on this topic.  It starts at 2:00 PM EST today May 21st 2008. 

Bellow are a couple of screenshots from the webinar that was hosted earlier today.

CLICK IMAGES TO ENLARGE

Also the picture bellow shows a nice graphic of how the Montego Networks HyperSwitch interacts with Netflow devices.

Again, I would suggest everyone interested in this topic CLICK HERE to register for the next Webinar by Lancope on this topic.  It starts at 2:00 PM EST today May 21st 2008.

John Peterson

Is Virtual Security Technology A Prime Target For Acquisition?

This week has been an interesting week in the virtual security blog world!  Simon Crosby of Citrix/XenSource stated in his podcast that he felt the virtualization vendors like VMWare and Citrix didn't have the competence to address the security challenges of virtualization and Chris Hoff blogged about it saying that the statement is a cop-out and that they should do more in securing their platforms. Alan Shimel also blogged on the topic and agreed with Hoff and I blogged about it agreeing with both Simon and Hoff. 

To restate my position on it; I think that Simon is correct in that virtualization vendors like VMWare and Citrix do not have the expertise today to address all of the security challenges.  I also agree with Hoff that they should address more of the security challenges.  So this leads me to my own opinion that some of the virtualization vendors will acquire security technologies to differentiate  themselves from others and acquire the expertise.  Many say that the virtualization market will become commoditized and  that security can help protect its value. 

Think about it.  Would you rather buy a Virtual Environment or a Secure Virtual Environment?!

So.. Onto the topic of this blog!  Is Virtual Security Technology A Prime Target For Acquisition?

I'd love your opinion so please comment!!

What triggered my blog on this topic was this rumor I heard today.  Some buzz started today that one of the virtual security startups just agreed behind closed doors to be acquired by one of the big guys.  But, who could it be?  Reflex Security, Catbird, Blue Lane, Altor Networks, VMSight, Embotics, etc.

I have an idea of who it could be but don't want to spread rumors that could be false.  The other question is whether or not there is an atmosphere of acquisition frenzy brewing in the virtualization market. 

Please comment on your thoughts - Just click the comments link bellow.

Virtualization Vendors Are Not In The Security Business?

Simon Crosby, CTO of Citrix/XenSource made a pretty bold statement yesterday that has some people agreeing with his position and others disagreeing.  In an interview with searchsecurity.com he publicy stated that virtualization vendors are not competent to try and secure virtual environments and therefore looks to 3rd party security companies to solve these concerns. 

Listen to the podcast here

Who are these 3rd party security companies?  Well, there are a number of startup companies such as Montego Networks, Blue Lane, Catbird, Altor Networks as well as some of the big guys that are working on helping the virtualization vendors with these security concerns.

I tend to agree with Simon that the virtualization vendors don't currently have the expertise to deliver appropriate security controls for virtual environments BUT should they?

Well, Chris Hoff who blogs on the topic of virtualization security a lot seems to think that they should deliver security tools and and by not delivering solutions to secure the environment they are doing their customers a disservice.

"Further, I don't expect that the hypervisor should be the place in which all security functionality is delivered, but simply transferring the lack of design and architecture forethought from the hypervisor provider to the consumer by expecting someone else to clean up the mess is just, well, typical."  Said Chris Hoff in his blog on this topic

I've spoken with a number of research analysts, venture capitalists and customers on this topic over the last several months and whenever I tell them what Montego Networks is off building they ALL seem to ask the same questions.  One of those questions is:  Why isn't VMWare or Citrix/Xensource doing this?  My response has always been that "they have publicly stated they do not want to and plan on leveraging an eco-system of security vendors to provide this". 

Well, Simon's public statement is right in line with what I've been saying all along.  The other question I get when I describe how Montego has security built into a virtual switch we've created is; shouldn't this technology be in the VMWare Virtual Switch?  And my response is "absolutely!  But it isn't!  so, someones got to do it."

So, I agree with Chris Hoff and I also agree with Simon Crosby.  The virtualization vendors don't have the expertise BUT I feel they should provide SOME security tools to ensure the environment is safe. 

There are some virtualization vendors that I have spoken with that are planning on using security as a differentiator and its my prediction that one of them will acquire security technology to do this.   Its often easier to acquire vs. try and built it yourself given you don't currently have the expertise.

So who's problem is it to solve??  Virtualization Vendors or Security Vendors??

I see the finger pointing game starting!

-John Peterson

CTO / Montego Networks

Netflow visibility inside Virtual Environments

I blogged on this topic a few weeks ago but given the huge interest in this topic I’ve decided to blog on it again. One of the major concerns in virtualized environments is the lack of visibility of the communication between virtual machines. With this lack of visibility a number of challenges start to appear such as security, monitoring and capacity planning.  It’s hard to secure what you can’t see or don’t know about and it’s hard to determine when you need to add more resources when you don’t have a clear picture into what applications are consuming them.

This problem is widely known and as a result there are a few companies that are starting to pop up that are building Virtual Network Visibility tools. But should you buy yet another tool to gain visibility into your Virtual Network communication when you may already have a tool for your physical network? Should you have to have separate tools for your physical network and virtual network?

One common method of gaining visibility into network communication is through a technology called Netflow. Netflow was originally developed by Cisco Systems but has since become a defacto standard for Network Monitoring and Network Behavioral Analysis. Companies such as Lancope, Mazu Networks, Plixer International and Arbor Networks all have products that enable network visibility, monitoring and analysis. These tools typicaly take Netflow feeds from a switch of some sort.  Knowing that some of these tools may have already been deployed in physical environments, IT staff will now need to consider  whether or not to buy new visibility tools to give them visibility into their virtual environment communication or try and leverage existing solutions already deployed in their physical environments.

Up until recently there has been no elegant way to export Netflow records from virtual environments such as VMWare and as a result companies have had consider purchasing new visibility tools that would often antiquate their existing physical solutions. This is due to their migration from physical environments to virtual environments.

Montego Networks now has Netflow capability in its HyperSwitch product which runs inside VMWare and enables security, visibility and control for the virtual environment by leveraging existing tools. Through its API’s and standards based methods Montego can enable customers to leverage existing infrastructure purchases to gain visibility and control within the virtual environment.

So, enough of the commercial and lets get on to the technical meat of this new Netflow enablement within the virtual environment.

Let’s say that you have a virtual machine that is infected with a BOT and it is communicating to a Command and Control Site of a BOT-Army. How would you know this? Well, you could have a NetFlow tap at a network switch close to your internet connection. But what if you have some sort of communication between VM’s on a non standard port that you are not aware of? Maybe a machine got infected and is sending data from the database virtual machine to a web server virtual machine and then feeding that info from the web server virtual machine to the internet. Your Netflow tap on the internet facing switch would see traffic coming from the web server virtual machine to the internet but wouldn’t see that data was being taken from the database, put on the web server and then fed out to the internet. Kinda tricky to hunt this problem down isn’t it?

So, whats needed is Netflow all the way into the virtual environment so that it can be fed to the same tools in your physical environment for easy correlation.

Take a look at the attached screen shot which shows Lancope and Montego Networks in action.

<---Click to Enlarge

With this level of visibility now you can see who is talking to who, when are they communicating and how much traffic is being consumed by which applications and which virtual machines.  This can now all be done by leveraging existing Netflow analytics tools.

This screen shot is showing flow data of Virtual Machines talking either to the Internet or to other virtual machines within the same environment.  You will notice from the flow data that one of the Virtual Machines has iTunes running on it.  An IT Administrator may have not sanctioned this or even know about it.  But with Flow records you can now see!  Like a new pair of glasses for your virtual environment.  With this visibility you can now go in to the Montego HyperSwitch and enable a firewall policy to block that iTunes traffic as an example.
 

Lancope is just one example here and its important to note that, because Netflow is a defacto standard for this type of visibility, other tools such as those from Mazu Networks, Plixer International and others can be used as well.  They all have their unique advantages and disadvantages but the point here is that dependent upon your prior network purchases in this area you will now be able to leverage existing tools vs. having to purchase new ones in many cases.

Check out Montego Networks at Networld Interop 2008 in the Lancope booth to see the solution in action!

John Peterson
CTO Montego Networks

Securing Virtual Environments Through Partnerships

I’m back from the RSA 2008 Security Show in San Francisco and it was another great year of business development activity for security vendors. It felt like there was a decent amount of end user customers at the show but a lot more vendors touting their wares and looking to do work with each other. I sat and listened to many vendors complain about this and listened to them complain about how they spend money year after year for these shows and rarely get to talk to customers. It felt to them that they hear more from other vendors that come up to their booth asking about partnering or OEM’ing their technology. Well, this does get old pretty fast when you are looking to sell product to justify your existence but for me it was refreshing to talk with other companies about partnering. I had the opportunity to talk to customers also but it was really exciting for me to have partnership discussions.

Why? Well over at Montego Networks where we are focusing on securing a new type of network (one that’s virtual) we believe in security through partnerships. Securing virtual environments is like exploring new frontier or a planned venture to Mars. Research scientists, chemists, doctors, collective minds and in this case a unity of security vendors we feel is the best approach to getting ready for this venture to the new Virtual World.

Virtual Environments need to be studied jointly in order to understand the new security risks, performance impacts and how to effectively secure it.  Montego Networks plans to do that and has announced its HyperVSecurity Alliance at RSA and has joined forces with Cyberoam, Lancope StillSecure and Plixer International in an effort to provide Anti-Malware, Network Access Control, Intrusion Prevention, Behavioral Analysis and Network Monitoring for the virtual environment.

See: 

http://www.montegonetworks.com/node/54

http://www.eweek.com/c/a/Security/Partnerships-are-Key-in-Virtualization-Security/ 

By establishing this type of alliance research engineers and vendors will be able to journey to the new Virtual Datacenter with all of the needed components and insight on securing networks. At the epicenter of this alliance is a security frame work designed by Montego Networks that allows various technologies to plug in to the center of the virtual environment which is the switching infrastructure.

Through Montego Networks HyperSwitch, which has the ability see virtual network communication between systems (virtual desktops & servers), a frame work is created that allows for user defined policy that can send traffic off to various places. An example of this is via the HyperSwitches Policy Based Switching engine which allows a user to create a policy that dictates that all email traffic will be directed to an Anti-Virus Gateway or its NetFlow capability which exports flow information to a Behavioral Analysis Engine. 

After these various systems do what they do with the data, they are also able to respond back to the frame work via an API called NSCP (Network Security Control Protocol) to instruct it to tack appropriate action. This could be an IDS system invoking a firewall policy or a Behavioral Analysis system telling the frame work to throttle back (slow down) a users traffic flow. The possibilities are limitless!

So, much like the frontier to the USA from England where we needed Doctors, Lawyers, Law Enforcement, Builders and Farmers, virtualization needs a coalition of security forces that can provide Anti-Virus, IPS, Firewall, Network Monitoring, Behavioral Analysis, etc. etc.   

The goal is to all co-exist in the virtual environment vs. fight for the same piece of land. I think this makes sense because all is needed in the virtual world!

Stay tuned, as the alliance will get bigger and stronger and give customers choice and independence as they look to secure the virtual datacenter. Learn your ABC’s! Anything But 100% Cisco, Let Freedom Ring!

 

NetFlow and Visibility in the Virtual Environment

With so much talk about securing communications within the virtual environment and potential hypervisor based attacks, we sometimes forget about the visibility problem within the virtual environment.

Today's blog is about just that. Visibility!

We've all probably heard the saying, its hard to secure what you can't see and that understanding your environment is the first step to security.  Well, with virtualization, understanding whats going on in your virtual environment is even a challenge.  Because virtual switches are not as feature rich as physical switches we are left unable to do many of the things we've done in the physical world that enables visibility.  One of the features that exists in physical switches that is commonly used as a security and visibility tool is Netflow.

Over the past week or so I've begun speaking with VMWare customers and Netflow enabled vendors like Mazu Networks (who has an awesome product) and they both have been struggling to figure out an elegant way of gaining visibility into the VM to VM communication within the virtual infrastructure.  You see, in the physical world people turn on Netflow on their switches so that they can do reporting and behavioral analysis but in the virtual world there is no Netflow enabled virtual switch (at least not until now - I'll get to that in a moment). 

So for companies like Mazu Networks and Lancope and for their customer base that is migrating parts of their network to virtual networks, there exists a significant challenge to the business of behavioral based analysis.  Investment in tools that use Netflow enabled switches now starts to become obsolete for parts of the network that is now virtual. 

We've heard vendors to date talk about Virtual Patch Management, Virtual Firewall, Virtual IPS but these talks leave customers confused on what they really need and doesn't necessarily solve all of the security and visibility challenges they thought they had already addressed.  Hmm.. Maybe whats needed is the ability to enable all of these things.  What about Virtual Behavioral Analysis!  Wow, another Virtual Security product that we haven't thought about!  Maybe someone could just virtualize a Behavioral Analysis product and run it inside VMWare,  put the world "Virtual" in front of the name of the technology and call it a day?  Hmmm.. Thats probably not a good idea due to the performance impacts you could encounter.  One of the biggest challenges with security is how to do all of the things we've done in the physical world in the virtual world without impacting performance.

So, back to visibility... Netflow is a technology originally invented by Cisco that sends flow records to a listening device that does some data crunching on those flow records to give you a visual picture of the data in the network.  With this data you can determine abnormalities in traffic patterns, see who the top talkers are in a network as well as home in on what network applications are running in the environment.  With this information you are now better equipped with the right level of knowledge of the environment to start putting security controls in place.  The problem is that it doesnt exist in the virtual switch provided by VMWare, Citrix, etc..

So, how can we do Netflow in the virtual environment so that we can have "Virtual Behavioral Based Analysis"?  Well after looking into this problem and talking with Netflow experts at Mazu Networks, Montego Networks has now enabled Netflow in its Virtual Security Switch. 

Heres how it works:

VM1 is sending traffic to VM2 and VM3 is sending traffic to VM9 and VM5 is sending traffic to the physical network.  Well, for the VM to VM communication, any physical Mazu or Lancope boxes will have either no visibility or have to get creative and put a solution in place thats not optimal or practical.  Vendors in this space are also probably concerned about shrinking revenue if more of the physical network starts to erode away as virtual networks take off and customers are probably concerned about investment in products that are no longer able to provide maximum value.

So as traffic enters Montego Network's Virtual Security Switch we will send a Flow record to a Mazu Networks or a like listening device on the physical network.  Since we see VM to VM communication we can extend this capability to 3rd parties by simply sending them a Netflow record for them to analyze and tada!  You have Behavioral Analysis for your virtual environment.  Notice the Netflow text on the bellow graphic.  It depicts collecting data from the virtual servers and sending a Netflow record somewhere.

 

Montego Networks spotted on radar

 

Montego Networks has been flying under radar for the past year and this week increased its elevation just enough to be seen on the virtualization industries radar detector. Montego Network’s announcement of securing virtual network communications between VM’s has everyone buzzing but what has caught most people’s attention is Montego Network’s technology that enables 3^rd party security vendors to do the same thing (VM to VM). Now, I’m the CTO of Montego Networks, so my comments here are a bit biased but also first hand. So, when I tell you that it’s been a great announcement, I truelly feel it has. Everyone I have spoken with in the analyst and press community thus far has embraced the idea of security vendors working together to provide a solid solution vs. every vendor trying to be all things to everybody.

So, what does this really mean and how does it work?

Let’s say you have VM1 (Virtual Machine) and VM2 (Virtual Machine) and they need to be able to transfer data between each other but only once or twice a week. This means you can’t have them 100% isolated. Because you have a communication need between them, it probably makes sense to only open up the channels (TCP/UDP Ports) that they need to communicate on vs. opening up all channels. This helps mitigate exposure. So, let’s say you open up port 6667 and only port 6667 for them to communicate with each other. Well, this is now a bit more secure than the other option of leaving all ports open but let’s say this is a very very critical server and you want deep packet inspection done on all of its traffic. The reason you want to do this is because there is the potential that worms and BOTnet communication could occur over this port 6667 but the only way to determine that is to do deep packet inspection.  I am using port 6667 as the example because I spoke with someone that had a real live case where one of their Linux VM's got infected with this BOTnet:  http://www.energymech.net/ on port 6667

Now, I could put some sort of virtual IPS product inline and look at Physical to Virtual communication for all of the VM’s (VM1, VM2, VM3, VM4, etc.) but I don’t care to take that kind of performance hit and I also already have a physical IPS handling Physical to Virtual. What I really needs is IPS between the VM’s which I haven’t been able to find from any vendor yet and even if I did find such a solution on the market I don’t care to take the performance hit of doing IPS between ALL VM’s.

So, now that you understand the challenge, how can Montego help and what’s this HyperVSecurity thing they talked about in their press release that allows other vendors to interoperate with them. Well, with Montego’s Policy Based Switching technology you, the administrator can control what types of VM to VM traffic you would like to have inspected by a 3^rd party security solution. I would simply set up a policy that says VM1 to VM2 on port 6667 will have its traffic sent to a StillSecure virtual IPS product and once a week when that traffic starts to flow it will be sent over to the IPS product for further inspection. Or if traffic starts to flow outside that once a week norm, it will still be sent for inspection. This way if some attacker tries to get in on that port he will have to make sure he can get past the IPS that now is able to VM to VM IPS.

Pretty cool huh? I think so.

 Now, back to Montego coming out of stealth mode…

You’ll start to hear and see a lot more innovation coming out of Montego Networks now that we’ve popped slightly above radar and the industry knows we are here but is scrambling trying to figure out what exactly we do, how sustainable will this new startup be and if we really have what we say we have. I’m certain competing companies will throw FUD and make all sorts of comments about what we do, how it performs, etc. etc. and all I can say is to just keep an eye on the after burners because we are starting to get lift off.

-JP

Virtual Environments will be more secure than their physical counter parts by 2010

Montego Networks Prediction:

Virtual Environments will be more secure than their physical counter parts by 2010.

Neil McDonald of Gartner reported in 2007 that throughout 2009, 60% of virtual environment deployments would be less secure than their physical counter parts.

Although I tend to believe Neil’s prediction I’m a bit optimistic about the markets awareness of the security concerns within virtualized environments and feel companies will start to address those concerns by 2009. I also believe that by the end of 2009 the majority of companies virtualizing will have built virtualized environments that are more secure than their physical counter parts.

Now, you may be thinking I’m either crazy or that I’m just one of these guys that just states the opposite of what someone else says!

Well, not at all. I’ve been studying the virtual security market for some time now and after talking with many companies that are deploying virtualization I’m starting to get the sense that people get it (security). It’s pretty evident that when people are made aware of what seems to be the obvious (security), that something clicks and they get it right away. In fact, many times the light bulbs start turning on and people start thinking about more creative ways to secure severs by taking advantage of virtualization which enables them to do things they’ve never been able to do before. 

So, although I agree that there has been this issue of security being once again forgotten and that 60% of virtual environments will be less secure up until 2009, I’m not so sure I’m going to underestimate the market and think that this pattern will continue much longer after that.

Take a look at the following graphic and it depicts the various layers in a network. History has proven itself time and time again that a new network layer is built first and security always comes along afterwards.

 

Well, one of the challenges we’ve seen with these physical networks is that it’s pretty costly, time consuming and a burden to purchase, install and administer security. Then once it’s in place and being run, you have to fork lift upgrade certain parts of your security infrastructure due to bandwidth demands and changes in application security concerns.

What virtualization brings to the table is not only cost savings for server consolidation, power consumption and datacenter space but the ability to do all of those things for parts of your security infrastructure as well.

Imagine instead of having to deploy engineers to install 20 firewalls across your datacenter, you could sit from a single workstation with a couple of guys and install 20 firewalls in hours vs. days. The reason this is possible is because now firewalls have just went virtual! You can roll them out as software images or virtual appliances without leaving the comfort of your cubical. 

Imagine being able to “virtual-lift upgrade” vs. “fork-lift upgrade” a new firewall, UTM appliance, IPS or whatever by simply powering off a Firewall Virtual Machine and powering on a new one.  Imagine being able to improve your performance by taking advantage of the multi-core processing and blade server computing trends vs. waiting for the next super fast security ASIC chip.

In the past it’s been difficult to get security as close as possible to the servers and desktops without having to deploy host based solutions. The reason for this is because we have been constrained by the physical limitations of our hardware purchases from the likes of Cisco, Extreme and Foundry. Then for vendors that have thought about putting security in a switch there has always been the price per port debate. Also, many don't want to take the risk and replace Cisco for a new startup building a new switch (ie. Force 10's Switch + IPS product).  Typically switching ports are cheap and security is more expensive and when trying to combine the two, you end up with a switch that costs a lot of money. So imagine having a 200+ port switch with a Firewall built in for $300 bucks. How could this be so? Because its virtual, and because its 100% software.

Did he just elude to a firewall for every port?  Does each Server or Desktop have firewalling between every other Server & Desktop on the same switch?  Absolutely! all because of virtualization!

Software makes it easier to bring the price per port down. When things are in software you can deploy multiple copies of them to scale your network capacity without breaking the bank. Virtualization also allows you to do things like “Freeze” and “Thaw” servers and desktops automatically when vulnerability is detected. If a denial of service is occurring against a Virtual Server you can always VMotion that server to a network with more capacity without an administrator having to lift a finger. Imagine an attack happening on a machine and instead of it being quarantined it makes a snapshot image of the infected machine and freezes it in its current bad state so you can go back and analyze how someone broke in. As you can see, there are lots of new capabilities brought to the security round table.

Virtualization will make security solutions even more powerful and increase the adoption rate of security in general due to the massive cost savings that can be appreciated through virtualization. For these reasons I see the market quickly leveraging virtualization to make Virtual Environments more Secure than their counter parts. Virtualization will enable the innovations in security that has been since UTM and Reputation based Anti-Spam.

VMWare, Virtual Iron, Citrix and others, thanks from the security industry for the innovation!

John Peterson, Montego Networks, Co-Founder & CTO

Network World Focus on Security in 3/17/08 issue

It looks like virtual security is getting some attention this week as seen on the front page of Network World.  There are multiple articles in this issue that talk about the security challenges in the virtual environment.  I suggest everyone interested in the topic take a read.

After reading the articles, I did want to put out a short blog today to bring clarity to some of the vendor hype and mis-information that has been floating around lately.  I've heard many people say that Reflex, Blue Lane and Catbird provide security between virtual machines.  This isn't true.  What these vendors do is provide "monitoring" between virtual machines as stated on page 48 of Network World's article on virtual security.  What monitoring gives you in the case of Reflex is IDS (detection) and while useful it does not provide what many THINK it does.  Many think it provides prevention. 

<-- Click to enlarge

The way they provide monitoring is by taking a port on the virtual switch and enabling "promiscuous mode" and hanging a virtual security appliance off of that port.  Its in a sense like setting up a span port on a physical switch and mirroring all traffic out that span port.

This is definitely helpful from a visibility perspective  but does not give  you  VM to VM isolation or VM to VM intrusion prevention.  Take a look at the attached graphic from Reflex.  They displayed this graphic today on a webinar about PCI compliance.  You'll notice the VM's on the right can all talk to each other and the VM's on the left can all talk to each other.

<--Click to Enlarge

Now from the picture you can see that it does provide prevention from the GROUP of VM's on the left if they try to talk to the GROUP of VM's on the right.  I've yet to see anyone deploy their VM's like this however it does make sense to put your VM's in trust Zones. 

I am of the opinion however to put every server on their own trust zones and set up policy between those zones.

-JP

High Availability Security In Your Virtual Environment

How many times have security products been the blame for network outages?  Many right? 

If something goes down and the network team gets a call, they immediately point their finger at the Firewall.  If a user can't access something on the network, its the Firewall.  If something is running slow on the network, guess what! 

Its the firewall.

And with Intrusion Prevention products, because they were very unstable during the early years and would crash or generate false positives a lot, customers started demanding that these devices had some failure mechanisms in them.  Customers demanded "Fail Open".  Fail Open to a security guy doesn't make a whole lot of sense because it basically says, if there is a problem with the metal detector at the airport, it should just "Fail Open" and let everyone into the gate area to board airplanes!

I'd rather block all traffic until I know it was secure, but I live in a world where most people don't think like me.  So.... Why the heck am I blogging about this in a virtualization blog?

Well, I know that Virtual Networks function much like Physical Networks and since network engineers don't always trust security devices I understand that the same set of requirements placed on physical security products will be placed on virtual security products.

Why wouldn't the networking guys demand that virtual security products have either "Fail Open" or what I feel is a better solution "Fail Over".

"Fail Open" is not really possible with virtual security products because true fail open means that you have some sort of physical relay or in the case of optical networks, mirrors that short circuit software to allow bits to bypass and flow around the software application.

"Fail Over" however is possible and customers are going to ask for the same things I believe when it comes to uptime on a virtual network as they do a physical network.

Take a look at the attached picture.  It depicts a software solution that has two firewall type products running in Active / Passive. 

CLICK PIC TO ENLARGE

So, as you are looking at security solutions for your virtual environment, you should ask the question of whether or not they provide any high availability and if so, what level of high availability.  Active / Active, Active / Passive, Statefull, Stateless, and everything you've asked of your physical vendors.

My guess is that if you ask and they don't have it, they will start developing it and marketing its ability.  Its a battle that cant be won completely.  Customers will always want high availability be it virtual or physical.

Until the next post...

JP

Virtual Environments will be more secure than their physical counter parts by 2010

Montego Networks Prediction:

Virtual Environments will be more secure than their physical counter parts by 2010.

Neil McDonald of Gartner reported in 2007 that throughout 2009, 60% of virtual environment deployments would be less secure than their physical counter parts.

Although I tend to believe Neil’s prediction I’m a bit optimistic about the markets awareness of the security concerns within virtualized environments and feel companies will start to address those concerns by 2009. I also believe that by the end of 2009 the majority of companies virtualizing will have built virtualized environments that are more secure than their physical counter parts.

Now, you may be thinking I’m either crazy or that I’m just one of these guys that just states the opposite of what someone else says!

Well, not at all. I’ve been studying the virtual security market for some time now and after talking with many companies that are deploying virtualization I’m starting to get the sense that people get it (security). It’s pretty evident that when people are made aware of what seems to be the obvious (security), that something clicks and they get it right away. In fact, many times the light bulbs start turning on and people start thinking about more creative ways to secure severs by taking advantage of virtualization which enables them to do things they’ve never been able to do before. 

So, although I agree that there has been this issue of security being once again forgotten and that 60% of virtual environments will be less secure up until 2009, I’m not so sure I’m going to underestimate the market and think that this pattern will continue much longer after that.

Take a look at the following graphic and it depicts the various layers in a network. History has proven itself time and time again that a new network layer is built first and security always comes along afterwards.

 

Well, one of the challenges we’ve seen with these physical networks is that it’s pretty costly, time consuming and a burden to purchase, install and administer security. Then once it’s in place and being run, you have to fork lift upgrade certain parts of your security infrastructure due to bandwidth demands and changes in application security concerns.

What virtualization brings to the table is not only cost savings for server consolidation, power consumption and datacenter space but the ability to do all of those things for parts of your security infrastructure as well.

Imagine instead of having to deploy engineers to install 20 firewalls across your datacenter, you could sit from a single workstation with a couple of guys and install 20 firewalls in hours vs. days. The reason this is possible is because now firewalls have just went virtual! You can roll them out as software images or virtual appliances without leaving the comfort of your cubical. 

Imagine being able to “virtual-lift upgrade” vs. “fork-lift upgrade” a new firewall, UTM appliance, IPS or whatever by simply powering off a Firewall Virtual Machine and powering on a new one.  Imagine being able to improve your performance by taking advantage of the multi-core processing and blade server computing trends vs. waiting for the next super fast security ASIC chip.

In the past it’s been difficult to get security as close as possible to the servers and desktops without having to deploy host based solutions. The reason for this is because we have been constrained by the physical limitations of our hardware purchases from the likes of Cisco, Extreme and Foundry. Then for vendors that have thought about putting security in a switch there has always been the price per port debate. Also, many don't want to take the risk and replace Cisco for a new startup building a new switch (ie. Force 10's Switch + IPS product).  Typically switching ports are cheap and security is more expensive and when trying to combine the two, you end up with a switch that costs a lot of money. So imagine having a 200+ port switch with a Firewall built in for $300 bucks. How could this be so? Because its virtual, and because its 100% software.

Did he just elude to a firewall for every port?  Does each Server or Desktop have firewalling between every other Server & Desktop on the same switch?  Absolutely! all because of virtualization!

Software makes it easier to bring the price per port down. When things are in software you can deploy multiple copies of them to scale your network capacity without breaking the bank. Virtualization also allows you to do things like “Freeze” and “Thaw” servers and desktops automatically when vulnerability is detected. If a denial of service is occurring against a Virtual Server you can always VMotion that server to a network with more capacity without an administrator having to lift a finger. Imagine an attack happening on a machine and instead of it being quarantined it makes a snapshot image of the infected machine and freezes it in its current bad state so you can go back and analyze how someone broke in. As you can see, there are lots of new capabilities brought to the security round table.

Virtualization will make security solutions even more powerful and increase the adoption rate of security in general due to the massive cost savings that can be appreciated through virtualization. For these reasons I see the market quickly leveraging virtualization to make Virtual Environments more Secure than their counter parts. Virtualization will enable the innovations in security that has been since UTM and Reputation based Anti-Spam.

VMWare, Virtual Iron, Citrix and others, thanks from the security industry for the innovation!

John Peterson, Montego Networks, Co-Founder & CTO