We've probably all heard that you can't secure what you can't see and that statement is even more profound when it comes to virtual environments. This is because it is extremely challenging to see what is going on at a micro vs. macro level within a virtual environments network. The virtualization vendors such as VMWare and Citrix have provided embedded tools into their management consoles that show a macro level of visibility but its not enough to identify security events in the environment. Take a look at the attached picture. It simply shows VMWare's ability to monitor virtual network performance statistics from a bits per second perspective.
With only this level of detail how can one determine which network applications are causing spikes. Is it FTP traffic that is occuring at a high volume at an unuseal time of day? If that were occuring, could that be indicative of either a breach or some sort of problem? What if FTP isn't even an authorized service in the virtual environment but there is a high volume of it? Did someone install a rouge FTP service so they could steal information from the server at will?
These types of questions can't really be answered without a micro level of detail into the packets flowing in, out and within the virtual environment. Now, what I am highlighting is not security in the traditional sense of prevention but using visibility as a means to first identify, then pin point the source of an issue so that it can properly be mitigated. Having constant visibility can also ensure that other security products in the environment are performing as expected. What if a Montego HyperSwitch with firewalling enabled is configured with many policies but someone forgot to create an FTP block policy. One could think they are protected from rouge FTP services transmiting data out of the network, but without constant visibility monitoring, can you be certain?
Some vendors, namely Reflex Security will get you to believe that their IPS / IDS solution that is inline and running in the virtual environment is the right and only approach. Or they will tell you to hang a virtual IDS off a span port in the virtual environment and you will at least have visibility into the attacks that are taking place. Well, sure... You now have attack visibility but at the performance cost of your virtual environment. Signature matching technologies are great, I'm a huge believer; however they don't scale very well in shared computing environments such as virtual ones. IDS systems also don't typically track protocol and network service (FTP, HTTP, etc.) utilizations; which is another important part of visibility.
So, what do we do to gain visibility without the performance headache? Well, for starters its probably best to put your IDS/IPS solutions in the physical environment where performance will be less of a concern. In fact, you can span a virtual switch's traffic out to a physical NIC as easy as you can to a virtual one. So why do it virtual and have to pay a 60% CPU utilization tax? Another solution is to IDS inspect only the things you care about. Why IDS inspect SSL traffic if you know your solution can't unencrypt SSL. Its just a waste of compute cycles isnt it? Policy based switching helps you with directing only the things you care about to an IDS (attack visualization product). Montego's HyperSwitch also can help you with the traffic redirection of only the things you care about.
Another method of visibility which I tend to be a fan of is one of packet analysis (aka NetFlow). NetFlow was invented by Cisco some time ago and has gained popularity in the physical world and definately has a use in the virtual world. NetFlow is lightweight. Let me say that again, its light weight! It only sends a summation of packet detail to an analytical engine which can do some number crunching, packet comparison, etc. etc. to make some sense out of whats going on. Lancope, an Atlanta based visibility company that provides Network Visibility, Security Visibility and User Visibility has this tool on their website that is a Netflow Bandwidth calculator. You'll see from playing with this ( http://www.lancope.com/netflowcalculator.aspx ) calculator that it doesn't consume a lot of network bandwidth to transmit these network accounting records. It also doesn't cause a lot of CPU overhead to send these records to an analytical engine sitting somewhere in the network.
Lancope's analytical engines have the ability to do the following for you within your virtual environment:
- •Monitor and Alert network behavior of VMs
- •Track Vmotion movement of VMs accross physical servers
- •Monitor and Alert on communication between VMs
- •Identify users accessing VMs
- •Identify unauthorized or rouge VMs
- •Monitor and Alert when VM’s go online or offline
- •Identify network services running on VMs
- •Monitor Network / Application performance of VMs
Display active hosts accessing VMs
...and probably a slew of other things I'm not aware of. A screen shot of their product is bellow:
You'll notice from the screenshot that you are able to visualize who is talking to who, how much traffic they have sent and received and something called a concern index (not seen on this screenshot).
Now, a concern index is a number that increases as Lancopes analytical engines monitor suspicious activity on a session. A high counter can be indicative of a security problem. Its another way of identifying (visualizing) compromised hosts (virtual machines) without having to do signature matching like a heavy weight IPS engine. Example: Lets say you have a VM that has a BOT on it and is "owned". The Lancope product is monitoring this long life session. Let's say that session is established for several hours or maybe even days or months. Lets also say that the conversation appears to be mostly unidirectional from a public ip address not belonging to your enterprise. Lancope would increase a the concern index on this since this server hasn't typically had this type of behavior. Once the concern index reached a certain level it could then fire off an email, send you a text message or something saying: Warning, Warning, Danger, Danger Will Robinson!!! You're virtual server may be infected with a BOT, please investigate immediately!!!
This example is VISIBILITY which helps you with SECURITY. There are a number of other things you can do with NetFlow and Lancope products that have less to do with security and more to do with operational efficiencies. Things like, helping you answer questions of: How do I know what network applications are taking up the most bandwidth? When should I move those applications over to a server with more horsepower? When did these VM's vmotion over here and was there a traffic condition / CPU condition that caused that to occur? I could go on and on but thats a topic for another blog entry.
So, my suggestion is to take a look at what NetFlow has to offer. Montego Networks supports NetFlow transmission and Lancope supports NetFlow analytics and with both you can regain what was lost visibility.
I hope this was helpful to you all!