Friday, November 12, 2010

Cisco switches and virtual machines

A very interesting thing happened recently. A server running on centos , was setup as a base OS for 3 windows virtual machines - a domain controller, a database server, and an antivirus server. Last week the database server stopped responding. A restart didn't solve the problem. Even pinging didn't work. So the admin decided to restart the physical server itself. After the restart all the virtual machines stopped responding! A few more restarts , and checks on the network settings of the physical server, vm and there was no change. The admin disabled the physical network card and restarted the vms and servers. After re enabling the card suddenly the dc started responding and the physical server stopped responding! That's when the admin decided to take a break and re-assess the situation.
He had one physical server, and 3 vms. One of the vms was responding on the LAN. The physical server and the other two vms were not responding to any requests. But he had done restarts before and this was a new thing. The only change he could point out was that he had replaced one of the 3com switches with a cisco 2900 managed switch. This was to setup a gigabit uplink. It was a shot in the dark but we removed the server network cable from the cisco switch and plugged it into another 3com switch in the rack, and presto , all the servers and vms started responding. :) after doing a bit of reading, a particular setting on the cisco sets it to accept traffic only from one IP per port. that's why even though the dc was responding, the physical server was not allowed to respond.

Now that's what I call security. Good stuff, cisco!
Sent from my BlackBerry® smartphone from VIVA

2 comments:

  1. that is one unique issue!

    my question, if the setting allowed traffic only from one IP per port, how was this cisco router used for uplink connection? didn't that give an issue?

    ReplyDelete
  2. Could be that the uplink doesnt have this setting, otherwise whats the point of an uplink ? :)

    ReplyDelete