So I've learned this week that vSphere 5.1 VmKernels no longer support multiple gateways. This is causing me some confusion with how to properly setup my networking for my SAN. I thought it was best practices to seperate the traffic so I created seperate vlans for management and for data (SAN) traffic. Since they are in different subnets they both have their own gateway. When trying to configure this I started having quite a bit of problems before I realized that the gateway must stay the same. I contacted VMware and originally they told me the gateways can be changed before finally stating they were incorrect and that there can only be 1 gateway. The response I received from them at that point is that I do in fact want to split the vlans and to just leave the gateway on my vmkernel for san traffic alone they stated that as long as my vlans were properly setup that vmware would just know where to send the data and it would not matter. When this "magically" was not working they told me I must have a vlan issue and that they could not help me.
Could anyone give me some insight as to what the best method is for this? I found an article that states you can manually add a second gateway via the CLI but when I tried I received an error stating the route already existed.
Vlan 18 (172.27.18.x/24 w/gw 172.27.18.1) - management
Vlan 40 (172.27.40.x/24 w/gw 172.27.40.1) - data/SAN
Any help would be greatly appreciated.