For a customer of mine, we are in the process of implementing a vSAN environment, where stretched clustering is used. They use a witness location, which is (independently) connected to both data-locations. For the vSAN networking, they do not use VRRP or another way to stretch there routing, but they do use stretched layer-2 networking.

So their environment looks like this:

In order to make sure that HA is handled correctly, it is advised to create a gateway-address within the vSAN network, in order to make sure that connectivity on the storage level is used to determine isolation. More information on this, in the blog of Duncan Epping:

http://www.yellow-bricks.com/2017/11/08/vsphere-ha-heartbeat-datastores-isolation-address-vsan/ 

So we created a VRF (Virtual Routing and Forwarding, a method to create a virtual router, within a physical entitiy (in this case a switch)) on both switch-stacks on both data-locations, in order to make sure that connectivity from the host to the switch-stack can be tested. With the use of a VRF we are still able to make the vSAN network unconnected to the rest of the network.

The switch stack on data-location 1, used ip-address 10.17.31.253 and the switch stack on location 2, used ip-address 10.17.31.254 (fictional ip-addresses, of course). Both addresses where than added as isolation addresses, to test the connectivity of the hosts to the environment. So when a host is unable to connect to both ip-addresses it will assume it is isolated.

Then we needed to add the witness network, but in order to make sure that connectivity was correctly configured, we needed a way to use the correct connection to connect from the witness to the hosts on the different data-locations. So traffic to and from the witness location to location 1 needed to use the connection to location 1 and traffic to and from location 2, needed to use the connection to location 2.

We created two transit VLAN’s, one from the witness location to location 1 and one from the witness location to location 2. In the routing table on the witness-location switch, we created static routes to the hosts on location 1, going through the transit network towards location 1 and the same for location 2.

On the VRF’s on the data-locations, we created static routes for the witness-vlan through  the directly connected transit networks. So when a host in location 1 tries to connect to the witness location, it will use the transit VLAN from location 1 to the witness location (and vice versa). And when a host in location 2 tries to connect to the witness location, it will use the transit VLAN from location 2 to the witness location (and vice versa).

Of course, we also needed to take into account the failure a connection from the witness locations to one of the data locations, so we also created static routes to the same entities with a higher cost, through the “other” data-location. So when one of the connections to the witness locations would fail, traffic would be rerouted through the other data-location.

In the end, the environment looks like this:

So, it might not win the beauty contest, but it gets the job done (in Dutch that’s a saying ;)). It is also not very scalable, but for a small environment, it works.