NSX Controllers east to west traffic.

In the typical network space with VLANs where we have our core,distribution and access layer. We typically run the default gateway as a HSRP address. Whichever router/switch is the active n its pair the mac address is sent to each individual hosts and they arp for their default gateway which would be the HSRP address. Otherwise, for east to west traffic in the same VLAN a arp request is handled by the VM to the physical network to ask how to find another mac address it is trying to talk to.

Thinking differently in the network overlay world since we are overlaying networks and tunneling all traffic something has to sit off to the side and tell us how to reach our default gateway and how to get to guests within the same subnet. This concept is known as a controller. A controller will build routes and tell a ESXi host how to find the mac address of a corresponding VM or how to reach its default gateway which is otherwise known as a LIF or logical interface that lives on either a Logical distributed router or a edge service gateway. The controller is responsible for telling the ESXi host where to send the traffic next hop wise.

In this example I am showing two physical hosts, two VM’s in the same VXLAN and two controllers.


So lets first start a ping from Tenant-A-1 to Tenant-A-2.


So everything is pinging YAY.

Now how does this work? If you recall I said the controllers are whats making this possible. One of the controllers has to tell the ESXi host where Tenant-A-1 lives how to get to Tenant-A-2.

So lets log onto the controllers and run some commands to figure out how it knows this is possible.


What the command show control-cluster logical-switches vtep-table 5000 is telling me is I have 2 hosts that participate in that VXLAN/VNI 5000. and The first host will be referenced as Connection-ID 3. The second Connection-ID 2. This will make sense in a few seconds.

Okay, so that is great thats how that works now how does the ESXi host know where to send traffic to while these are pinging?Arp-5

The following two commands explain it all here. The first shows the mac addresses of both VM’s on VNI/VXLAN 5000. The second will explain how to resolve arp to ip the same way a normal Layer 3 router or switch would. So the controller tells the host how to get to each corresponding VM it needs to get to. So lets VMotion everything to live off of and see what happens.


There you have it the controller knows how to get to the two mac addresses of either tenant-a-1 or tenant-a-2.

Post a comment or leave a trackback: Trackback URL.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: