Routing with NSX-T (part 1)
As someone who has worked with NSX-V for quite some time, routing within NSX-T is a little bit difficult to grasp. The terminology might be only slightly different, the concepts are completely different.
Within NSX-V, you had E-W routing, through a DLR and N-S routing through one or more ESG’s, either in HA or in ECMP. That was all pretty straight forward. The addition of statefull service could only be performed by ESG’s and if so, the ESG’s would not be able to be part of an ECMP based routing-engine.
All pretty easy concepts.
Within NSX-T this is different. There are multiple ways to look at routing (and stateful services). There are a couple of topics to talk about, all relevant to routing within NSX-T. Let’s review the topics:
- Tiering: Tier-0 and Tier-1
- Router Types: Logical Router / Service Router / Distributed Router
- Edge Node / Edge Node Cluster
First let’s look at the tiers. Within NSX-V you had the possibility to create a multi-tier routing entity, for instance if you where a service provider or had other reasons to have multiple tiers. All was done through the use of multiple ESG’s, put together into a hierarchical way.
Within NSX-T you get two tiers, but they are explicitly defined within the product. So you can easily deploy your routing in a tiered fashion. It is however not necessary to do this, if you want to, you can use tier-0 for routing. However within tier-0 there is no place for services Load Balancing, so if you want to do some load balancing, you have to add some tier-1 entities. NAT can be done on both tier-0 and tier-1, VPN is done on tier-0.
So essentially, you deploy one or more tier-0 gateways (the new name from 2.4 and onwards) and after that connect one or multiple tier-1 gateways to that tier-0 gateway.
The routing is then done in a hierarchical way. The tier-1 has to be connected to a tier-0 and this will lead to an (automatically) created tier-transit-link, which will be used to exchange routing information between the tiers.
So within NSX-T we get some distinct router types. The following entities can be distinguished:
- Logical Router
- Distributed Router
- Service Router
The Logical Router (LR) is basically a combination of the Distributed Router (DR) and the Service Router (SR). The Distributed Router, like the DLR in NSX-V, is responsible for routing traffic in a distributed (duh) fashion. This can be either on tier-0 or tier-1 (or a combination of the two). So everything that can be routed within the virtual environment, will be handled by this distributed part of the logical router. This will (like with NSX-V, be performed by every hypervisor in the fabric (attached to the same Transport Zone). So a logical router can have a DR and an SR component, which are deployed on different parts of the underlying infrastructure. The two components work together.
The Service Router is an entity that will handle all the stateful services, like load balancing, NAT and VPN. This is an instance that is not distributed, but it will run on an Edge Node, where it will service all that use it. The Service Router can either be combined with a tier-0 and a tier-1 gateway, dependent on the type of service.
Edge Nodes (Clusters)
Edge Nodes are physical or virtual machines where Service Routers can be placed in. It is a decoupling of the “physical” and “logical” part of the functionality. Within V each ESG was represented by a virtual machine (or two, when using HA). Within T, you do need an Edge Node, but within that Edge Node you can create one or multiple tier-0 and tier-1 gateways, each with there SR-based functionality. There are some limits when it comes to the amount of tier-0 and tier-1 gateways that can be placed on a single Edge Node, but I will not touch that, since this might change in future releases. Please look at the VMware maximums guides, to see the actual limits.
The Edge Nodes can be physical or virtual and when virtual they come in different sizes (small, medium, large), with different limits and features.
The Edge Nodes can be combined into Edge Clusters, thus giving redundancy and balancing of functionality. When creating tier-0 or tier-1 gateways, you select the Edge Node Cluster to which it will be deployed. This can be changed though, so it’s not set in stone.
So, that describes the theory, in the next post, I will be deploying routing within my test-environment.
4 thoughts on “Routing with NSX-T (part 1)”
Why NSX-T is confusing ? Even after i reading above post it is not clear where Tier-0 & tier-1 are used ? Which Tier can be used as DR and when it is called as DR and when it is called SR ?
Why Load balancing can be done on Tier-1 ? why not Tier-0 ?
If NAT can be done on Tier-0, will this gateway placed on edge node ? Where all these configuration will happen ?
Actually so many questions to ask…
Hi, I agree that it is sometimes confusing, especially when talking Load Balancing, which can only happen on T1. There is no technical reason for it, it has just been implemented this way, at this moment. It might change in the future, I know that T0 Load Balancing is something a lot of organizations would be interested in.
As to the DR and SR, that is pretty straight forward. When you are using Stateful Services, the activity is performed by the SR. The SR for both the T0 and T1 is always serviced by an Edge Node.
When it is plain (East-West) routing, that is done by the DR, both for T1 and T0 and the DR is serviced by the Host itself. When you are routing North-South, you will be using the SR. However, to make it even more confusing, when you connect the T1 to an Edge Node Cluster, routing between T1’s connected to the same T0 is passed through the SR of those T1. I have described this in another post: https://wordpress.com/post/my-sddc.net/426