3 Part Series
I intended this to be a single article but it got very long.
Goal:
Have NSX-T configured on bare-metal ESXi hosts with a two physical ethernet port hardware limitation. These will need to be configured in a three host cluster configured for both transport and edge serving VLAN and Overlay NVDS.
Environment:
- NSX-T Datacenter 2.5
- Three physical servers with two 25ge ports each
- Two Intel Gold Processors
- Bunch of RAM
- Two physical switches providing 802.1q trunks to each server port
- Transport Node TEPs will be on a subnet 99 (use whichever you like)
- Edge TEPs will be on a subnet 98 (use whichever you like but not the TN TEP subnet)
- Loaded with VMware Esxi 6.7
- Clustered in vCenter (HA/DRS)
- Shared storage provided by FC attach
- Two vmkernel ports one for management and other for ancillary traffic (vmotion etc..)
Challenges:
This is a good design for a special use case or development environment. In the real world you would most likely want to have at least 4 "physical" ethernet ports you could use. These would allow you to provide multiple physical uplinks for each NVDS. Then you would possibly require two more to service the ESXi hosts management and ancillary networking.
With this in mind you still need the two physical adapters assigned as an uplink to your Overlay and VLAN NVDS switches. This isn't the most redundant design however these are modular and there are some exceptions for downstream switch connectivity using these modular switches. This quasi makes up for the limit of two adapters.
The edge VM will force you to deploy the management interface to a non-NDVS switch port group on your current ESXi hosts. We will need to do a trick and a quick switch on the eth0 on the edge vms.
Solution:
- Create a two TEP IP pools
- Create logical segments using the VLAN type with the vmkernel required VLANs
- Create a logical segment for the edge VM
- Create a logical segment for the T0 router interface
- Create two Uplink Profiles for your vmnics
- Tag your TEP interface in one of the profiles
- Configure vmkernel port mappings in the Transport Node Profile
- Configure a vSwitch with no physical uplinks using the default VM Network
- Use this portgroup to assign to the edge VM on deployment
- Wait for the OVA deployment to complete
- Refresh the interface and change the "VM Network" adapter to your required LS **quickly**
This guide assumes you know the basics of an NSX-T deployment and are familiar with ESXi vSwitches and vmkernel ports.
This has a high probability of leaving your ESXi hosts with no networking if done improperly.