Scenario Practice

Scenario Practice

Scenario Assumption: You want to use Mellanox CX-5 smart NICs to improve the network performance of business VM instances for business requirements.

Assume that you have prepared following resources:

Two x86 servers, one gigabit switch, one 10-gigabit switch, and several category five cables.
Each x86 server is installed with a Mellanox CX-5 smart NIC.
Following files:
- A latest ZStack Cloud custom ISO (h76c).
- A latest ZStack Cloud installation package.
- Latest Multi-mn HA suites.
- A ZStack Cloud license (Enterprise Prepaid).
- A latest ZStack Cloud VPC vRouter Image (KVM).

Assume that you plan your hardware networks as follows:

Management Network

Table 1. Management Network
Node	Gigabit Network Interface 1	Gigabit Network Interface 2	Aggregate Network Interface	Network Bridge	IP Address	Netmask	Gateway
Management Node1	eth0	eth1	bond0	br_bond0	172.25.15.143	255.255.255.0	172.25.0.1
Management Node2	eth0	eth1	bond0	br_bond0	172.25.15.99	255.255.255.0	172.25.0.1

Business Network

Table 2. Business Network
Node	10-Gigabit Network Interface 1	10-Gigabit Network Interface 2	IP Address	Netmask	Gateway
Management Node1	enp179s0f0	enp179s0f1	-	-	-
Management Node2	enp179s0f0	enp179s0f1	-	-	-

Assume that you plan your scenario network as follows:

Management Network

Table 3. Management Network
Management Network	Configuration
NIC	bond0
VLAN ID	NoVLAN
IP Range	172.25.15.50~172.25.15.150
Netmask	255.255.255.0
Gateway	172.25.0.1

VPC Network

Table 4. VPC Network1
VPC Network1	Configuration
NIC	bond0
VLAN ID	2200
CIDR	192.167.0.1/24
Gateway	192.167.0.1
DHCP IP	Randomly Allocated

Table 5. VPC Network2
VPC Network2	Configuration
NIC	enp179s0f1
VLAN ID	2334
CIDR	192.169.0.1/24
Gateway	192.169.0.1
DHCP IP	Randomly Allocated

Figure 2. Smart NIC Scenario Network Topology

Detailed practice procedures are described as follows:

Energize and connect hardware.
Install operating systems.
Configure the management network.
Install HA suites.
Install the license.
Create a cluster with Network Acceleration Support enabled.
Add the two management nodes to the cluster.
Virtualize the physical NIC.
Create two L2 network whose Network Acceleration Mode are respectively Smart NIC and Standard. Attach the two L2 networks to the cluster.
Create a VPC vRouter and two VPC networks.
Create two business VM instance and attach the two VPC networks to each VM instance.
Compare the VM vNIC performance with the vDPA NIC performance.

Energize and connect hardware.

Check your hardware devices. Configure servers and network devices according to Figure 1.
Install operating systems.

Install operating system for the two servers according to following steps:
1. Check BIOS.
  
  After you energize a server, manually start the server and enter its BIOS, making sure that SR-IOV Support and Intel VT-d / AMD IOMMU are enabled.
2. Burn the custom ISO into a USB drive.
3. Make the server start from the USB drive.
4. Install the operating system following the wizard.
  
  Configure partitioning and begin to install the operating system for the server as a management node. The installation is an automatic process. During this process, set your root account and password. After the installation is finished, restart the management node and unplug the USB drive.

Configure the management network.

Run following commands on the two management nodes in turn to configure the management network according to the table Table 1 in turn.

# Take management node1 as an example.  # Create the aggregate NIC bond0 [root@localhost ~]# zs-bond-lacp -c bond0  # Attach the two gigabit network interfaces, eth0 and eth1, of the management node1 to bond0. [root@localhost ~]# zs-nic-to-bond -a bond0 eth0  [root@localhost ~]# zs-nic-to-bond -a bond0 eth1  # After finishing the link aggregation configurations, configure the LACP aggregation on the corresponding vSwitch network interfaces.  # Create the network bridge br_bond0 and set the IP address, netmask, and gateway of the management node1. [root@localhost ~]# zs-network-setting -b bond0 172.25.15.143 255.255.255.0 172.25.0.1  # Check whether the aggregate port bond0 is created successfully. [root@localhost ~]# zs-show-network

Install HA suites.

Install HA suites on either of the two management nodes to realize Dual-MN HA.

Take management node1 as an example.
1. Import HA suites to management node1 and unzip the files.
2. Install HA suites on management node1.
3. Use the VIP to access the UI of management node1 and finish the Cloud initialization.
4. Switch the primary and secondary management nodes. Use the VIP to access the UI of management node2 and finish the Cloud initialization.
Install the license.

Use the VIP to access the UI of either management node. Upload the Dual-MN license you prepared.
Create a cluster with Network Acceleration Support enabled.
Use the VIP to access the UI of management node1. On the main menu of ZStack Cloud, choose Resource Center > Hardware > Computing Facility > Cluster. On the Cluster page, click Create Cluster. Then, the Create Cluster page is displayed.
On the displayed page, set the following parameters:
- Zone: The current zone is displayed.
- Name: Enter a name for the cluster.
- Description: Optional. Enter a description for the cluster.
- Hypervisor: Choose KVM.
- CPU Architecture: Choose x86_64.
- Network Acceleration Support: Optional. Default: disabled. In this practice, enable this feature.
  Note: You can enable Network Acceleration Support for a cluster only when you create the cluster.
- Network Configuration:
  - VDI Network: Optional. Leave it as blank in this practice.
  - Migration Network: Optional. Leave it as blank in this prectice.
- Advanced:
  - VM CPU Model: Optional. Use the default setting in this practice.
  - Host CPU Model: Optional. Use the default setting in this practice.
Figure 3. Create Cluster
Add the two management nodes to the cluster.
On the main menu of ZStack Cloud, choose Resource Center > Hardware > Computing Facility > Host. On the Host page, click Add Host. Then, the Add Host page is displayed. Choose Manual Addition on the displayed page.
Take management node1 as an example. Set the following parameters:
- Name: Enter a name for the host.
- Description: Optional. Enter a description for the host.
- Virtualization Technology: Choose KVM.
- Tag: Optional. Do not set this parameter in this practice.
- Cluster: Choose the cluster you created with Network Acceleration enabled.
- Host Info: Choose IP Address as the addition method.
  - IP Address: Enter the host IP. In this practice, enter the IP address of management node1: 172.25.15.143.
  - SSH Port: Enter the host port. Default: 22.
  - User Name: Enter the host user name. Default: root.
  - Password: Enter the password corresponding to the user name.
- Scan Host IOMMU Setting: Optional. Default: disabled. In this practice, enable this feature.
  Note: The setting takes effect after you restart the host.
- Intel EPT Hardware Assist: Optional. Default: enabled.
- Add Another Host: Select this checkbox to add management node2.
Figure 4. Add Management Node1
Virtualize the physical NIC.
On the details page of the host, choose Associated Resource > Physical NIC, locate a Mellanox CX-5 smart NIC and click SR-IOV on its action list to virtualized the smart NIC into VF NICs of the specified number.
Create two L2 network whose Network Acceleration Mode are respectively Smart NIC and Standard. Attach the two L2 networks to the cluster.
1. Create an L2 network whose Network Acceleration Mode is Smart NIC and attach it to the cluster.
  On the main menu of ZStack Cloud, choose Resource Center > Network Resource > L2 Network Resources > L2 Network. On the L2 Network page, click Create L2 Network. Then, the Create L2 Network page is displayed.
  On the displayed page, set the following parameter:
  - Zone: The current zone is displayed.
  - Name: Enter a name for the L2 network.
  - Description: Optional. Enter a description for the L2 network.
  - Type: Choose L2VlanNetwork.
  - Cluster: Optional. Choose the cluster you created with Network Acceleration Support enabled.
  - Network Acceleration Mode: Choose Smart NIC.
  - VLAN ID: Enter 2334 in this practice.
  - NIC Name: Enter the name of the L2 network device. In this practice, enter dpdkbond2.
  Figure 5. Create L2 Network ( Smart NIC)
2. Create an L2 network whose Network Acceleration Mode is Standard and attach it to the cluster.
  On the Create L2 Network page, set the following parameters:
  - Zone: The current zone is displayed.
  - Name: Enter a name for the L2 network.
  - Description: Optional. Enter a description for the L2 network.
  - Type: Choose L2VlanNetwork.
  - Cluster: Optional. Choose the cluster you created with Network Acceleration Support enabled.
  - Network Acceleration Mode: Choose Standard.
  - VLAN ID: In this practice, enter 2200.
  - NIC Name: Enter the name of the L2 network device. In this practice, enter bond0.
  Figure 6. Create L2 Network (Standard)
Create a VPC vRouter and two VPC networks.
1. Create a VPC vRouter.
  Upload the VPC vRouter image you prepared to Cloud and create a management network and vRouter offering according to Table 3.
  
  On the main menu of ZStack Cloud, choose Resource Center > Network Resource > vRouter > VPC vRouter. On the VPC vRouter page, click Create VPC vRouter. Then, the Create VPC vRouter page is displayed.
  On the displayed page, set the following parameters:
  - Name: Enter a name for the VPC vRouter.
  - Description: Optional. Enter a description for the VPC vRouter.
  - vRouter Offering: Choose a vRouter Offering.
  - Cluster: Optional. Specify a cluster for the host on which the VPC vRouter is to be started.
  - Storage Allocation Policy: Specify how the Cloud allocates a primary storage. The following two policies are supported:
    
    System Allocation: The Cloud allocates a primary storage according to the preconfigured policy.
    
    Custom: Select a primary storage as needed.
    
    Primary Storage: Select a primary storage for the VPC vRouter.
  - Host: Optional. Specify a host to start the VPC vRouter. In this practice, choose the management node1.
  - Default IPv4: Optional. Leave this parameter as blank in this practice.
  - Assign Management Network IP: Optional. Leave this parameter as blank in this practice.
  - DNS: Optional. Leave this parameter as blank in this practice.
  - CPU Pinning: Optional. Leave this parameter as blank in this practice.
  Figure 7. Create VPC vRouter
2. Create the VPC network1.
  Create the VPC network1 according to Table 4.
  
  On the main menu of ZStack Cloud, choose Resource Center > Network Resource > L3 Network Resources > VPC Network. On the VPC Network page, click Create VPC Network. Then, the Create VPC Network page is displayed.
  On the displayed page, set the following parameters:
  - Name: Enter a name for the VPC network. Enter VPC Network1 in this practice.
  - Description: Optional. Enter a description for the VPC network.
  - L2 Network: Choose an L2 network (Network Acceleration Mode: Standard).
  - VPC vRouter: Optional. Choose a VPC vRouter.
  - Network Range Method: Choose CIDR.
    
    CIDR: Enter 192.167.0.1/24.
    
    Gateway: Optional. Enter 192.167.0.1 in this practice.
    
    DHCP Service: Optional. Enabled by default.
    
    IP Allocation Policy: Optional. Choose Random in this practice.
    
    DHCP IP: Optional. Leave this parameter as blank in this practice.
  - DNS: Optional. Leave this parameter as blank in this practice.
  Figure 8. Create VPC Network1
3. Create the VPC Network2.
  Create the VPC network2 according to Table 5.
  On the displayed page, set the following parameters:
  - Name: Enter a name for the VPC network. Enter VPC Network2 in this practice.
  - Description: Optional. Enter a description for the VPC network.
  - L2 Network: Choose an L2 network (Network Acceleration Mode: Smart NIC).
  - VPC vRouter: Optional. Choose a VPC vRouter.
  - Network Range Method: Choose CIDR.
    
    CIDR: Enter 192.169.0.1/24.
    
    Gateway: Optional. Enter 192.169.0.1 in this practice.
    
    DHCP Service: Optional. Enabled by default.
    
    IP Allocation Policy: Choose Random in this network.
    
    DHCP: Optional. Leave this parameter as blank in this practice.
  - DNS: Optional. Leave this parameter as blank in this practice.
  Figure 9. Create VPC Network2
Create two VM instances. Attach each one with the two VPC networks you created before.
On the main menu of ZStack Cloud, choose Resource Center > Resource Pool > > Virtual Resource > VM Instance. On the VM Instance page, click Create VM Instance. Then, the Create VM Instance page is displayed. Choose Fast Creation.
On the displayed page, set the following parameters:
- Name: Enter a name for the VM instance.
- Quantity: Enter 2 in this practice.
- Tag: Optional. Do not set tags for the VM instance in this practice.
- Offering: Choose Basic Offering.
  - Instance Offering: Choose an instance offering.
  - Image: Choose an image.
  - Data Volume: Optional. No data volume is attached by default.
- Network Configuration:
  - Network: Attach the VPC network1 and VPC network2 to the VM instance.
  - Make Default: Make the VPC network2 as the default network.
  - Enable SR-IOV: Optional. Disable SR-IOV in this practice.
  - Assign IP: Optional. The checkbox is not selected by default. The IP is randomly allocated by DHCP.
  - MAC Address: Optional. The checkbox is not selected by default. The MAC is randomly allocated by DHCP.
  - Security Group: Optional. Do not set security groups in this practice.
  - EIP: Optional. Do not set EIPs in this practice.
- User Data: Optional. Do not set User Data in this practice.
Figure 10. Create VM Instance

Compare the VM vNIC performance with the vDPA NIC performance.

In this practice, use an iperf3 tool to test the network performance of the two VM instances. Compare the vNIC performance and vDPA NIC performance from 3 aspects: bandwidth, packet loss rate, and throughput.

Query NIC information of the two VM instance.

Name	NIC Type	IP Address
VM Instance1	vNIC	192.167.0.166
VM Instance1	vDPA	192.169.0.214
VM Instance2	vNIC	192.167.0.193
VM Instance2	vDPA	192.169.0.253

UDP small packet performance testing (packet size: 16 byte).

Assume that the VM instance1 is the client and VM instance2 is the server.

The testing involves two rounds.

First Round: Send UDP packets from the VM instance2 (vNIC) to the VM instance1 (vNIC). View the bandwidth, packet loss rate, and throughput.
Second Round: Send UDP packets from the VM instance2 (vDPA) to the VM instance1 (vDPA). View the bandwidth, packet loss rate, and throughput.

# Install iperf3 on both the client and the server. [root@localhost ~]# yum install iperf3  # First-Round Testing  # Run the following command on the server: [root@localhost ~]# iperf3 -s  # Run the following command on the client: [root@localhost ~]# iperf3 -u -l 16 -b 100m -t 30 -c 192.167.0.193 -i 1  # iperf3 commands information # -s //Run in the server mode. # -u //UDP mode. # -l //UDP small packet size. Unit: byte. # -b //Target bandwidth. Unit: bits/sec. # -t //Testing period. Unit: s. # -c //Run in the client mode and specify the server IP. # -i //Echo interval. Unit: second.  # The first-round testing lasts about 30 seconds.  # Second-Round Testing  # Run the following command on the client: [root@localhost ~]# iperf3 -u -l 16 -b 100m -t 30 -c 192.169.0.253 -i 1  # The second-round testing lasts about 30 seconds.

Test Result

UDP Small Packet Testing	Server - Client	Bandwidth (Mbits/sec)	Packet Loss Rate (%)	Throughput (pps)
First Round	VM Instance2 (vNIC) - VM Instance1 (VNIC)	12.1	0.46	95144.3
Second Round	VM Instance2 (vDPA) - VM Instance1 (vDPA)	19.2	0.74	151418

Figure 12. UDP Small Packet Testing Result

UDP big packet performance testing (packet size: 1460 byte).

Assume that the VM instance1 is a client and VM instance2 is a server.

The testing involves two rounds:

First Round: Send UDP packets from VM instance2 (vNIC) to VM instance1 (vNIC). View the bandwidth, packet loss rate, and throughput.
Second Round: Send UDP packets from VM instance2 (vDPA) to VM instance1 (vDPA). View the bandwidth, packet loss rate, and throughput.

# First-Round Testing  # Run the folliwng command on the client (the UDP packet size is 1460 by default): [root@localhost ~]# iperf3 -u -b 1G -t 30 -c 192.167.0.193 -i 1  # The first-round testing lasts about 30 seconds.  # Second-Round Testing  # Run the following command on the client (the UDP packet size is 1460 byte by default): [root@localhost ~]# iperf3 -u -b 1G -t 30 -c 192.169.0.253 -i 1  # The second-round testing lasts about 30 seconds.

Testing Result

UDP Big Packet Testing	Server - Client	Bandwidth (Mbits/sec)	Packet Loss Rate (%)	Throughput (pps)
First Round	VM Instance2 (vNIC) - VM Instance1 (vNIC)	267	51	94735.1
Second Round	VM Instance2 (vDPA) - VM Instance1 (vDPA).	843	1.8	148343.5

Now, you have successfully put NICs into use.

Products

Solutions

Help & Support

Contact Us