Login
- Table of Contents
- Related Documents
-
| Title | Size | Download |
|---|---|---|
| 02-IPCC configuration | 114.84 KB |
Configuring IPCC
About IPCC
Intelligent Proactive Congestion Control (IPCC) is an improvement of Explicit Congestion Notification (ECN). It intelligently identifies the interface congestion state and proactively sends Congestion Notification Packets (CNPs) to control the RDMA over Converged Ethernet version 2 (RoCEv2) packet sending rate, ensuring low latency and high throughput. For more information about ECN, see QoS configuration in ACL and QoS Configuration Guide.
RoCEv2 packets
IPCC takes effect only on RoCEv2 packets. RoCEv2 uses UDP encapsulation. Figure 1 shows the RoCEv2 packet format.
· Ethernet header—Contains source and destination MAC addresses.
· IP header—Contains source and destination IP addresses.
· UDP header—Contains source and destination port numbers. The destination port number of RoCEv2 packets is 4791.
· Payload—Message payload.
· CRC—Cyclic redundancy check.
· FCS—Frame check sequence.
· InfiniBand Base Transport Header—Contains key information about an RoCEv2 packet. IPCC is related to the following fields:
¡ Opcode—Indicates the RoCEv2 packet type, which can be one of the following:
- ConnectMsg—Communication management (CM) connection setup packet. The packet is used for RoCEv2 connection setup. The sender and receiver use the RoCEv2 connection to exchange data packets.
- Send—Send packet. The sender uses the packet to request data from the receiver, without specifying the address where the receiver stores data.
- Write—Write packet. The packet carries the address, key, and length of data to be written to the receiver.
- Read—Read packet. The packet carries the address, key, and length of data to be read from the receiver.
- ACK—ACK packet. The packet is a response message returned by the receiver.
The Send, Write, and Read RoCEv2 packets are RoCEv2 data packets.
¡ Dest QP—Destination Queue Pair (QP), identifying an RoCEv2 flow. This field is similar to the destination port number. It is a key value for setting up an RoCEv2 flow table.
IPCC mechanism
IPCC is usually deployed in data center networks. Enable IPCC on the interface that connects the forwarding device to the packet receiver implement intelligent congestion control.
As shown in Figure 2, IPCC works as follows:
1. Set up an RoCEv2 flow table.
After you enable IPCC for an interface on the forwarding device, the interface replicates RoCEv2 data packets passing the interface and sends the packets to the CPU. The CPU processes the packets as follows:
a. Generates an RoCEv2 flow table based on the quadruple information (client IP, server IP, client QP, and server QP) in the RoCEv2 connection setup packets.
b. Associates the RoCEv2 flow table with the interface based on the RoCEv2 data packets.
By maintaining the RoCEv2 flow table, the forwarding device can obtain address and forwarding path information about each RoCEv2 flow.
2. Calculate the number of CNPs to be sent.
After you enable IPCC for a queue on the interface, the forwarding device monitors the length of the queue and calculates the number of CNPs to be sent as follows:
¡ When the length of the queue increases, the forwarding device needs to send a small number of CNPs if the cache usage of the queue is low. If the cache usage of the queue is high, the forwarding device needs to send a large number of CNPs to reduce queue congestion and forwarding latency.
¡ When the length of the queue decreases, the forwarding device does not need to send CNPs if the cache usage of the queue is low. If the cache usage of the queue is high, the forwarding device needs to send a small number of CNPs to reduce queue congestion while ensuring throughput and latency performance.
3. Construct and send CNPs.
The forwarding device constructs CNPs based on the calculated number and address information in the RoCEv2 flow table and sends the CNPs to the sender. Then, the sender reduces the RoCEv2 packet sending rate.
Figure 2 IPCC mechanism
Restrictions: Licensing requirements for IPCC
To use IPCC, you must install a license. If the license expires or is uninstalled, no IPCC functions are available. To resolve this issue, you must install a valid license again. For more information about licenses, see license management in Fundamentals Configuration Guide.
IPCC tasks at a glance
To configure IPCC, perform the following tasks:
3. (Optional.) Specifying an IPCC
Enabling IPCC
Restrictions and guidelines
IPCC takes effect on only RoCEv2 packets.
For IPCC to take effect on an interface, you must also enable IPCC for a queue on the interface. To enable IPCC for a queue, execute the ipcc queue command.
Procedure
1. Enter system view.
system-view
2. Enter interface view.
interface interface-type interface-number
3. Enable IPCC.
ipcc enable
By default, IPCC is disabled.
Enabling IPCC for a queue
About this task
After you enable IPCC for a queue, the forwarding device monitors the length of the queue and calculates the number of CNPs to be proactively sent. Then, the forwarding device constructs CNPs based on the calculated number and address information in the RoCEv2 flow table and sends the CNPs to the sender. After receiving the CNPs, the sender reduces the RoCEv2 packet sending rate.
Restrictions and guidelines
For IPCC to take effect on a queue, you must also enable IPCC for the interface of the queue. To enable IPCC for an interface, execute the ipcc enable command.
Procedure
1. Enter system view.
system-view
2. Enable IPCC for the specified queue.
ipcc queue queue-id
By default, IPCC is disabled for a queue.
Specifying an IPCC mode
About this task
Configure IPCC high throughput mode for services with a high traffic volume and low latency sensitivity. Configure IPCC low latency mode for services with a low traffic volume and high latency sensitivity.
Procedure
1. Enter system view.
system-view
2. Specify an IPCC mode.
ipcc mode { high-throughput | low-latency }
By default, IPCC operates in high throughput mode.
Display and maintenance commands for IPCC
Execute display commands in any view.
|
Task |
Command |
|
Display statistics of CNPs that are proactively sent by the device. |
display ipcc statistics [ interface interface-type interface-number ] |
IPCC configuration examples
Example: Configuring IPCC
Network configuration
As shown in Figure 3, the sender is deployed with storage services carried by RoCEv2. Enable IPCC on interface Ten-GigabitEthernet1/0/2 on Device to control the RoCEv2 packet sending rate of the sender, ensuring low latency and high throughput for the RoCEv2 service.
Procedure
1. Configure IP addresses and subnet masks for the interfaces, as shown in Figure 3. (Details not shown.)
2. Configure IPCC:
# Enable IPCC.
<Device> system-view
[Device] interface ten-gigabitethernet 1/0/2
[Device-Ten-GigabitEthernet1/0/2] ipcc enable
[Device-Ten-GigabitEthernet1/0/2] quit
# Enable IPCC for queue 4.
[Device] ipcc queue 4
# Configure IPCC to operate in low latency mode.
[Device] ipcc mode low-latency
Verifying the configuration
# View statistics of CNPs that are proactively sent by the device.
<Device> display ipcc statistics
Total statistics:
RoCEv2 flow entries: 1
Interface statistics of XGE1/0/2
CNP number: 3
RoCEv2 flow entries: 1
Flow information:
---------------------------------------------
DIP SIP DQP
---------------------------------------------
12.110.2.2 11.110.2.2 2090
----------------------------------------------
