测来测去15：DPDK VxLAN Inner L4 CSUM OFFLOAD

IXGBE

以IXGBE驱动为例，看一下如何让把内层报文Checksum的计算Offload给网卡。

本质上来说，是在DPDK的mbuf结构中，将L2 Header的长度配置为外层VxLAN报文+内层L2 Header的总长度，这样对网卡来说，该mbuf对应的报文就是一个L2 Header长得令人发指的普通非隧道报文，但是这样就可以计算内层L3/L4 Header的Checksum了。

ixgbe_tx_offload

在DPDK的IXGBE驱动代码中有如下结构体：

/** Offload features */
union ixgbe_tx_offload {
    uint64_t data[2];
    struct {
        uint64_t l2_len:7; /**< L2 (MAC) Header Length. */
        uint64_t l3_len:9; /**< L3 (IP) Header Length. */
        uint64_t l4_len:8; /**< L4 (TCP/UDP) Header Length. */
        uint64_t tso_segsz:16; /**< TCP TSO segment size */
        uint64_t vlan_tci:16;
        /**< VLAN Tag Control Identifier (CPU order). */

        /* fields for TX offloading of tunnels */
        uint64_t outer_l3_len:8; /**< Outer L3 (IP) Hdr Length. */
        uint64_t outer_l2_len:8; /**< Outer L2 (MAC) Hdr Length. */
#ifdef RTE_LIBRTE_SECURITY
        /* inline ipsec related*/
        uint64_t sa_idx:8;  /**< TX SA database entry index */
        uint64_t sec_pad_len:4; /**< padding length */
#endif
    };
};

这里面l2_len是占了7个比特位，也就是说你外层报文头长度+内层L2 Header长度不要大于127Byte。

另外代码中还有一些ol_flag的配置，也需要结合需求一起配置。

配置方法

以下内容节选自：https://doc.dpdk.org/guides/prog_guide/mbuf_lib.html

但是我感觉好像没什么人看到过的样子….

checksum of out_ip:

mb->l2_len = len(out_eth)
mb->l3_len = len(out_ip)
mb->ol_flags |= PKT_TX_IPV4 | PKT_TX_IP_CSUM

set out_ip checksum to 0 in the packet
This is supported on hardware advertising DEV_TX_OFFLOAD_IPV4_CKSUM.

checksum of out_ip and out_udp:

mb->l2_len = len(out_eth)
mb->l3_len = len(out_ip)
mb->ol_flags |= PKT_TX_IPV4 | PKT_TX_IP_CSUM | PKT_TX_UDP_CKSUM

set out_ip checksum to 0 in the packet
set out_udp checksum to pseudo header using rte_ipv4_phdr_cksum()
This is supported on hardware advertising DEV_TX_OFFLOAD_IPV4_CKSUM and DEV_TX_OFFLOAD_UDP_CKSUM.

checksum of in_ip:

mb->l2_len = len(out_eth + out_ip + out_udp + vxlan + in_eth)
mb->l3_len = len(in_ip)
mb->ol_flags |= PKT_TX_IPV4 | PKT_TX_IP_CSUM

set in_ip checksum to 0 in the packet
This is similar to case 1), but l2_len is different. It is supported on hardware advertising DEV_TX_OFFLOAD_IPV4_CKSUM. Note that it can only work if outer L4 checksum is 0.

checksum of in_ip and in_tcp:

mb->l2_len = len(out_eth + out_ip + out_udp + vxlan + in_eth)
mb->l3_len = len(in_ip)
mb->ol_flags |= PKT_TX_IPV4 | PKT_TX_IP_CSUM | PKT_TX_TCP_CKSUM

set in_ip checksum to 0 in the packet
set in_tcp checksum to pseudo header using rte_ipv4_phdr_cksum()
This is similar to case 2), but l2_len is different. It is supported on hardware advertising DEV_TX_OFFLOAD_IPV4_CKSUM and DEV_TX_OFFLOAD_TCP_CKSUM. Note that it can only work if outer L4 checksum is 0.

segment inner TCP:

mb->l2_len = len(out_eth + out_ip + out_udp + vxlan + in_eth)
mb->l3_len = len(in_ip)
mb->l4_len = len(in_tcp)
mb->ol_flags |= PKT_TX_IPV4 | PKT_TX_IP_CKSUM | PKT_TX_TCP_CKSUM |
  PKT_TX_TCP_SEG;

set in_ip checksum to 0 in the packet
set in_tcp checksum to pseudo header without including the IP
payload length using rte_ipv4_phdr_cksum()
This is supported on hardware advertising DEV_TX_OFFLOAD_TCP_TSO. Note that it can only work if outer L4 checksum is 0.

checksum of out_ip, in_ip, in_tcp:

mb->outer_l2_len = len(out_eth)
mb->outer_l3_len = len(out_ip)
mb->l2_len = len(out_udp + vxlan + in_eth)
mb->l3_len = len(in_ip)
mb->ol_flags |= PKT_TX_OUTER_IPV4 | PKT_TX_OUTER_IP_CKSUM  | \
  PKT_TX_IP_CKSUM |  PKT_TX_TCP_CKSUM;

set out_ip checksum to 0 in the packet
set in_ip checksum to 0 in the packet
set in_tcp checksum to pseudo header using rte_ipv4_phdr_cksum()
This is supported on hardware advertising DEV_TX_OFFLOAD_IPV4_CKSUM, DEV_TX_OFFLOAD_UDP_CKSUM and DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM.