1.. SPDX-License-Identifier: GPL-2.0 2 3=============================================== 4XFRM device - offloading the IPsec computations 5=============================================== 6 7Shannon Nelson <shannon.nelson@oracle.com> 8Leon Romanovsky <leonro@nvidia.com> 9 10 11Overview 12======== 13 14IPsec is a useful feature for securing network traffic, but the 15computational cost is high: a 10Gbps link can easily be brought down 16to under 1Gbps, depending on the traffic and link configuration. 17Luckily, there are NICs that offer a hardware based IPsec offload which 18can radically increase throughput and decrease CPU utilization. The XFRM 19Device interface allows NIC drivers to offer to the stack access to the 20hardware offload. 21 22Right now, there are two types of hardware offload that kernel supports. 23 * IPsec crypto offload: 24 * NIC performs encrypt/decrypt 25 * Kernel does everything else 26 * IPsec packet offload: 27 * NIC performs encrypt/decrypt 28 * NIC does encapsulation 29 * Kernel and NIC have SA and policy in-sync 30 * NIC handles the SA and policies states 31 * The Kernel talks to the keymanager 32 33Userland access to the offload is typically through a system such as 34libreswan or KAME/raccoon, but the iproute2 'ip xfrm' command set can 35be handy when experimenting. An example command might look something 36like this for crypto offload: 37 38 ip x s add proto esp dst 14.0.0.70 src 14.0.0.52 spi 0x07 mode transport \ 39 reqid 0x07 replay-window 32 \ 40 aead 'rfc4106(gcm(aes))' 0x44434241343332312423222114131211f4f3f2f1 128 \ 41 sel src 14.0.0.52/24 dst 14.0.0.70/24 proto tcp \ 42 offload dev eth4 dir in 43 44and for packet offload 45 46 ip x s add proto esp dst 14.0.0.70 src 14.0.0.52 spi 0x07 mode transport \ 47 reqid 0x07 replay-window 32 \ 48 aead 'rfc4106(gcm(aes))' 0x44434241343332312423222114131211f4f3f2f1 128 \ 49 sel src 14.0.0.52/24 dst 14.0.0.70/24 proto tcp \ 50 offload packet dev eth4 dir in 51 52 ip x p add src 14.0.0.70 dst 14.0.0.52 offload packet dev eth4 dir in 53 tmpl src 14.0.0.70 dst 14.0.0.52 proto esp reqid 10000 mode transport 54 55Yes, that's ugly, but that's what shell scripts and/or libreswan are for. 56 57 58 59Callbacks to implement 60====================== 61 62:: 63 64 /* from include/linux/netdevice.h */ 65 struct xfrmdev_ops { 66 /* Crypto and Packet offload callbacks */ 67 int (*xdo_dev_state_add) (struct xfrm_state *x); 68 void (*xdo_dev_state_delete) (struct xfrm_state *x); 69 void (*xdo_dev_state_free) (struct xfrm_state *x); 70 bool (*xdo_dev_offload_ok) (struct sk_buff *skb, 71 struct xfrm_state *x); 72 void (*xdo_dev_state_advance_esn) (struct xfrm_state *x); 73 74 /* Solely packet offload callbacks */ 75 void (*xdo_dev_state_update_curlft) (struct xfrm_state *x); 76 int (*xdo_dev_policy_add) (struct xfrm_policy *x); 77 void (*xdo_dev_policy_delete) (struct xfrm_policy *x); 78 void (*xdo_dev_policy_free) (struct xfrm_policy *x); 79 }; 80 81The NIC driver offering ipsec offload will need to implement callbacks 82relevant to supported offload to make the offload available to the network 83stack's XFRM subsystem. Additionally, the feature bits NETIF_F_HW_ESP and 84NETIF_F_HW_ESP_TX_CSUM will signal the availability of the offload. 85 86 87 88Flow 89==== 90 91At probe time and before the call to register_netdev(), the driver should 92set up local data structures and XFRM callbacks, and set the feature bits. 93The XFRM code's listener will finish the setup on NETDEV_REGISTER. 94 95:: 96 97 adapter->netdev->xfrmdev_ops = &ixgbe_xfrmdev_ops; 98 adapter->netdev->features |= NETIF_F_HW_ESP; 99 adapter->netdev->hw_enc_features |= NETIF_F_HW_ESP; 100 101When new SAs are set up with a request for "offload" feature, the 102driver's xdo_dev_state_add() will be given the new SA to be offloaded 103and an indication of whether it is for Rx or Tx. The driver should 104 105 - verify the algorithm is supported for offloads 106 - store the SA information (key, salt, target-ip, protocol, etc) 107 - enable the HW offload of the SA 108 - return status value: 109 110 =========== =================================== 111 0 success 112 -EOPNETSUPP offload not supported, try SW IPsec, 113 not applicable for packet offload mode 114 other fail the request 115 =========== =================================== 116 117The driver can also set an offload_handle in the SA, an opaque void pointer 118that can be used to convey context into the fast-path offload requests:: 119 120 xs->xso.offload_handle = context; 121 122 123When the network stack is preparing an IPsec packet for an SA that has 124been setup for offload, it first calls into xdo_dev_offload_ok() with 125the skb and the intended offload state to ask the driver if the offload 126will serviceable. This can check the packet information to be sure the 127offload can be supported (e.g. IPv4 or IPv6, no IPv4 options, etc) and 128return true of false to signify its support. 129 130Crypto offload mode: 131When ready to send, the driver needs to inspect the Tx packet for the 132offload information, including the opaque context, and set up the packet 133send accordingly:: 134 135 xs = xfrm_input_state(skb); 136 context = xs->xso.offload_handle; 137 set up HW for send 138 139The stack has already inserted the appropriate IPsec headers in the 140packet data, the offload just needs to do the encryption and fix up the 141header values. 142 143 144When a packet is received and the HW has indicated that it offloaded a 145decryption, the driver needs to add a reference to the decoded SA into 146the packet's skb. At this point the data should be decrypted but the 147IPsec headers are still in the packet data; they are removed later up 148the stack in xfrm_input(). 149 150 find and hold the SA that was used to the Rx skb:: 151 152 get spi, protocol, and destination IP from packet headers 153 xs = find xs from (spi, protocol, dest_IP) 154 xfrm_state_hold(xs); 155 156 store the state information into the skb:: 157 158 sp = secpath_set(skb); 159 if (!sp) return; 160 sp->xvec[sp->len++] = xs; 161 sp->olen++; 162 163 indicate the success and/or error status of the offload:: 164 165 xo = xfrm_offload(skb); 166 xo->flags = CRYPTO_DONE; 167 xo->status = crypto_status; 168 169 hand the packet to napi_gro_receive() as usual 170 171In ESN mode, xdo_dev_state_advance_esn() is called from xfrm_replay_advance_esn(). 172Driver will check packet seq number and update HW ESN state machine if needed. 173 174Packet offload mode: 175HW adds and deletes XFRM headers. So in RX path, XFRM stack is bypassed if HW 176reported success. In TX path, the packet lefts kernel without extra header 177and not encrypted, the HW is responsible to perform it. 178 179When the SA is removed by the user, the driver's xdo_dev_state_delete() 180and xdo_dev_policy_delete() are asked to disable the offload. Later, 181xdo_dev_state_free() and xdo_dev_policy_free() are called from a garbage 182collection routine after all reference counts to the state and policy 183have been removed and any remaining resources can be cleared for the 184offload state. How these are used by the driver will depend on specific 185hardware needs. 186 187As a netdev is set to DOWN the XFRM stack's netdev listener will call 188xdo_dev_state_delete(), xdo_dev_policy_delete(), xdo_dev_state_free() and 189xdo_dev_policy_free() on any remaining offloaded states. 190 191Outcome of HW handling packets, the XFRM core can't count hard, soft limits. 192The HW/driver are responsible to perform it and provide accurate data when 193xdo_dev_state_update_curlft() is called. In case of one of these limits 194occuried, the driver needs to call to xfrm_state_check_expire() to make sure 195that XFRM performs rekeying sequence. 196