How Can the Packet Size Be Greater than the MTU?

时间:2023-01-15 09:36:30

    下面吧啦吧啦了很久,其实就是从下面图可以看出:

抓包其实加了一个钩子函数,拷贝一份skb而已,然后传到wireshark里面,

这里的原因是运用了tso机制,因此NIC负责切分MSS发包,

但是tcp协议不负责切分,因此会出现抓包有大于1500的情况,

其实这种情况,现实中也遇到过。

How Can the Packet Size Be Greater than the MTU?




So you’ve got a problem and you decide to fire up Wireshark and take a capture. When you look at the packets you see a bunch of them that are far larger than the 1500 byte MTU.

How Can the Packet Size Be Greater than the MTU?

HOW CAN THIS BE?!?!?

There’s something you need to know about taking captures on the host that is sending data. Let’s say you’re uploading some data to a server while capturing packets on your machine. You look at the capture and see something like this:

How Can the Packet Size Be Greater than the MTU?

Clearly these large packets exceeding the MTU must be part of the problem, right? Probably not. Here’s why.

Many operating systems and NIC drivers support TCP Segmentation Offload (TSO) aka Large Segment Offload (LSO) aka Generic Segment Offload (GSO). What this means is that the TCP stack sends a chunk of data for the NIC to break up into Maximum Segment Size (MSS) pieces to send on the network. TCP might hand the NIC 16k of data and the NIC will break it into MSS sized bites: 11 segments of 1460 bytes and one segment of the remaining 324 bytes. This offloads the task to the NIC and saves overhead on the host’s resources. It’s a performance thing.

Here’s the kicker: Wireshark uses libpcap or winpcap to grab the data before it gets handed to the NIC.

Check it out:

How Can the Packet Size Be Greater than the MTU?

So you don’t see the actual packets that are put on the wire unless you capture outside the sending host with a tap or span port. This is one of several reasons it’s a good idea to capture traffic outside of the hosts involved in the connection whenever possible.

Here’s what the data looks like captured on the sender and then arriving at the receiver after it has been segmented:

How Can the Packet Size Be Greater than the MTU?

This behavior makes TCP sequence number analysis a pain in the ass. If you’re a network troubleshooter using packet analysis, you’ve GOT to be comfortable doing sequence number analysis.

I saw someone post on reddit the other day asking about sequence number interpretation in tcpdump output. The most upvoted comment said that they had been looking at tcpdump output for 15 years and that they had never had to calculate sequence numbers.

WHUT?

I mean, what have you been doing for 15 years, son?

Anyway.

There’s another side to it that I recently saw for the first time. Large Receive Offload (LRO) or Receive Segment Coalescing (RSC). The is the same thing but in reverse. The NIC coalesces TCP segments it receives from a remote host into larger packets before sending them up to the TCP stack. Again, by offloading this to the NIC, it’s a performance enhancement but a pain in my ass.

Check out this capture taken on the client. Notice that this large frame is coming from the server and there’s no way it could have traversed a WAN without fragmentation, so it must be LRO.

How Can the Packet Size Be Greater than the MTU?

One time, I got annoyed so much at this behavior that I wrote a perl script to break large packets in a capture file into MSS sized packets just to make sequence number analysis easier. I don’t know if anyone is interested in that, but I could post it up if y’all wanted. Of course, if you plan ahead you could just disable segmentation offloading before taking the capture.

So next time you take captures on a host sending and receiving traffic, do not be alarmed if you see Really Big Packets™.

Share this post! Spread the packet gospel!