strange ip_reass issue on powerpc board, need help

[Rui Zhengxin]

On Tuesday, November 05, 2013 7:34 AM Nick Withers wrote:

> On Fri, 2013-11-01 at 08:33 +0800, Rui Zhengxin wrote:
>> On Friday, November 01, 2013 6:47 AM Nick Withers wrote:
>> 
>> Thanks Nick,
>> 
>>> On Thu, 2013-10-31 at 17:16 +0800, Rui Zhengxin wrote:
>>>> Hi all,
>>> 
>>> Hey Rui,
>>> 
>>>> My board is MPC8309SOM, the uec network adapter driver is port from 8313 tsec, it works well.
>>> 
>>> MVME3100 here, with the MPC8540 and a tsec.
>>> 
>>>> Now I meet a strange ip_reass issue, it cause a crash.
>>>> The condition is send a fragment packet with a non-zero ip_offset value but not set IP_MF flag.
>>> 
>>> Is that legal in the first place? You're talking about the "More
>>> Fragments" flag, yeah? Shouldn't that *always* be set for packets in a
>>> fragmented datagram?
>>> 
>>> 'Course if it ain't legal one still doesn't want to target to crash :-P
>>> 
>>>> It's very easy to make the fragment packet by ping the target board 65000 bytes after clearing PC's ARP table first.
>>> 
>>> What does clearing the ARP table here accomplish?
>> 
>> Sorry, I forgot to tell you, my PC is Windows XP,
>> I find that in Windows XP, if clear ARP table first, then send a big ip packet, the first  fragment of is abnormal.
>> It's just the last fragment, as I said "a non-zero ip_offset value but not set IP_MF flag".
>> The attached is the capture result of wireshark, the third packet is the abnormal fragment.
> 
> That looks like the last fragment of a datagram (and hence there aren't
> "More Fragments"), so makes sense, I reckon. Compare it to packet #47
> (not the reassembly shown against #47 in Wireshark, but the actual
> fragment itself), for instance (the last fragment of the first complete
> ping datagram).

The abnormal fragment packet can be made easily in windows XP.
I'm not sure this is a bug of windows or not :-P

>>> In any event, I haven't been able to crash mine, running RTEMS 4.11 from
>>> the start of October.
>>> 
>>>> I trace the code, after call ip_reass, the mbuf chain may become not correct(ps: not everytime), the last mbuf's m_next pointer
>>>> not set to NULL. When the upper layer discard the fragment by calling m_freem, the invalid ext_free function pointer cause
>>>> crash. I don't know the reason why the mbuf chain is changed.
>>>> 
>>>> If you have MPC8313 or other powerpc board, can you help me verify the issue? Very tks for you help.
>>> 
>>> Any other thoughts? My pings're of the form "sudo ping -s 65000 -i 0.25"
>>> from a FreeBSD 9.2-STABLE host and aren't actually being responded to
>>> above 25152 B of data.
>> 
>> I  test on QEMU run i386 image, the issue not appear.
>> If send such an abnormal fragment, the "netstats -mp" command shows the MBUF ftable counter increased,
>> then after some time(ttl timeout), the IP Statistics of fragments timed out counter increased.
> 
> Which seems fair, doesn't it, on the RTEMS side at least (not so much
> the crashing), assuming it hasn't seen all fragments of a datagram?

As the ip_reass code, if receive a new fragment with different HASH(the seed is source ip and id),
the fragment is added to a new reassemble queue(ftable counter increased),
if a timer expires on a reassembly queue, the queue will be discarded(fragments timed out counter increased),
so the work flow path of QEMU i386 is right.

> Are you perhaps dumping and refreshing your ARP cache *while* doing
> pings, interrupting one of 'em?

Yes, the crash  may occur when clear ARP cache while doing big size pings.

> 
> So this is a driver you've ported over, isn't it? Could it be that the
> hardware you've ported it to doesn't support Large Receive Offload but
> the original hardware did... Or something?

If not clear ARP cache, this crash not occur, 
the target board reply the ping request correctly. 
For the normal fragment packet(packet received one by one in order),
the driver has no problem.


I add a do-nothing function after ip_enq in ip_reass, it seems no crash anymore.
I really don't know the reason :-(

>>> I've also got pretty big mbuf (2 MiB) / mbuf cluster (4 MiB) buffers
>>> configured, perhaps that's something to do with it?
>>> 
>>>> Thanks & Best regards,
>>>> Rui Zhengxin
>>> 
>>> Hope that helps... It probably doesn't! :-(
>> 
>> 
>> 
>> Thanks & Best regards,
>> Rui Zhengxin



Thanks & Best regards,
Rui Zhengxin