tcpreplay for testing network stacks

[Chris Johns]

On 29/4/21 4:30 pm, Christian MAUDERER wrote:
> Am 29.04.21 um 00:54 schrieb Vijay Kumar Banerjee:
>> On Wed, Apr 28, 2021 at 3:27 PM Chris Johns <chrisj at rtems.org> wrote:
>>>
>>>
>>>
>>> On 29/4/21 3:00 am, Vijay Kumar Banerjee wrote:
>>>> On Wed, Apr 28, 2021 at 10:49 AM Christian Mauderer <oss at c-mauderer.de> wrote:
>>>>>
>>>>> Hello Vijay,
>>>>>
>>>>> On 28/04/2021 18:25, Vijay Kumar Banerjee wrote:
>>>>>> On Wed, Apr 28, 2021 at 12:41 AM Christian MAUDERER
>>>>>> <christian.mauderer at embedded-brains.de> wrote:
>>>>>>>
>>>>>>> Hello Vijay,
>>>>>>>
>>>>>>> Am 27.04.21 um 18:48 schrieb Vijay Kumar Banerjee:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I came across the tcpreplay tool and it looks like a nice tool for
>>>>>>>> testing the network stacks. It can be used to capture network traffic
>>>>>>>> and then play it back, this will help with testing the network packets
>>>>>>>> from different network stacks.
>>>>>>>
>>>>>>> Sounds like an interesting tool.
>>>>>>>
>>>>>>>>
>>>>>>>> My proposal is to add the tcpreply as a host-side tool in rtems-tools
>>>>>>>> and use it with the network interface where the network application is
>>>>>>>> running. The only issue that I see with the whole idea is that the
>>>>>>>> tcpreplay is GPLv3 licensed. Will that be compatible for rtems-tools?
>>>>>>>> The github repository says that it's compatible with UNIX and Windows
>>>>>>>> with cygwin.
>>>>>>>
>>>>>>> The more difficult problem could be the missing Mac and FreeBSD support.
>>>>>>>
>>>>>> That's a good point.
>>>>>>
>>>>>>> What would be the advantage of having tcpreply in rtems-tools? Do you
>>>>>>> want to use it for automated tests?
>>>>>>>
>>>>>> Yes. I was thinking about capturing the pcap format packets in
>>>>>> temporary files and then running tcpreplay to check for any network
>>>>>> issues. I haven't planned exactly how that will be implemented but
>>>>>> roughly this is the idea.
>>>>>>
>>>>>
>>>>> In what form would you automate that? Some test case in RTEMS or some
>>>>> special repository? I assume that you need some special setup for
>>>>> specific (simulator) targets anyway, don't you? So a general purpose
>>>>> test for all targets will be difficult.
>>>>>
>>>> I was not thinking about a special repository. It would be nice to
>>>> have it as test case or as an rtems-test config where the script will
>>>> capture the packets and feed them to tcpreplay.
>>>
>>> Capture from where? How do these traces deal with the differences in networks?
>> more tools :). What I was attempting is tcpdump to get the pcap and
>> then tcpreplay. I was basically in search of some way to effectively
>> test the network stacks.
>>
>>> Does tcpreply rewrite various fields, for example a DHCP server response? I
>>> suspect it does not.
>>>
>> I'm not sure.
>>
>>> Tcpreplay may work out to be a good tool we use but I think there needs to be
>>> some more ground work to see if you can "virtualize" a network. If you can do
>>> this you control the arena the devices operate in and all the characteristics
>>> and then play back would be possible and portable.
>>>
>>> VDE may be an interesting part of this. I seem to remember it has a DHCP server
>>> so if you use this with a tap that VDE manages rtems-test could use that
>>> interface.
>>>
>> That sounds like a great idea.
>>
>>> A tricky area to look into is if a VDE port can map to a physical port? If this
>>> can be done you could mix simulation and real hardware and that would be amazing
>>> to have.
>>>
>> I'll search about this.
> 
> Could be a nice application, yes. But note that this would be rather a test of
> the driver on the specific hardware. And it sounds like a rather complex setup.

I did not expect the set up I posted to be the one to use. It was an example of
using VDE.

> I'm not sure whether such a thing would be easy to automate in a way that a lot
> of people could use it.

The challenge is seeing if it can. I suspect there will need to be some set up
done. What that is, I do not know.

>> Thanks for the great ideas, I'm indeed getting some direction.
>>
>> Best regards,
>> Vijay
>>>>> If it is about testing the stacks and not the drivers, it might would be
>>>>> possible to write some kind of dummy network driver that can read pcap
>>>>> format (or some other raw packet format) directly and hands the packets
>>>>> to the network stack. Such a driver maybe could even provide packets to
>>>>> a test frame again so that you can check responses.
>>>>>
>>>> The objective is to test the stacks. The dummy network driver sounds
>>>> like a great idea, thanks. I'll explore this direction more.
>>>
>>> I prefer real drivers are used as it lets us extend testing to VLAN and
>>> multicast (driver hashes and filter testing).
>>>
>>> A solution needs to find a suitable lowest common denominator. This means
>>> hardware and simulation and possibly the different networking stack options
>>> we have.
>>>
> 
> After thinking a night over it you maybe also should note that we already have
> some tests that test the stack. For example the pf01 in libbsd uses loopback
> devices for that. You can already test big parts of the stack with that.

It would seem the requirements are still be worked out which is a good thing :)

Chris