libbsd network stack optimization tips & tricks
sebastian.huber at embedded-brains.de
Wed May 8 05:00:28 UTC 2019
On 07/05/2019 18:39, Jonathan Brandmeyer wrote:
> On Mon, May 6, 2019 at 12:16 AM Sebastian Huber
> <sebastian.huber at embedded-brains.de> wrote:
>> Some known problems are:
>> 1. RTEMS is supposed to be a real-time operating system and this has
>> some implications for the mutex synchronization primitives. In RTEMS an
>> atomic read-modify-write operation is used in the fast path mutex obtain
>> and release operations. Linux and FreeBSD provide only random fairness
>> (Futex) or no fairness at all, but can use only one atomic
>> read-modify-write operation in the fast path.
>> 2. The NETISR(9) service is currently disabled in libbsd:
>> 3. If you use TCP, then a lock contention in the TCP socket locks may
>> appear. This leads to costly priority inheritance operations.
> Is there a fast path available here? Ie, can we reduce the cost of
> priority inheritance by controlling the relative priorities of the
> IRQS and the receiving task? For our application, we have a
> well-defined separation between soft realtime code and hard realtime
> code. Network I/O is on the soft side of the line, so there's some
> flexibility in setting task priorities.
You can try to set the priority of the receiving task to the one of the
IRQS and set its processor affinity to processor 0.
>> 4. There is no zero copy receive available (it is just a matter of time
>> to implement it).
> Are RTEMS zero-copy sockets also based on the stack in FreeBSD?
> zero_copy_sockets(9) requires buffer sizes to be at least a whole page
> large. This NIC does not support jumbo frames. Is RTEMS also
> developing zero-copy receive for standard frame sizes? RTEMS doesn't
> use the MMU for much more than access control today, so will it simply
> remove the page-based restriction entirely?
Since there is no user space separation in RTEMS you can directly use
the mbufs in your application. See test zerocopy01.
>> I would try to use UDP and make sure your packet consumer runs on the
>> same processor as the IRQS server task. Maybe implement a zero copy
>> receive as well.
> We're using TCP's flow control to match the data flow rate to the
> hardware's capabilities. I'd rather avoid implementing manual flow
> control of a UDP stream if I can avoid it.
> There are two IRQS tasks active. Its not obvious which core ends up
> with which GEM's interrupts.
In the latest RTEMS and libbsd it should be the IRQS on processor 0.
Sebastian Huber, embedded brains GmbH
Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone : +49 89 189 47 41-16
Fax : +49 89 189 47 41-09
E-Mail : sebastian.huber at embedded-brains.de
PGP : Public key available on request.
Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.
More information about the users