RTEMS classic networking: fixed FXP driver works under QEMU on Linux

[Pavel Pisa]

Hello all,

the driver works after hours of problem seeking
in incorrect directions.

I have debugged, examined and patched both, RTEMS and QEMU.
The main problem is quite simple. Update of RTEMS interrupts
processing disable level type interrupts when they arrive
and the driver/daemon has to re-enable interrupt source
on the interrupt controller level.

Generally, idea to disable interrupt source at hard interrupt
time and do all processing outside of interrupt is the
the best solution for RT system. But I consider actual
behavior seriously broken. It is for longer description.
But if you consider shared interrupts (all PCI ones on PC
for example) then the correct behavior is to use hard IRQ
to disable/gate interrupt on given device level
(not all sharing devices at controller level) and release
worker thread for each device from corresponding
hard IRQ and left the scheduler to select between these
according to the priority. When more important/critical
device finishes its IRQ processing, it enables IRQ
on given device level and processing of its interrupts
in time is possible.

Actual state pushes device drivers to attempt re-enable
IRQ on controller level. But if IRQ is re-enabled by hight
priority device worker thread in case that there exists
lower priority device sharing the IRQ which is still signalling
IRQ, then without gating on device level this has to lead
to livelock. Cotroller fires IRQ dispatch, that releases
high priority device driver, that finds nothing to do and reenables
IRQ on the controller level.

There is option to make things work even with shared IRQs
if gating on device level is not available. And it is
to have counted disable on the controller level which ensures
that IRQ is re-enabled only after all devices worker
threads finish processing. But priority rules are broken in that
case as well.

I know that there is interrupt server option but base
interrupts should be working and not broken.

Shared edge trigerred interrupts processing is complex
problem either. I think that I have provided analysis
years ago for RTEMS.

....

found it

https://lists.rtems.org/pipermail/users/2008-May/018775.html

If I could find month somewhere, I would try provide
changes which could go under critique and testing .....

Anyway, back to solve Saeed Ehteshamifar problem with
lack of network supporting environment for his dynamic
loading task. I have tested it on Linux, Debian.
I have done setup without helper scripts and toolsfrom
QEMU or other system. I decided to use separate
"network segment" for testing. The wire for L2/ethernet
layer is created by
 
  ip tuntap add tap1 mode tap user pi
  ip link set tap1 up

You need root access.
I have connected this stub to Linux TCP/IP networking
subsystem by

  ip addr add 192.168.3.1/24 dev tap1
  ip link set tap1 up

You need to select address from private address range.
Check that whole range, in above case 192.168.3.0
to 192.168.3.255 does not overlap with networks
visible from your computer. It should be really
isolated island. The kernel does default setup
of routing to the range

  ip route show

  192.168.3.0/24 dev tap1  proto kernel  scope link  src 192.168.3.1

To keep range separate you should not have enabled forwarding

  cat /proc/sys/net/ipv4/ip_forward

should show 0 or you need to setup iptables or check and set
routing rules to keep island network.

I start RTEMS application by command

qemu-system-x86_64 -enable-kvm -kernel $APP_BINARY \
      -vga cirrus \
      -append "--console=/dev/com1" \
      -serial stdio \
      -net nic,vlan=1,macaddr=be:be:be:10:00:01,model=i82557b \
      -net tap,ifname=tap1,vlan=1,script=no,downscript=no  \

You can add "-s -S" for debugging by GDB "target remote localhost:1234"
Be carefull, if you want to set breakpoints then setting them when
in real mode at address 0xffff0 is not good idea. You need
to be in virtual space of RTEMS application after its load
to set software breakpoints. But there are two or three HW
breakpoints emulated. Decide for some function and use

  hbreak rtems_fxp_attach
  c

When the function is reached you can insert regular SW
breakpoints.

When application runs, you can access it from Linux system.
The configured address has to be withing island network range.
I.e.

  ping 192.168.3.66

The netwok card should be found by ARP broadcast coming
to QEMU where RTEMS responds that it is its address.
You can use 

  arping -i tap1 192.168.3.66

or 

 arping -i tap1 be:be:be:10:00:01

to check lowest level connection.

The OMK template includes application "appnet" which uses
RTEMS integrated telnet to access RTEMS shell remotely.

  telnet 192.168.3.66

See

  http://rtime.felk.cvut.cz/gitweb/rtems-devel.git/tree/HEAD:/rtems-omk-template/appnet

The config

static struct rtems_bsdnet_ifconfig netdriver_config = {
        .name = "fxp1" /*RTEMS_BSP_NETWORK_DRIVER_NAME*/,
        .attach = rtems_fxp_attach /*RTEMS_BSP_NETWORK_DRIVER_ATTACH*/,
        .next = NULL,
        .ip_address = "192.168.3.66",
        .ip_netmask = "255.255.255.0",
        .hardware_address = ethernet_address,
        .ignore_broadcast = 0,
        .mtu = 0,
        .rbuf_count = 0,
        .xbuf_count = 0,
        .port = 0,
        .irno = 0,
        .bpar = 0,
        .drv_ctrl = NULL
};

struct rtems_bsdnet_config rtems_bsdnet_config = {
        .ifconfig = &netdriver_config,
/*      .bootp = rtems_bsdnet_do_bootp,*/
/*      .bootp = rtems_bsdnet_do_dhcp,*/
#if 0
        .bootp = rtems_bsdnet_do_dhcp_failsafe,
#endif
        .network_task_priority = 0,             /* 100        */
#if 1
        .mbuf_bytecount = 256 * 1024,           /* 64 kbytes  */
        .mbuf_cluster_bytecount = 256 * 1024,   /* 128 kbytes */
#else
        .mbuf_bytecount = 2048 * 1024,          /* 2 MB */
        .mbuf_cluster_bytecount = 2048 * 1024,  /* 2 MB */
#endif
        .hostname = NULL,
        .domainname = NULL,
        .gateway = "192.168.3.1",
        .log_host = NULL,
        .name_server = { "NULL" },
        .ntp_server = { "NULL" },
        .sb_efficiency = 0,
        .udp_tx_buf_size = 0,                   /* UDP TX: 9216 bytes */
        .udp_rx_buf_size = 0,                   /* UDP RX: 40 * (1024 + sizeof(struct sockaddr_in)) */
        .tcp_tx_buf_size = 0,                   /* TCP TX: 16 * 1024 bytes */
        .tcp_rx_buf_size = 0,                   /* TCP TX: 16 * 1024 bytes */
};

This is for static IP addresses on isolated island network.
You can interconnect the island with external systems
over that IP level forwarding or use ETHERNEL level bridge.
Something like

  brctl addbr my_bridge
  ip link set my_bridge up
  brctl addif my_bridge eth0
  brctl addif my_bridge tap1
  brctl show my_bridge

or by newer command

  ip link add my_bridge type bridge
  ip link set tap1 master my_bridge
  ip link set dev eth0 down
  ip addr flush dev eth0
  ip link set dev eth0 up
  ip link set eth0 master my_bridge
  ip link set dev my_bridge up

If you do not need to connect to the system running under QEMU
from host system, then you can use simple NAT integrated in QEMU
for outgoing connections. QEMU options

      -net nic,vlan=1,macaddr=be:be:be:10:00:01,model=i82557b \
      -net user,vlan=1 \

Best wishes,

               Pavel

PS: it would be nice to have some example for libbsd based x86
    or other arch QEMU setup on Linux.