Fwd: rtems-tester

Wed Jul 29 10:48:04 UTC 2015

It looks like the following e-mail didn’t make it to the list, so I’m retrying without the patches attached.  I’ll send them with git send-mail from another machine.

	Ric

Begin forwarded message:

> From: Ric Claus <claus at slac.stanford.edu>
> Subject: Re: rtems-tester
> Date: July 22, 2015 at 3:35:14 PM GMT+2
> To: Chris Johns <chrisj at rtems.org>
> Cc: rtems-users at rtems.org
> 
> 
> On Jul 22, 2015, at 3:19 AM, Chris Johns <chrisj at rtems.org> wrote:
> 
>> On 22/07/2015 3:13 am, Ric Claus wrote:
>>> Hello All,
>>> 
>>> First, I apologize for the length of this message.  I hope you can bear with me as there are several points I’d like some help with.
>>> 
>> 
>> Thanks for the review and detailed post.
>> 
>>> I’m following https://git.rtems.org/rtems-tools/tree/doc/rtems-tester.txt to try out the xilinx_zynq_a9_qemu RTEMS BSP built with --enable-smp in the configure line.
>>> 
>>> I found the use of the term BSP in this document confusing.  "./rtems-test —list-bsps" lists names that are different from those in the RTEMS tree.  For example, modifying the example as follows won’t work:
>>> 
>>> $ ../../rtems.git/configure —target=arm-rtems4.11 \
>>>                   --enable-tests --enable-rtemsbsp=xilinx_zynq_a9_qemu_smp
>>> 
>>> Maybe some clarification here might help the next person.
> 
> I’ve attached some patches to hopefully help with this.
> 
>>> 
>> 
>> I wonder if the confusion is in some of the BSP names and the option
>> naming in the tester. I wonder if the option '--list-bsps' in the tester
>> should be '--list-testers’ ?
> 
> Good idea.  But if you do that, maybe --rtems-bsp should change, too.  Maybe to --tester, --tester-bsp or --rtems-tester, or ...?
> 
>> 
>> RTEMS uses a configure option --enable-smp to specify an SMP build while
>> the tester's configuration is fixed in the tester so we need separate
>> tester configurations for a uniprocessor test run and an SMP test run
>> for the same BSP. Testing needs to be fixed and controlled and so I do
>> not want the tester to support options, eg SMP.
> 
>> 
>> To further confuse things there are BSPs in RTEMS whose names include
>> SMP, for example realview_pbx_a9_qemu_smp. Does building
>> realview_pbx_a9_qemu_smp require the --enable-smp configure option ? I
>> do not know. There is no consistent behavior which leads to
>> realview_pbx_a9_qemu and realview_pbx_a9_qemu_smp as well as
>> xilinx_zynq_a9_qemu plus the --enable-smp option.
> 
> That brings up another idea: Maybe leave the --list-bsps and --rtems-bsp alone but let them only return and accept, resp., BSP names found in the RTEMS repository.  For the additional bits (if any), require a --bsp-variant option that takes a string that is joined with the BSP name to create the tester name.  --list-bsps could show the BSP: list of variants.
> 
> I don’t know.  I think I don’t like this either, except that it provides some clarity.
>> 
>> Unifying the arch/bsp naming is an on going effort. It is difficult to
>> change in the current build tree and my hope with the waf and RTEMS 5.0
>> effort we can get a step closer. I have introduced the BSP naming of
>> 'arch/bsp' as a standard to the RSB 3rd party building as well as the
>> waf build support for packages such as examples-v2. I would like this to
>> flow across the whole project.
> 
> Of course, this is the better idea.
>> 
>> For the short term to help users I have added a script called rtems-bsps
>> to the top of the source tree. It will find all the available BSP in the
>> source and the path helps you figure out the variants.
>> 
>>> Next, for me: 
>>> 
>>> $ arm-rtems4.11-run arm-rtems4.11/c/xilinx_zynq_a9_qemu/testsuites/samples/hello/hello.exe
>>> 
>>> returns nothing, and the GDB example returns:
>>> 
>>> (gdb) tar sim
>>> Connected to the simulator.
>>> (gdb) load
>>> Loading section .start, size 0x888 vma 0x0
>>> Loading section .text, size 0x28b2c vma 0x8c0
>>> Loading section .init, size 0xc vma 0x293ec
>>> Loading section .fini, size 0xc vma 0x293f8
>>> Loading section .rodata, size 0x1380 vma 0x29408
>>> Loading section .ARM.exidx, size 0x8 vma 0x2a788
>>> Loading section .eh_frame, size 0x74 vma 0x2a790
>>> Loading section .init_array, size 0x4 vma 0x2a804
>>> Loading section .fini_array, size 0x4 vma 0x2a808
>>> Loading section .jcr, size 0x4 vma 0x2a80c
>>> Loading section .data, size 0x970 vma 0x102800
>>> Start address 0x40
>>> Transfer rate: 1411616 bits in <1 sec.
>>> (gdb) r
>>> Starting program: /u1/home/claus/development/rtems/bld/xilinx_zynq/a9_qemu/arm-rtems4.11/c/xilinx_zynq_a9_qemu/testsuites/samples/hello/hello.exe
>>> [Inferior 1 (process 42000) exited with code 060]
>>> 
>>> Can someone explain why, please?  Should this work?  
>>> 
>> 
>> The ARM GDB simulator does not support the same Zynq hardware. The
>> rtems-bsps script shows the following are based on the gdbarmsim BSP:
>> 
>> arm1136jfs c/src/lib/libbsp/arm/gdbarmsim
>> arm1136js c/src/lib/libbsp/arm/gdbarmsim
>> arm7tdmi c/src/lib/libbsp/arm/gdbarmsim
>> arm920 c/src/lib/libbsp/arm/gdbarmsim
>> armcortexa9 c/src/lib/libbsp/arm/gdbarmsim
>> 
>> Last time I tried running these BSPs there were problems. I cannot
>> remember what they were.
> 
> Okay.  I guess I just had to learn that arm-rtems4.11-run and arm-rtems4.11-gdb with a sim target isn’t valid for some BSPs.
> 
>> 
>>> On the other hand, QEMU works and gives:
>>> 
>>> qemu-system-arm -no-reboot -serial null -serial mon:stdio -net none -nographic -M xilinx-zynq-a9 -m 256M -smp cpus=1 -kernel $(find bld/xilinx_zynq/a9_qemu/ -name hello.exe)
>>> Warning: nic cadence_gem.0 has no peer
>>> Warning: nic cadence_gem.1 has no peer
>>> 
>>> 
>>> *** BEGIN OF TEST HELLO WORLD ***
>>> Hello World
>>> *** END OF TEST HELLO WORLD ***
>>> 
>> 
>> Great.
>> 
>>> However, with 2 cores enabled:
>>> 
>>> qemu-system-arm -no-reboot -serial null -serial mon:stdio -net none -nographic -M xilinx-zynq-a9 -m 256M -smp cpus=2 -kernel $(find bld/xilinx_zynq/a9_qemu/ -name hello.exe)
>>> 
>>> it fails with:
>>> 
>>> Number of SMP cpus requested (2), exceeds max cpus supported by machine `xilinx-zynq-a9' (1)
>>> 
>> 
>> Same executable as the previous run ?
> 
> Yes.
> 
>> 
>> Where did the qemu come from ?
> 
> $DEV_ROOT/rtems/src/rtems-source-builder/bare/source-builder/sb-set-builder --prefix=$DEV_ROOT/rtems/4.11 devel/qemu
> 
>> 
>>> I verified as per the "SMP support for Raspberry Pi 2” thread that —enable-smp was really set during configure, the cpuopts.h files have "#define RTEMS_SMP 1”, and ZYNQ_CPUS = 2 in $DEV_ROOT/rtems/bld/a9_qemu/arm-rtems4.11/c/xilinx_zynq_a9_qemu/lib/libbsp/arm/xilinx-zynq/Makefile and config.status.  ticker.exe runs and smp01.exe produces no output in QEMU, when I leave the -smp parameter off.  What is the status of the ZYNQ BSPs?  Is the 2nd core nominally started, or is that something I need to work on?  Why is QEMU thinking there is only one core configured?
>>> 
>>> Next, I tried to adapt the instructions in rtems-tester.txt to use QEMU:
>>> 
>>> $DEV_ROOT/rtems/test/rtems-tools.git/tester/rtems-test \
>>> --log=log_xilinx_zynq_a9_qemu \
>>> --rtems-bsp=xilinx_zynq_a9_qemu_smp \
>>> --rtems-tools=$DEV_ROOT/rtems/4.11 \
>>> $DEV_ROOT/rtems/bld/xilinx_zynq/a9_qemu/arm-rtems4.11/c/xilinx_zynq_a9_qemu/testsuites
>>> 
>>> Initially, this failed with:
>>> 
>>> RTEMS Testing - Tester, v0.2.0
>>> error: BSP script not found: xilinx_zynq_a9_qemu
>>> 
>>> I think there is an incorrect line in:
>>> 
>>> $DEV_ROOT/rtems/test/rtems-tools.git/tester/rtems/testing/bsps/xilinx_zynq_a9_qemu_smp.mc
>>> 
>>> realview_pbx_a9_qemu should be xilinx_zynq_a9_qemu, right?
>>> 
>> 
>> Yes I agree. Patch ?
> 
> Attached.  
> 
>> 
>>> Once I made that change, the tester seems to run, but most tests fail with:
>>> 
>>> error: qemu.cfg:65: execute failed: qemu-system-arm -no-reboot -monitor none -serial stdio -nographic -net none -M xilinx-zynq-a9 -m 256M -smp cpus=2 -kernel bld/xilinx_zynq/a9_qemu/arm-rtems4.11/c/xilinx_zynq_a9_qemu/testsuites/fstests/fsbdpart01/fsbdpart01.exe: exit-code:1
>>> warning: switched to dry run due to errors
>> 
>> Is this all the log contains ?
> 
> Yes, but with a line for every test.
> 
>> 
>> Can you please try the --report-mode=all option ?
> 
> Okay, that does show that all tests are being marked invalid due to the “The number of SMP cpus...” line.  Where does one configure QEMU with the appropriate number of cores?
> 
> It stopped making progress after printing the line for tmoverhd.exe.  It’s been hanging there for the last half hour or so.  The last line in the log is:
> 
> error: qemu.cfg:66: execute failed: qemu-system-arm -no-reboot -serial null -serial mon:stdio -nographic -net none -M xilinx-zynq-a9 -m 256M -smp cpus=2 -kernel /home/claus/development/rtems/bld/xilinx_zynq/a9_qemu/arm-rtems4.11/c/xilinx_zynq_a9_qemu/testsuites/tmtests/tmoverhd/tmoverhd.exe: exit-code:1
> 
>> 
>>> 
>>> Are these all due to the "-smp cpus=2” parameter?
>>> 
>> 
>> I suppose so. Did the uniprocessor test run work ?
> 
> It looked like it, but every test had status invalid due to missing test output.  I changed qemu.cfg to modify the options from "-monitor none -serial stdio" to "-serial null -serial mon:stdio” (patch attached).  That resulted in:
> 
> Passed:   566
> Failed:     3
> Timeouts:   6
> Invalid:    1
> -------------
> Total:    576
> 
> Failures:
> smpcapture02.exe
> sp69.exe
> sptimecounter02.exe
> Timeouts:
> spintrcritical06.exe
> spintrcritical07.exe
> spintrcritical11.exe
> spintrcritical14.exe
> spintrcritical12.exe
> spintrcritical15.exe
> Invalid:
> spfatal20.exe
> Average test time: 0:00:00.665570
> Testing time     : 0:06:23.368485
> 
>> 
>> Chris
> 