[GSoC] RTEMS Tester Improvements

Chris Johns chrisj at rtems.org
Mon Mar 13 22:18:16 UTC 2017

On 14/03/2017 03:30, Gedare Bloom wrote:
> On Mon, Mar 13, 2017 at 11:50 AM, Tanu Hari Dixit <tokencolour at gmail.com> wrote:
>> Hello all,
>> I am interested in applying for GSoC under RTEMS. I am interested in the
>> idea RTEMS Tester Improvements (https://devel.rtems.org/ticket/2919). I have
>> used rtems-tester in the past and also have a working knowledge of python.
> What platforms have you run tester on?
>> I have a few questions and will be grateful if they are answered.
>> 1) Which simulator recipes need to be added?
> The first place to look is to compare what is supported in tester
> versus what we have scripts for in the older rtems-testing.git repo
> under sim-scripts/
> https://git.rtems.org/rtems-testing/tree/sim-scripts
> I personally would also be interested to add simulator recipes for the
> gem5 (gem5.org) simulator. I made a start at this a long time ago and
> got it to work reasonably well, but appear to have misplaced the
> actual changes. Related to this would be adding complementary recipes
> for building simulators in the RSB. I have run rtems under gem5 for
> the sparc64/usiii and arm/realview_pbx_a9_qemu BSPs in the past and
> would be able to help with simulator setup to get you to the point of
> transitioning from manual/shell-scripted steps into python automation.
>> 2) What are the improvements that need to be done? I couldn't find the
>> appropriate tickets or pointers.
> I believe it is primarily to increase the range of simulators that are
> supported. Perhaps Chris has other ideas for infrastructure
> improvements, e.g. there is almost always improvements that can be
> made to report usability such as visualizing aggregated testing
> results, tracking down failed tests quicker, and so forth.

The follow is the work I see needing to be done.

The current format for the BSPs and scripting is crypt and I am 
wondering if moving to YAML would help make it simpler. On top of a YAML 
change we can add per site and/or per user specifics that allow local 
hardware to be used. For example line 58 and 59 in the script 
are only useful to me and it is wrong to have this data in the repo. We 
need a way to allow users to provide the needed values they can 
configuration control for their project.

Further to this there is a need to add support for testing state 
'expected-fail' I added to the testsuite in RTEMS. We also need to add 
to each BSP the expected test results so this tool can evaluate regressions.


More information about the devel mailing list