[ESA summer of code, 2014] RTEMS testing

[Chris Johns]

On 14/05/2014 7:22 am, Kumar Amit Mehta wrote:
> On Tue, May 13, 2014 at 01:18:47PM -0500, Joel Sherrill wrote:
>> The names are unfortunately close so read carefully.
>>
> So there are in total four different repositories.

Yes currently there is and this is an important observation. I comment 
more below...

>> rtems/testsuites is C/C++ for test cases and executables. These are what is run on target hardware and simulators.
>>
> git://git.rtems.org/rtems.git
>

The RTEMS project's code base. This is RTEMS and its tests. It is kept 
separate from the tools and user related tools and utilities so releases 
are not joined. This means RTEMS and the user tools and utilities can 
evolve independently and on individual time lines.

>> rtems-testing/sim-scripts is bash scripts to automate running all the above tests.
>>
> git://git.rtems.org/rtems-testing.git
>
>> rtems-test is a newer Python based scripts to automate running the same tests
>>
> git://git.rtems.org/rtems-tools.git
>

This is where rtems-testing.git and rtems-source-builder.git repos will 
end up as the work evolves and those repos will go away. The work to do 
this comes under the "RTEMS Tools Project" heading and the end goal is 
to provide a single resource that provides a suitable "ecosystem" for 
users. Ecosystem is a bit of a buzz word at the moment and means the way 
users work with and interact with RTEMS on development hosts. It 
currently includes the tools, patches for the tools and testing.

The rtems-test command uses a Python framework that was taken from the 
RSB. RSB specific parts where removed and abstracted through Python 
class interfaces to provide a base for rtems-test. In time I will 
refactor the RSB and move it into this repo and making it share the same 
Python code base. The code the Python framework has a range of improvements.

The rtems-test will absorb the role the sim-scripts currently does. 
These scripts provide a great base and understanding on how to interact 
with simulators but it does not interact with gdb in a way that lets 
tests run on real hardware. The rtems-test is a single command for all 
BSPs and currently supports simulators and gdb. The sim-scripts is a 
script with specific options per BSP. The single command allows us to 
extend the testing framework to provide the ability to manage tests 
results so we can see when BSPs regress or improve. A change covers all 
BSPs at once. I will soon commit the covoar code to rtems-tools.git so 
the rtems-test command can generate coverage reports. This type of 
integration brings an important part of the RTEMS testing to the fingers 
of all users via a common user interface.

Chris

>> rtems-tools is the place we put patches for third party programs like GCC, gdb, and qemu unto they are merged into the upstream repository and released.
>>
> git://git.rtems.org/rtems-source-builder.git
>
> Thank you for clarification.
>
> Thanks,
> Kumar
> _______________________________________________
> rtems-users mailing list
> rtems-users at rtems.org
> http://www.rtems.org/mailman/listinfo/rtems-users
>