[GSoC] RTEMS Tester Improvements

[Tanu Hari Dixit]

Hello Chris, Gedare, Joel,
Thank you for answering my questions. Sorry for the late reply, it
took me time to go through the tasks suggested.
>
> What platforms have you run tester on?

I was searching for a way to compile and execute a single test and the
community suggetsed rtems-tester. So I ran the tester on sparc/erc32
(and no simulator).

I am unable to find a relevant doc on how to run rtems-tester on qemu.
Though there is this rtems-tools.git/tester/rtems/testing/qemu.cfg so
I know that it is supported. Also I couldn't find any other hint in
the rtems-tools repository as to which simulators (except for qemu)
are supported. rtems-testing/sim-scripts support gdbsim, skyeye and
tsim (other than qemu). So we can add support for these and gem5 (as
Gedare suggested). Please correct me if I am wrong.

Furthermore, if you could please point me to a doc that explains using
rtems-tester on qemu? I have tested qemu for running an application,
the steps for which I have documented here
(http://tokencolour.github.io/2017/03/16/how-to-run-applications-on-qemu-rtems.html).

I can play with visualizing aggregated testing results with plotting
techniques (like matplotlib or maybe something that the devs
recommend) :).

Also, where can I find more about "expected fail" state? Is it for
those tests that fail nevertheless and need to be skipped? Can you
please point me to an example? Also, what kind of regression will one
need to perform? Do the final states of the test need to maintain an
overall state of the system so as to recommend/predict the state of
affairs next time the tests are run (i.e. a predictor (which has
memory) kind of thing)?

I also think that converting macro files to yaml won't be very
difficult. I might be very myopic here and it would be great if I get
a heads up. What will be the challenge here?

For the generic serial console, will pySerial do the trick, on
Windows? Do you have some other package in mind?

I have a problem here, though. I don't own a Zedboard and can't afford
one. I can check with my department at college once I reach there, but
I don't have my hopes too high. Will this pose a problem in the course
of the project?

I am yet to build for xilinx_zynq_zc706, so I'll ask more questions on
the configuration control once I am done with it. Also, I'm trying out
the old and new approach for Coverage reporting. I'll ask questions on
it, once I try it.

Thank you,
Tanu Hari Dixit.

On Wed, Mar 15, 2017 at 2:37 AM, Chris Johns <chrisj at rtems.org> wrote:
> On 15/3/17 1:02 am, Gedare Bloom wrote:
>> On Tue, Mar 14, 2017 at 9:58 AM, Gedare Bloom <gedare at rtems.org> wrote:
>>> On Tue, Mar 14, 2017 at 1:50 AM, Chris Johns <chrisj at rtems.org> wrote:
>>>> On 14/3/17 9:18 am, Chris Johns wrote:
>>>>>
>>>>> The follow is the work I see needing to be done.
>>>>>
>>>>
>>>> I have created a ticket for this project. Please update and add
>>>> BSP/Simulator section to track those tasks.
>>>>
>>>>  https://devel.rtems.org/ticket/2927
>>>>
>>> Does this supersede https://devel.rtems.org/ticket/2919
>>>
>>
>> Note: I classify Testing Tool Improvements as part of improving our
>> Developer Ecosystem rather than as a Testing project. A subtle
>> distinction perhaps, but I feel it is important to understand where we
>> draw these kinds of lines.
>>
>
> I agree, one side is what a test is and how to create a test executable
> and as you say this project is about the ecosystem used to execute these
> tests and manage the results.
>
> Chris
> _______________________________________________
> devel mailing list
> devel at rtems.org
> http://lists.rtems.org/mailman/listinfo/devel