stress program in RTEMS

[Joel Sherrill]

Sorry for a long email. I think a button got pushed. :)

On 10/23/2013 2:31 AM, Bornet Romain wrote:
> Hi!
> 
> I'm not aware of any standard stress test but I had a look at the ones presented here: ftp://ftp.rtems.com/pub/rtems/esa_validation_report_450/RTEMS_DependabilityTesting/index.html
> ftp://ftp.rtems.com/pub/rtems/esa_validation_report_450/RTEMS_DependabilityTesting/ 
> 
> There are stress and robustness tests explained in details and whose implementation is available...

Where is this code available? I thought it was dependent
on a proprietary tool. I don't know of anything that was
in that report that is completely publicly available.
I personally never saw any code on that and all opinions
from here down are from the reports and my > 10 year old
recollection. Hopefully someone at ESA/ESTEC can comment.
If the code/tests are still useful, then it would be
nice to have them.

I haven't looked at those > 10 years but recall that
not much came out of it. The folks doing the work uncovered
a few things that were useful and wrote up a lot of cases
that were pointless. They pushed arbitrary values for each
data type through the API calls they were interested in.
This is useful but doesn't say anything about real applications.

This is the technical list of what they found:

+ the Classic API did not check for NULL parameters. This
was per the original specification and was by design. 0x0000
was a valid address on some targets at that point in history.
We added NULL checks in response.

+ A few range checks on parameters were incorrect. These
were the real bugs found. It wasn't many.

+ A sentence in the manual was inverted. It had a not or
was missing a not. I don't recall which. The code was correct.

Most of the failure cases documented were for do something
like configuring a system with 1MB RAM with 1000 tasks and being
surprised to get a fatal error early in the BSP. At the time
on the BSP they used, this happened on the second line of code
executed in bsp_start(). It failed silently. Not perfect but not
worth writing up > 1000 times.

As a matter of perspective, I asked for a period of off-contract
paid time from OAR to investigate this report. Every issue was
easy enough that I fixed them while reading the report.
The "estimation" phase resulted in all the issues being
fixed. No further engineering was done. 4.6 has a gap in the
numbering because a release was being staged and I stopped it.
We bumped the last digit and that one went out a few days later
with all issues fixed.

This report is also my prime example of "please ask for help".
The work was done, reports written and contract closed before
we ever heard about it. We didn't get to review, comment, etc.
I use this as an example of please let a core developer review
any reports. If I hadn't felt pride in RTEMS, this report would
have been ignored and had zero impact. If we had talked with the
people doing the work, we could have addressed the issues as
they went and possibly improved their work with our insights.

> I didn't run them on my system until now but kept the link "in case I have time"...
> 
>>> I'm currently writing a new article about RTEMS.
> Looking forward to read it ,hopefully in GLMF[http://www.unixgarden.com/index.php/category/gnu-linux-magazine] or 
> Open Silicium[http://www.unixgarden.com/index.php/category/open-silicium] :-)
> 
> Hope this helps,
>     Romain Bornet
> 
> 
> -----Original Message-----
> From: rtems-users-bounces at rtems.org [mailto:rtems-users-bounces at rtems.org] On Behalf Of Pierre Ficheux
> Sent: mercredi 23 octobre 2013 08:18
> To: rtems-users at rtems.org
> Subject: stress program in RTEMS
> 
> Hi,
> 
> I'm currently writing a new article about RTEMS. I'm looking for an example of stress task such as "stress" or "hackbench" on Linux.
> 
> What's the best way to generate stress for RTEMS ? using floating floating point operation?
> 
> Thx
> 



-- 
Joel Sherrill, Ph.D.             Director of Research & Development
joel.sherrill at OARcorp.com        On-Line Applications Research
Ask me about RTEMS: a free RTOS  Huntsville AL 35805
Support Available                (256) 722-9985