ESA SOCIS - Fault injection tools topic

[Joel Sherrill]

On 7/29/2013 9:45 AM, Gedare Bloom wrote:
> This seems like a good project. Ballista looks like it is "dead"
> upstream. You'll want to scope your project to decide what / how much
> of RTEMS you can reasonably instrument with fault injection. I also
> wonder if it would be sensible to first start with a tool that can
> support test input "fuzzing", which is related to but somewhat
> different than fault injection. I think "fuzzing" is more
> appropriately related to test coverage /testing, whereas fault
> injection can include also non-valid inputs and faults, such as
> flipping bits in code or modifying values on-the-fly.

Fuzzing from our side would be a good test tool to have. I think it is
related to this report which was done years ago. We got the report
off their ftp site and I fixed all I could figure out.

FWIW Most of the report is junk failures. They configure their test
case with more objects than fit into memory and the BSP locked
up about 3 lines into the C code. They wrote this failure up more
than 1000 times.

Another area which we need help in is the coverage and testing
scripts. The coverage reporting really should be by directory or
capability. It is now reported in "lumps" that are too large.
Also rewriting as much of this as possible in Python including the
sim-scripts is highly desirable.

We are slowly moving to continuous integration testing and there
are many pieces in the puzzle. This is one.
> On Mon, Jul 29, 2013 at 4:25 AM, Krzysztof Mięsowicz
> <krzysztof.miesowicz at gmail.com> wrote:
>> Hi!
>>
>> My name is Krzysiek. I participated in ESA SOCIS 2012 for RTEMS and worked
>> on improving test coverage. I'm interested in participating in this edition
>> of SOCIS again.
>>
>> I browse Open Projects page of RTEMS and I have question about topic related
>> to fault injection tools. Have you any tips or suggestions to this topic
>> (especially which tools are most desired or promising and how this project
>> should look in details).
>>
>> I googled a bit about fault injection and free tools and find some:
>> Ballista, LFI, Fuzz, or JACA which potentially could be applied to RTEMS
>> testing. But I'm looking forward to any tips or thoughts from you and
>> additionaly is it possible to be selected again this year.
>>
>> Thanks in advance for your replies.
>> Krzysiek Mięsowicz
>>
>> _______________________________________________
>> rtems-devel mailing list
>> rtems-devel at rtems.org
>> http://www.rtems.org/mailman/listinfo/rtems-devel
>>
> _______________________________________________
> rtems-devel mailing list
> rtems-devel at rtems.org
> http://www.rtems.org/mailman/listinfo/rtems-devel


-- 
Joel Sherrill, Ph.D.             Director of Research & Development
joel.sherrill at OARcorp.com        On-Line Applications Research
Ask me about RTEMS: a free RTOS  Huntsville AL 35805
Support Available                (256) 722-9985