[Bug 1549] It must be easier to write tests

[bugzilla-daemon at rtems.org]

https://www.rtems.org/bugzilla/show_bug.cgi?id=1549

--- Comment #11 from cynt6007 at vandals.uidaho.edu 2013-04-06 18:12:21 CDT ---
(In reply to comment #10)
> Cynthia.. does the title of this PR " It must be easier to write tests" get
> addressed in the Open Project? 
> 
> Looking back over the discussion, that is the only thing that was never
> discussed. We now have a set of test templates to state from and a script to
> instantiate those templates as starting points. I am sure could be more
> templates.
> 
> Are we trying to have a standard way to document the requirements met by a
> test?
> 
> And tie the requirements to source, tests and coverage?

I copied the description into
http://www.rtems.org/wiki/index.php/RTEMS_Test_Template and noted what was
completed and what was left to do...

There were several comments that went into different directions, but I didn't
completely capture all of those, as they were not in the description, but below
is my understanding of them, and where they could be addressed...

Comment 1: looks like Sebastian Huber solved part of the problem.
Comment 2: new stuff was added, looks like requirements traceability and
simulator verification of screen-shots.  I should explicitly state requirements
traceability in http://www.rtems.org/wiki/index.php/RTEMS_Test_Specification ,
and simulator verification in
http://www.rtems.org/wiki/index.php/RTEMS_Test_Screen_Validation
Comment 3: a new thread related to some sort of coverage testing hook, which I
thought was probably address in
http://www.rtems.org/wiki/index.php/RTEMS_Test_Coverage
Comment 4: back to requirements traceability, but this time, embedded into the
code. I could explicitly state requirements traceability in
http://www.rtems.org/wiki/index.php/RTEMS_Test_Specification is to be embedded
in the init.c
Comment 5: some vague argument related to what format the output text should be
in... I'll probably write up the tests should be commented in RTEMS doxygen
format and gloss over with a statement that there must exist a technology to
convert doxygen comments into .texi (possibly even doxygen does this, but I'm
not sure)
Comment 6: some question as to where to place comments, and the format of them,
probably could be in
http://www.rtems.org/wiki/index.php/RTEMS_Test_Specification

Comment 7: Is there a specific direction you wanted to go with this? There are
some really good ideas for how to write up the additional issues brought up in
the previous comments.

My personal take on this is the write-ups will need to be easy to understand
and the results easy to maintain.  Although doxygen may not be the "be all" and
"end all" of documentation, it would meet Sebastian's desire to have the
comments in the same file as the code. The way to do requirements traceability
using doxygen would be: 
Grouping by class using something akin to.
http://www.longacre-scm.com/blog/index.php/2010/08/using-doxygens-test-command-with-c
For traceability, add custom doxygen tags, such as: 
@requirement (in the the source file)
and
@see (in the test file pointing to the functions being tested)
Again, if doxygen is not the format of choice, the tags should help with
requirements traceability using another static analysis tool, it's just that we
use doxygen in other parts of the code.

IMO the doxygen commenting (in a way towards getting tests standards compliant)
of the existing tests should be a GCI task, which I will work on writing up, if
that would meet the need.

-- 
Configure bugmail: https://www.rtems.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.