[Bug 1549] It must be easier to write tests

[bugzilla-daemon at rtems.org]

https://www.rtems.org/bugzilla/show_bug.cgi?id=1549





--- Comment #7 from Joel Sherrill <joel.sherrill at oarcorp.com>  2010-06-11 07:36:12 ---
PRs should be about single issues.  There is too much in this PR.

I thought from your initial list of issues that it would propose some cleanup
on the Makefile.am's and a script to generate a template for a new test.  That
would IMO be a valid PR and a useful thing. 

There are other issues which should be other PRs.  And I question whether they
should be PRs at first or discussions on the mailing list to get broader
visibility until an approach is accepted by the community.

I see these issues which need community wide (not PR yet) discussion.  And they
should NOT be mixed.  They are independent.

+ One test per directory or multiple: 

Personally I like self-contained tests but I first defer to Ralf on a
feasibility statement since it depends on Makefile structure.  Given 
feasibility, we can discuss advantages/disadvantages.

+ Switching from Texinfo for user manuals.

Personally a large investment for nothing IMO and I would switch to Docbook
like the Linux Manuals.  But not a PR issue yet and this is labour intensive
for
little pay off.  So it would have to have HUGE payoff and a clear way to
do a good job of the conversion.

+ Test screen validation.

Chris and I have discussed building some infrastructure to check the screens
against the target output.  I am hoping this is a 4.11 issue.  We do not
have a technical approach yet.

+ Test documentation

Yes the current docs suck.  But Doxygen is not the tool for every job.  
I think we need test documents which tie the test back to specific 
requirements.  And the code should reference those requirements.
And with this information, the coverage engine could verify that we
have executed the code that implements the requirements.  This kind of
approach is necessary as we attempt to address the guidelines from
specifications for flight software (e.g. DO-178B and similar).

So IMO the test docs should be requirements references.  The code should
point back to standards, etc

If the test is a "sample" or "example" then it is not testing requirements
but showing the user how to do something.  That needs a README.

Doxygen is NOT a requirements management tool.

-- 
Configure bugmail: https://www.rtems.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.