[Bug 1549] It must be easier to write tests

[bugzilla-daemon at rtems.org]

https://www.rtems.org/bugzilla/show_bug.cgi?id=1549


Chris Johns <chrisj at rtems.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |chrisj at rtems.org




--- Comment #2 from Chris Johns <chrisj at rtems.org>  2010-06-10 20:35:49 ---
> 1. An extra directory for each test case is not needed.  We can use different
> file prefixes for each test case.

That is true but all files in one directory is not such a great idea either.

> 4. The *.doc files are pretty useless in the current format.  We should define
> guidelines how do to document tests.  We should write them in a machine and
> human readable form.  I propose Doxygen.  If someone does not like Doxygen it
> is trivial to write a parser that transforms the Doxygen comments into
> something else.

I support doxygen as an internal software documentation tool how-ever I am
currently not convinced about using it for formal external documentation. We
already have a documentation format, texinfo. 

Can the test docs be written in texinfo format and included from a top level
document in the current doc tree to create a test document ? The upside here is
the ability to include this in the current manuals, daily documentation build
etc without needing to include a new standard, tool or custom filter script.

> 5. These files are useless and hard to maintain.  The test output should
> clearly state if a test failed or passed or whatever.  The test output should
> be in a standard format to allow programs to determine the test outcome easily.

Joel and I have been chatting about the rtems-testing tree. We are looking to
take this source and to autotools it so it can be installed to a $PREFIX with
any host specific issues managed. If this works out and passes critical reviews
we hope the package could become a noarch RPM. The idea is to develop the
ability to test on known simulators and to have a suitable means to know if a
test has passed, failed, expected to fail, or unexpected pass. The hope is
opening this infrastructure up and providing it with easy to use documented
commands will improve the quality of patches received by RTEMS. To this end the
screen trace files will become important. The feeling is a parser is needed to
manage the test output and the provided trace data to determine the test
result. A direct diff does not work. What is needed will unfold over time as
this work progresses.

-- 
Configure bugmail: https://www.rtems.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.