[PATCH v2] eng: Add guidelines for validation tests

[Gedare Bloom]

On Thu, Jun 15, 2023 at 11:09 PM Sebastian Huber
<sebastian.huber at embedded-brains.de> wrote:
>
> Update #3717.
> ---
> v2:
>
> * Mention that file name should be unique.
> * Clarify file name parts.
> * Add examples for enabled-by expressions.
>
>  eng/req/howto.rst | 80 +++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 80 insertions(+)
>
> diff --git a/eng/req/howto.rst b/eng/req/howto.rst
> index 0de3c01..125dac5 100644
> --- a/eng/req/howto.rst
> +++ b/eng/req/howto.rst
> @@ -1127,6 +1127,86 @@ the following post-condition states.
>            parameter in past calls to ${../if/directive:/name} shall not be
>            accessed by the ${../if/directive:/name} call.
>
> +Validation Test Guidelines
> +--------------------------
> +
> +The validation test cases, test runners, and test suites are generated by the
> +``./spec2modules.py`` script from specification items.  For the placement and
> +naming of the generated sources use the following rules:
> +
> +* Place architecture-specific validation test sources and programs into the
> +  ``testsuites/validation/cpu`` directory.
> +
> +* Place BSP-specific validation test sources and programs into the
> +  ``testsuites/validation/bsps`` directory.
> +
> +* Place all other validation test sources and programs into the
> +  ``testsuites/validation`` directory.
> +
> +* Place architecture-specific unit test sources and programs into the
> +  ``testsuites/unit/cpu`` directory.
> +
> +* Place BSP-specific unit test sources and programs into the
> +  ``testsuites/unit/bsps`` directory.
> +
> +* Place all other unit test sources and programs into the
> +  ``testsuites/unit`` directory.
> +
> +* Use dashes (``-``) to separate parts of a file name.  Use only dashes, the
> +  digits ``0`` to ``9``, and the lower case characters ``a`` to ``z`` for file
> +  names.  In particular, do not use underscores (``_``).
> +
> +* The parts of a file name shall be separated by dashes and ordered from most
> +  general (left) to more specific (right), for example ``tc-task-construct.c``.
> +
> +* The file names associated with tests shall be unique within the system since
> +  the test framework prints out only the base file names.
> +
> +* Use the prefix ``tc-`` for test case files.
> +
> +* Use the prefix ``tr-`` for test runner files.
> +
> +* Use the prefix ``ts-`` for test suite files.
> +
> +* Use the prefix ``tx-`` for test extension files (test support code).
> +
> +* Tests for fatal errors shall have ``fatal`` as the most general file part,
> +  for example ``ts-fatal-too-large-tls-size.c``.
> +
> +* Validation test suites shall have ``validation`` as the most general file
> +  part, for example ``ts-validation-no-clock-0.c``.
> +
> +* Unit test suites shall have ``unit`` as the most general file part, for
> +  example ``ts-unit-no-clock-0.c``.
> +
> +* Architecture-specific files shall have the architecture name as a file part,
> +  for example ``ts-fatal-sparc-leon3-clock-initialization.c``.
> +
> +* BSP-specific files shall have the BSP family or variant name as a file part,
> +  for example ``tc-sparc-gr712rc.c``.
> +
> +* Architecture-specific or BSP-specific tests shall use the ``enabled-by``
> +  attribute of the associated specification item to make the build item
> +  conditional, for example:
> +
> +  .. code-block:: yaml
> +
> +      ...
> +      build-type: objects
> +      enabled-by: arm
> +      type: build
> +      ...
> +
> +  .. code-block:: yaml
> +
> +      ...
> +      build-type: test-program
> +      enabled-by: bsps/sparc/leon3
> +      type: build
> +      ...
> +
> +
> +
I think too many new lines

Otherwise, looks good

>  Verify the Specification Items
>  ------------------------------
>
> --
> 2.35.3
>
> _______________________________________________
> devel mailing list
> devel at rtems.org
> http://lists.rtems.org/mailman/listinfo/devel