[PATCH v2] eng: Add guidelines for validation tests

[Sebastian Huber]

Update #3717.
---
v2:

* Mention that file name should be unique.
* Clarify file name parts.
* Add examples for enabled-by expressions.

 eng/req/howto.rst | 80 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 80 insertions(+)

diff --git a/eng/req/howto.rst b/eng/req/howto.rst
index 0de3c01..125dac5 100644
--- a/eng/req/howto.rst
+++ b/eng/req/howto.rst
@@ -1127,6 +1127,86 @@ the following post-condition states.
           parameter in past calls to ${../if/directive:/name} shall not be
           accessed by the ${../if/directive:/name} call.
 
+Validation Test Guidelines
+--------------------------
+
+The validation test cases, test runners, and test suites are generated by the
+``./spec2modules.py`` script from specification items.  For the placement and
+naming of the generated sources use the following rules:
+
+* Place architecture-specific validation test sources and programs into the
+  ``testsuites/validation/cpu`` directory.
+
+* Place BSP-specific validation test sources and programs into the
+  ``testsuites/validation/bsps`` directory.
+
+* Place all other validation test sources and programs into the
+  ``testsuites/validation`` directory.
+
+* Place architecture-specific unit test sources and programs into the
+  ``testsuites/unit/cpu`` directory.
+
+* Place BSP-specific unit test sources and programs into the
+  ``testsuites/unit/bsps`` directory.
+
+* Place all other unit test sources and programs into the
+  ``testsuites/unit`` directory.
+
+* Use dashes (``-``) to separate parts of a file name.  Use only dashes, the
+  digits ``0`` to ``9``, and the lower case characters ``a`` to ``z`` for file
+  names.  In particular, do not use underscores (``_``).
+
+* The parts of a file name shall be separated by dashes and ordered from most
+  general (left) to more specific (right), for example ``tc-task-construct.c``.
+
+* The file names associated with tests shall be unique within the system since
+  the test framework prints out only the base file names.
+
+* Use the prefix ``tc-`` for test case files.
+
+* Use the prefix ``tr-`` for test runner files.
+
+* Use the prefix ``ts-`` for test suite files.
+
+* Use the prefix ``tx-`` for test extension files (test support code).
+
+* Tests for fatal errors shall have ``fatal`` as the most general file part,
+  for example ``ts-fatal-too-large-tls-size.c``.
+
+* Validation test suites shall have ``validation`` as the most general file
+  part, for example ``ts-validation-no-clock-0.c``.
+
+* Unit test suites shall have ``unit`` as the most general file part, for
+  example ``ts-unit-no-clock-0.c``.
+
+* Architecture-specific files shall have the architecture name as a file part,
+  for example ``ts-fatal-sparc-leon3-clock-initialization.c``.
+
+* BSP-specific files shall have the BSP family or variant name as a file part,
+  for example ``tc-sparc-gr712rc.c``.
+
+* Architecture-specific or BSP-specific tests shall use the ``enabled-by``
+  attribute of the associated specification item to make the build item
+  conditional, for example:
+
+  .. code-block:: yaml
+
+      ...
+      build-type: objects
+      enabled-by: arm
+      type: build
+      ...
+
+  .. code-block:: yaml
+
+      ...
+      build-type: test-program
+      enabled-by: bsps/sparc/leon3
+      type: build
+      ...
+
+
+
 Verify the Specification Items
 ------------------------------
 
-- 
2.35.3