New validation test suites

[Chris Johns]

On 17/1/22 7:04 pm, Sebastian Huber wrote:
> On 17/01/2022 08:52, Chris Johns wrote:
>> My understanding of the status of these patches is the remaining topic is the
>> release dependencies. I have not had time to give this any consideration however
>> I have a feeling it will not be easy or simple because of the inter-dependency
>> of the repos and sub-module relationship. I hope I am wrong about this.
> 
> In order to work with the generated test programs you have to use the
> specification items which are in rtems-central.  Most of the validation tests
> are defined through transitions from pre-conditions to post-conditions. Working
> directly with the generated sources is too complicated. I don't know what
> complicates a release here, the rtems-central is just another repository which
> needs a release branch. In the release branch, the submodules track the
> corresponding release branch of the referenced repository.

A release has tarballs of sources and not git repos so I am not sure how
branches help? How does the submodules in rtems-central get captured? The
current release scripts expand sub-modules which means they need to reconcile or
we will have different copies of the code in the release.

At the point of release how does the release manager know the rtems-central
scripts match the generated sources in the down stream repos that are released?

> Please note, that the RTEMS sources used by rtems-central are currently not
> one-to-one a commit of the RTEMS master branch. There are about 90 additional
> patches. The patch set with the new validation sets is a part of it.

Which is a concern and why a release needs to check and make sure everything is
in sync.

Ideally the rtems-central submodule hashes match the release tarball sources for
each repo. This is the task as I see it.

I am sure this can be resolved and I prefer it happens before being pushed so it
is not left as a task for the release manager.

Chris