New validation test suites

[Chris Johns]

On 15/2/22 12:43 am, Sebastian Huber wrote:
> On 03/02/2022 09:09, Sebastian Huber wrote:
>> On 30/01/2022 23:26, Chris Johns wrote:
>>>>>> We just have to create a
>>>>>> release branch for RTEMS 6 and reference release branch commits in the
>>>>>> submodules.
>>>>> How do you validate the generated sources in the sub-modules match those in
>>>>> the
>>>>> branched submodules? I think this should be done as part of the release
>>>>> procedure to catch any mistakes.
>>>> I will write something about this in the release procedure documentation.
>>> Thank you, I appreciate this. This is a big help.
>>
>> I added a note to the post-branch procedure:
>>
>> https://docs.rtems.org/branches/master/eng/release-process.html#post-branch-procedure
> 

Thank you.

I noticed the documenting of branches in the procedure. Is it required they be
present to run the procedure? An important phase of making a release is the
release candidates and these are run off master. Branching before we know we are
close to working is an important aspect of the pre-release testing as it brings
a range of parts together. This should happen before branching.

> How do we want to proceed here?

I am currently rather busy on something and will be for a while. As a result I
have been a little distracted and I apologise for this.

> I have no idea what should I do next.

As has been stated, making a release takes some time and effort and my
experience of doing it has taught me all the detail needs to be covered. Until I
can see rtems-central integrated into a release we cannot not depend on it.
Given the way the validation tests have been created merging makes rtems.git
depend on rtems-central if merged. In the past we have not taken care of these
things and it has fallen to me to sort out and I prefer that does not happen in
this case.

I should also state the rtems-central repo is on probation and even if it is
added to a release it will still remain like that for a reasonable period of
time. It is still not clear where the line of responsibility lies with that repo
and the long term maintenance of the qual effort. This may appear abrupt however
I need to be clear about this so there is no misunderstanding.

> The new validation tests are not the only work from the pre-qualification
> activity I would like to integrate. We also have work from Andrew Butterfields
> related to formal methods. There are changes in the GR712RC and GR740 BSPs, the
> gcov code coverage support, and build system changes to support the build of
> RTEMS libraries which contain only the pre-qualified feature set.

Making a release is documented here:

https://git.rtems.org/rtems-release/tree/README.txt

I run a single command that creates a complete release. How will the procedure
outlined in the eng manual be implemented here? [1]

What will an rtems-central tarball look like? I am specifically concerned the
export process to turn a git repo into a tarball will include a copy of the
sub-modules. This is the current way the release scripts work. If this is the
case we will have 2 copies of some repos in the release and I think that is a
bad idea.

Chris

[1] The release scripts assume an autoconf RTEMS and so I suspect what is there
is broken. This makes the rtems-central integration more complicated than it
need be. I have not looked at what is needed.