BSP maintenance (was Re: set_vector() on SPARC)

[Chris Johns]

Hi Thomas,

Thanks for responding.

On 20/10/20 5:52 pm, Thomas Doerfler wrote:
> Chris,
> 
> hm, isn't this something related to our tiering of architectures/BSPs?

The tiers as they are currently defined do not have the resolution these
questions raise. I am not sure tiers could capture the subtly of the issues I
have raised.

> Those who have a maintainer/test board should detect any fault due to
> the proposed change rather soon/at the next regular test. 

This is the intention but is it the reality and is it eve possible this will
happen? I have some doubts and the number of posted hardware test results would
seem to back this up.

The work in rtems-test I have done is about creating a documented way via our
ecosystem to publish test results. The rtems-test command is far from complete
for all use cases. It is solving my testing issues because I understand those
but it does not solve some issues others have. For example grmon and SPARC
hardware test results.

I can only encourage developers with access to boards to regularly run the
test-suite. I am also realistic that a functioning test lab does take time and
money to set up and it needs to be maintained. This year has been particularly
hard with offices being closed. The hard issue to resolve is when a change
touches boards a developer does not have access to or has not set up to be
available for testing.

I am doing everything I can to encourage users to run tests but we need to
provide them with open tools they can use. We should have those who provide
support services offer support in this area because it is important. Users have
vested interests in boards and architectures and I think they are the best
people to run the tests and report results. This may be idealistic.

I have to have a test lab because I am releasing RTEMS. I need to make sure a
base set of architectures and BSPs have test results on hardware before a
release can be made. I can only afford low cost boards like an old PC, Zedboard,
BBB, Rpi etc.

> Those who are not in constant testing... well they are not tier 1, right?

Yes but how does that fit in with idea of "being maintained" or "not being
maintained" and under what terms do we accept changes to BSPs and the interfaces
they use?

I suppose I am after some guidelines to help us manage expectations. We have the
expectations of developers making changes and those reviewing patches and we
have our users. Do we explained to our users that keeping RTEMS current and
modern means we may break BSPs and drivers? Do we explain we are in dark with
BSPs we do not see hardware test results for? I d not think we do. I am
wondering if doing so may feedback into how we maintain BSPs. Would we be more
aggressive with changes if we know a lack of test results for a BSP a user is
vested in may require extra support overheads to make work again?

I am attempting to see if can move from this being subjective to it reflecting
what we have. I would also like have it clearly understood by our users that
testing on hardware is the only means we have to know things work.

Chris

> 
> wkr,
> 
> Thomas.
> 
> Am 20.10.20 um 04:32 schrieb Chris Johns:
>> If these architectures are not being maintained as you suggest where do we
>> document this and how do we inform the users?
>>
>> If you did update the architectures to remove set_vector are they now maintained?
>>
>> Is updating the code without being able to test the changes maintaining an
>> architecture or BSP or do we require some level of testing?
>>
>