User manual and new build system

[Joel Sherrill]

On Sun, Nov 24, 2019 at 4:44 PM Chris Johns <chrisj at rtems.org> wrote:

> On 23/11/19 1:31 am, Sebastian Huber wrote:
> > On 22/11/2019 15:25, Joel Sherrill wrote:
> >> On Fri, Nov 22, 2019, 8:19 AM Sebastian Huber
> >> <sebastian.huber at embedded-brains.de
> >> <mailto:sebastian.huber at embedded-brains.de>> wrote:
> >>     I am about to add the documentation of the new build system to the
> user
> >>     manual. There is no longer a bootstrap of the sources necessary.
> Should
> >>     I remove all information for the bootstrapping from the user manual?
> >>     Should there be a chapter for the old build system?
> >>
> >> Don't remove the old process until the code is removed. I thought we had
> >> agreed to a final autoconf based release. A release with both will help
> ensure
> >> it is functionally equivalent and provide a baseline for comparison.
> >
> > I would remove the old build system as soon as possible. Shipping a
> release with
> > two build systems makes maintenance more difficult. Getting rid of
> > Automake/Autoconf would remove these packages from the RSB build for
> example.
>
> After a some consideration over the weekend I am not sure what we should
> do and
> I think we need to discuss it some more.
>
> We need to answer the important question of user churn. A change of this
> type
> will have a flow on effect to established users and any infrastructure
> they have
> to build and deploy RTEMS. An example is the `5/bsps` build sets in the
> RTEMS, a
> build system change breaks those configurations. I am sure there are many
> other
> scripts in use today. I also know there are sites running 4.11 and 5 in
> parallel
> and building each will break.
>
> We need to decide if this churn to our users is something we want now.
>
> I currently do not favor a release with both build systems being present.
>

My big concern is ensuring that we know the new build system works exactly
like
the old one. And by works, I mean produces the same binary code with the
same
configure options expressed in the new build system's manner. Here are some
things we need to consider. These are of varying complexity and type.

(1)  Users historically test "dot zero" releases and the "dot one"
incorporates
a lot of fixes that we really wish had been in the "dot zero".  This raises
the
question of when do we get enough community testing to have confidence
the new system has no breakages from the old one?

(2) rtems-bsp-builder will need to change to support the new build system.
I think this is a must have to ensure a certain level of no breakages.

(2a)  Related: Do we know the new build system builds all the configurations
rtems-bsp-builder performs?

(3) BSPs have a number of variables which can be overridden. How will we
know these still are supported and work?

(4)  an unknown number of instructions will have to change. [This may be
time to purge/convert/fix more wiki content. This is not necessarily a bad
thing -- just a stick to beat us to do something.]

(5) Developer disruption. I am hoping this doesn't have the negative impact
switching from CVS to git did. But it could. It isn't that the switch over
itself
is bad. It's just that we can forget all the side-effects and human
retraining.

My overarching concern with all of these is "risk reduction". The primary
risk is breaking something. And the risk reduction is (1) testing before
switch
over and (2) having a reference after switch over for behavior?

So what's the criteria for switching over? I have no idea what the
acceptance
criteria is. Can we define that?

What can that reference be? I think this is from heaviest to lightest.

+ Build systems in parallel on same branch for N time?

+ final release with old build system and then kill it.

+ final autoconf snapshot known to work -- implies testing the snapshot
which
we don't do right now..

+ (lightest) tag RSB, tools, rtems, and libbsd repos with a
"pre-autoconf-death"
tag. This at least lets someone easily go back to that point in time across
the repos.
You can't do this easily with git log because you would have to find the
right place
in multiple repositories.

I probably can be convinced that the lightest option is acceptable. Whatever
the reference point is, it just needs to be documented what it is and how to
recreate it.


> If we decide it is OK to have the churn we can proceed as we currently
> are. The
> build changes can be merged and the autotools build system can be removed.
>
> If we decide we would like to make RTEMS 5 the last autotools release we
> need to
> make the release and that would mean a focused effort by more than just
> me. We
> cannot leave all the hard work done by Sebastian for a new build system on
> hold.
>
> I agree on not letting the new build system hold.



> Either path effects someone, a developer, for example Sebastian, our users
> with
> churn or me and I hope other core developers with a release.
>

Churn will occur at some point. My concern is risk.


>
> A release means getting the release itself debugged then we need to test
> it. It
> also means there may be possible changes and we need to be mindful of the
> effect
> that might have on the work Sebastian has already done.
>

And knowing that we don't have good infrastructure to test those snapshots
right now,
I would lean to having tags in the repos. We have good testing on the repos.

Perhaps after the build system switch, we can focus on testing the
snapshots.

We only have a finite amount of volunteer time to make this transition.
Where is
the effort best spent?

My gut hunch is on updating the testing infrastructure and documentation
after
the switch to get back to full speed as quickly as possible.


> I look forward to hearing what our developers and community think.
>

Me too.

I'm not trying to put extra burden, work or churn on anyone. I'm just
concerned
that this build system transition has inherent risks. We need to do what we
can
to reduce risk before the switch, provide a known (good?) reference point,
and make sure we have a complete work list. And commitment to finish that
work list.

And we all have holidays. etc coming up.


--joel


>
> Chris
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20191126/d78b1bf4/attachment.html>