GitLab and BuildBot

Tue Feb 7 22:37:14 UTC 2023

On 7/2/2023 9:31 pm, Christian MAUDERER wrote:
> On 2023-02-07 07:03, Chris Johns wrote:
>> On 30/1/2023 10:12 pm, Christian MAUDERER wrote:
>>> Hello,
>>>
>>> recently the following tickets were added (beneath a few more related ones):
>>>
>>> https://devel.rtems.org/ticket/4790 - Setup Gitlab instance
>>> https://devel.rtems.org/ticket/4791 - Update BuildBot
>>>
>>> It's great that a patch review system and a CI/CD that builds every patch for
>>> RTEMS starts to get within reach. Thanks a lot to all involved in that for the
>>> efforts.
>>
>> It is exciting to see this happening. It will benefit the project and all who
>> give their time.
>>
>>> I reviewed earlier discussions related to CI/CD. From my point of view there are
>>> mainly two points that are missing in the tickets:
>>>
>>> First: From my point of view, we should make it simple for new users to
>>> register. Adding authentication using well-known services can help with that.
>>> GitLab supports (for example):
>>>    - GitHub: https://docs.gitlab.com/ee/integration/github.html
>>>    - GitLab.com: https://docs.gitlab.com/ee/integration/gitlab.html
>>>    - Google: https://docs.gitlab.com/ee/integration/google.html
>>>    - ...
>>> I think it would be good to select the most common ones (at least the three
>>> mentioned above) and add them as a goal to the ticket or a new one. What do you
>>> think?
>>
>> I suggest a ticket to handle authentication. If you create a ticket please
>> indicate it is unfunded. If this is handled else where and in another ticket
>> this one can be closed with a suitable reason.
>>
> 
> https://devel.rtems.org/ticket/4840

Thanks.

>> The addition of these authentication methods can be done when someone funds the
>> work. If a person or organisation thinks it is important please reach out to
>> Amar directly.
>>
>>> Second: It's still a bit unclear for me how the CI/CD with BuildBot will work.
>>> Will it be possible for anyone to help improve the CI/CD? An example to make it
>>> clear what I want to know: Let's assume an unprivileged developer has a patch
>>> set that allows building device tree files using the RTEMS build system, but the
>>> patches require a new tool like dtc. Let's further assume that the idea has been
>>> discussed and everyone agrees that it is a good idea (currently not yet the case
>>> for dtc). Problem is: The patches trigger a CI error because the new tool is
>>> missing and therefore can't be merged yet. How can the developer suggest a fix
>>> so that the patches can be accepted faster without having to wait for one
>>> specific maintainer to have enough time for adapting the CI config?
>>
>> There are details that will need to be worked out. Deployment of tools for
>> building in a CI flow is important. How complex and automated this will be will
>> depends on the funding provided. At this point in time the push is to get a
>> basic framework up that allows us to handle simple merge requests. The approach
>> is more organic in nature so funding can be targeted at the most important
>> foundation pieces. Further funding can build on that base. Before we get to
>> Gitlab and CI we need to rebase the servers and software on them. This part of
>> the effort is funded and under way. Amar is updating the tickets with the
>> progress.
> 
> Maybe my question hasn't been clear enough. Of course part of it depends on what
> is implemented. But every selected system also adds limitations. At the moment
> BuildBot is the suggested system in the tickets. It is a well known service and
> has a lot of usefull features for us that certainly make it a good choice.
> 
> But I never really worked with it, so I basically wanted to know what I have to
> expect. Some systems like (I think it was) jenkins.io can be configured to
> behave quite different depending on how you use it. You can either configure it
> via a static configuration that is put somewhere where only admins can access
> it. But you can also configure it in a way that build jobs are defined by yaml
> files in the repository similar to popular services like GitHub, GitLab or Travis.
> 
> Basically what I wanted to know is: What is possible / usual with BuildBot and
> which of the possible solutions do we want to use? 

A developer with a gitlab account will be encouraged to move their personal repo
to their gitlab account and doing that allows them to use the gitlab CI tools.
Gitlab will be running on FreeBSD and in a jail so any dependent services a
developer needs will need to be requested via a ticket. I hope you appreciate
there are limit here related to resources and the hardened nature of the
environment.

> Do we configure BuildBot
> statically and every change will be done only if someone pays for it?

The main CI build flows will be controlled and changed via a ticket. We need to
QA control that process to ensure the patches merged are suitable. I have no
idea yet how this will look and work but it will need to allow the project to
make updates. Wether this is widely available on the first pass or pass ten of
the work that needs to be done I cannot say.

> Will there
> be a repo with config files where everyone can suggest and help but only one or
> two admins can accept changes if they have time? Or will every repo contain a
> config yaml (or similar format) and every maintainer can accept a change to
> that? Or is the currently discussed solution something completely different?

Your personal repo will have the standard gitlab features. I am not sure what
they are. A merge request to a project repo that changes the gitlab CI config
will need to be approved. It should be treated like any other part of the
process. I also do not know how the project CI configs will operate in a
personal repo. That is another issue to work out.

> That shouldn't be a pure decision by the one who pays for the work but one that
> is (in the optimal case) discussed and specified in the tickets.

I did not know that was happening. I am sorry if you think it is and if I gave
you that impression.

> That way
> everyone in the community has at least a chance to have some influence on the
> system that he will have to use later.

That is fair. Every attempt is being made to involve the community. The tickets
are detailed and open. Anyone can fund the work. I will point out the funding is
not via a foundation and so it can be directed. It is not ideal but this is the
best solution I can find so far.

Chris