[PATCH] gdb: prefere python3 if it is installed

[Chris Johns]

On 20/8/21 5:42 pm, Christian MAUDERER wrote:
> Am 20.08.21 um 09:02 schrieb Chris Johns:
>> On 20/8/21 4:48 pm, Christian MAUDERER wrote:
>>> Hello,
>>>
>>> Am 20.08.21 um 03:49 schrieb Chris Johns:
>>>> On 20/8/21 3:16 am, Joel Sherrill wrote:
>>>>> On Thu, Aug 19, 2021 at 11:51 AM Gedare Bloom <gedare at rtems.org> wrote:
>>>>>>
>>>>>> I have no problem with this. I think it is sensible to look for
>>>>>> python3 before python2. At some point we'll have to stop looking for
>>>>>> python2 :)
>>>>>
>>>>> That is further in the future than I would have thought based on the
>>>>> CentOS project changes. I still see user organizations with no plans
>>>>> to move off CentOS 7. It will receive maintenance updates through
>>>>> 2024-06-30.
>>>>>
>>>>> But on CentOS 7, I can load a Python 3.6, newer GCC, etc. so it isn't
>>>>> that bad. I had to test something on a true 32-bit distribution this week
>>>>> and even CentOS 7 (i386) wasn't as painful as I expected to setup.
>>>>
>>>> There will come a time when a change made cannot be easily tested by us on
>>>> python2 and that will in effect end our support. We are not there yet.
>>>
>>> STOP.
>>
>> Please do not do this again.
>>
>> That discussion took a wrong direction. We discussed deprecating python2 a
>>> few times and I know that we will not do it before the big long living
>>> distributions drop it. I can live with that and I don't want to re-start this
>>> discussion with this patch.
>>
>> If Joel feels the need to make this statement he can and he has more than earned
>> the right to do so. His work and those he supports is important.
> 
> Sorry, I didn't want to offend you, Joel or Gedare.
> 
> I had that with discussions before that the original idea (here: giving priority
> to python3 over python2 when building gdb) started to be buried by a similar but
> slightly different one which has been already discussed multiple times (here:
> deprecating python2). I wanted to avoid that.
> 
> I should have worded that different. I'm sorry if I have offended you, Joel or
> Gedare or anyone else.

Thanks and none taken. I am clear on the focus of your posts so it is ok. :)

> 
>>
>>>
>>>>
>>>>>> On Thu, Aug 19, 2021 at 3:24 AM Christian MAUDERER
>>>>>> <christian.mauderer at embedded-brains.de> wrote:
>>>>>>>
>>>>>>> PS: I had the problem on the 5 branch of RTEMS source builder. I think
>>>>>>> we should apply a patch to both: master and the 5 branch.
>>>>>>>
>>>>>>> Am 19.08.21 um 10:34 schrieb Christian Mauderer:
>>>>>>>> More and more systems stop shipping python2. So we should start to
>>>>>>>> prefer python3 over python2. For building gdb it is not only necessary
>>>>>>>> to have a python binary installed, but also the matching python-devel
>>>>>>>> packet. On a lot of hosts that will now be more often python3-devel
>>>>>>>> and not python2-devel.
>>>>>>>> ---
>>>>>>>>
>>>>>>>> Note: Please see that patch more as an suggestion / base for
>>>>>>>> discussion. I'm open to better solutions.
>>>>>>>>
>>>>>>>> My problem that triggered this patch was a build of a toolchain on a
>>>>>>>> github CI/CD system that has been originally set up by one of our
>>>>>>>> users (see [1] for the log). It seems that on the "macos-latest"
>>>>>>>> machines a python2 is installed but no python2 headers are found.
>>>>
>>>> I have just built the latest RSB master GDB on a fully updated MacOS (Big Sur):
>>>>
>>>> config: tools/rtems-gdb-10.cfg
>>>> package: arm-rtems6-gdb-7ab567f-x86_64-apple-darwin20.1.0-1
>>>> building: arm-rtems6-gdb-7ab567f-x86_64-apple-darwin20.1.0-1
>>>> sizes: arm-rtems6-gdb-7ab567f-x86_64-apple-darwin20.1.0-1: 629.086MB
>>>> (installed:
>>>> 19.586MB)
>>>> cleaning: arm-rtems6-gdb-7ab567f-x86_64-apple-darwin20.1.0-1
>>>> reporting: tools/rtems-gdb-10.cfg ->
>>>> arm-rtems6-gdb-7ab567f-x86_64-apple-darwin20.1.0-1.txt
>>>> reporting: tools/rtems-gdb-10.cfg ->
>>>> arm-rtems6-gdb-7ab567f-x86_64-apple-darwin20.1.0-1.xml
>>>>
>>>> I support the latest MacOS with Xcode and have a dedicated Mac machine that
>>>> tracks the current RSB. It had wedged itself and I have cleared that so it is
>>>> now building the latest.
>>>
>>> So do I understand that correct: This is a "clean" Mac with no additional python
>>> installs or similar and it worked? What python versions are available on this
>>> this machine?
>>
>> It is working for me and I have not looked into it any more.
>>
>>> If I did understand you correctly, it seems that github has made some odd
>>> choices about their default config for the macos-latest machines. They seem to
>>> have python2 but no devel headers.
>>
>> You will need to raise that with them.
> 
> Yes, of course. I didn't want to raise it as a problem here. It was more a
> statement than a question or a complain.

Yeap, I understand.

> 
>>
>>>>>>>> Homebrew - which could be used earlier on MacOS to install the
>>>>>>>> necessary headers - dropped the python at 2 packet.
>>>>
>>>> Homebrew and MacPorts are a personal choice and I am fine with users heading
>>>> down this path however it is beyond this project's scope to support those
>>>> frameworks. I have posted the reasons in the past and the MacPorts maintainers
>>>> are aware of those reasons.
>>>
>>> No problem. I'm not a Mac user so I don't really have an opinion or experience
>>> what is the right way to install the requirements on a MacOS system. I'm really
>>> happy that it just works most of the time (a lot of it most likely thanks to
>>> you).
>>
>> Actually, all I do is politely raise issues with those that do the real work. I
>> think Homebrew and Macports are awesome projects but they bring in packages and
>> with that issues and when our users have a mix of package versions the level of
>> complexity exploded. I have no capacity to cover this.
>>
>>>>>>> So it seems that on a
>>>>>>>> modern MacOS there is no possibility to get python2 headers.
>>>>
>>>> As I have just built GDB this does not appear to be true. I would prefer we do
>>>> not overreach until we all agree there is a problem. If there is an issue I
>>>> would raise the problem in Apple'd developer bug system. I have raised a number
>>>> of issues over the years and to Apple's credit they have dealt with them all.
>>>
>>> Maybe I haven't made it enough clear: I had to do some guess work here. I have
>>> put it here as a basis for discussion. I'm completely OK if you can tell me that
>>> my conclusion has been wrong.
>>
>> I think the switch is a good idea. It may break something things but that is
>> also OK because we can then fix them well before a release.
>>
>>>
>>> It seems that it really is just an odd choice of defaults on github then.
>>>
>>
>> Sorry, again I am not sure.
> 
> Found the documentation on github. If someone is interested:
> 
> https://github.com/actions/virtual-environments/blob/main/images/macos/macos-10.15-Readme.md
> 
> 
> https://github.com/actions/virtual-environments/blob/main/images/macos/macos-11-Readme.md
> 
> 
> So all github macOS runners have some Python 2.7 installed.

Big Sur still ships Python2.

FYI the MacOS build reports have started to appear on the build list.

> 
>>
>>>>
>>>>>>>> If
>>>>>>>> python2 is still installed on the machine (for whatever reason), it is
>>>>>>>> not possible to successfully use RTEMS source builder to build a gdb.
>>>>>>>> If python3 is preferred instead, that should solve the problem.
>>>>
>>>> The change seems sensible.
>>>
>>> I think even if a clean MacOS doesn't have a problem (like I assumed), I still
>>> think it would be a good idea to prefer python3 if it is available. We are not
>>> talking about preferring it in our python-scripts but only to build gdb.
>>
>> I agree, lets switch.
>>
> 
> Thanks.
> 
>>>
>>> Do I have a "go" to:
>>>
>>> - create tickets for 5 and 6
>>>    ("RSB: Prefer python3 to build gdb if it is there" with some explanation in
>>> the description)

The 6 patch is OK to push when you are ready.

>>> - push that patch to 5 and 6 and close the tickets with it
>>>
>>
>> The 5 needs to be check on Windows before we switch. I think it will be fine but
>> it is a release hurdle if it does not as it makes a dot release a major piece of
>> work.
>>
>> Do you need a hand with Windows? If so please ask.
> 
> What would you see as the more difficult Windows build environment that should
> be tested? We have MSYS2 and Cygwin in our documentation. Do you test both on
> releases?

I only test MSYS2 and on real hardware with Windows 10. Cygwin is important to
Joel and DEOS and so I leave this task to him.

> 
>>
>>>>>>>> Note that at the moment I only tried it on my OpenSUSE-Linux machine.
>>>>>>>> For that I made sure that I have python2 and python3 installed but no
>>>>>>>> python-devel (which is python2 on OpenSUSE). Earlier I know that I
>>>>>>>> needed python-devel to build gdb. With this patch, I can build with
>>>>>>>> only python3-devel.
>>>>>>>>
>>>>>>>> I'll try to add that patch to the CI too to see whether it works on
>>>>>>>> MacOS. But I'm a bit unsure what other problems this patch could
>>>>>>>> trigger and therefore I want to start a discussion early.
>>>>>>>>
>>>>>>>> Best regards
>>>>>>>>
>>>>>>>> Christian
>>>>>>>>
>>>>>>>> [1] https://github.com/grisp/grisp2-rtems-toolchain/runs/3362717606
>>>>
>>>> I am looking forward to their hardware being shipped.
>>>
>>> I think there will be an official update soon. You most likely noted that the
>>> CI-run belongs to a pull request that updates the toolchain to work with the
>>> first final board that had been on my desk this week.
>>
>> Great and no I had not looked. Sorry, deep in libbsd (again and eval_path grr).
> 
> Good luck with the libbsd problems.

Thanks.

Chris