[GSoC 2020]: Need help in writing sed alternative in Python for RSB recipes

[Gedare Bloom]

Hi Mritunjay, Chris:

For the RSB, since it is user-facing, we need to be sure to minimize
its dependencies on the user environment and platform. So this pycli
can only be used if it is distributed standard with Python 2 and
Python 3.

Would someone be able to apply this patch and use it to build epics?
No, because (1) they won't have pycli available, and (2) they don't
have a path /home/mritunjay

If there is no standard way to do line editing out of the box in
Python, we might need to write our own little helper function to do
it. I think Chris should weigh in on this issue, and whether there is
anything to consider mimicking that does some custom Python scripting
in RSB.

Gedare

On Sat, Aug 15, 2020 at 8:18 AM Mritunjay Sharma
<mritunjaysharma394 at gmail.com> wrote:
>
>
> On Sat, Aug 15, 2020 at 7:42 PM Mritunjay Sharma <mritunjaysharma394 at gmail.com> wrote:
>>
>> [Update]: I have built a workable python cli package known as 'pycli'
>> which works as an alternative for 'sed'.
>>
>> I made the following changes in the config file:
>>
>> diff --git a/source-builder/config/epics-7-1.cfg b/source-builder/config/epics-7-1.cfg
>> index f51c658..6408534 100644
>> --- a/source-builder/config/epics-7-1.cfg
>> +++ b/source-builder/config/epics-7-1.cfg
>> @@ -34,12 +34,11 @@ URL:  https://epics.mpg.de/
>>  #
>>  # Changing the RTEMS Version in epics-base/configure/os/CONFIG_SITE.Common.RTEMS
>>  #
>> -sed -i 's/RTEMS_VERSION = .*/RTEMS_VERSION = 5/g' configure/os/CONFIG_SITE.Common.RTEMS
>> +pycli 'RTEMS_VERSION = .*' 'RTEMS_VERSION = 5' configure/os/CONFIG_SITE.Common.RTEMS
>>
>>  #
>>  # Changing the RTEMS Base in epics-base/configure/os/CONFIG_SITE.Common.RTEMS
>> -#
>> -sed -i "s/^RTEMS_BASE .*/RTEMS_BASE = \/home\/mritunjay\/development\/rtems\/\$\(RTEMS_VERSION\)\-arm/g" configure/os/CONFIG_SITE.Common.RTEMS
>> +pycli '^RTEMS_BASE .*' 'RTEMS_BASE = /home/mritunjay/development/rtems/$(RTEMS_VERSION)-arm/' configure/os/CONFIG_SITE.Common.RTEMS
>>
>>    cd ${build_top}
>>
>
>
> Kindly ignore this spike part, It was mistakenly sent.
>
>>
>> diff --git a/source-builder/config/spike-1-1.cfg b/source-builder/config/spike-1-1.cfg
>> index e731348..aff3f35 100644
>> --- a/source-builder/config/spike-1-1.cfg
>> +++ b/source-builder/config/spike-1-1.cfg
>> @@ -3,7 +3,7 @@
>>  #
>>  # This configuration file configure's, make's and install's RISC-V's spike simulator.
>>  #
>> -
>> +%define spike_version 01252686902fa30665fbecfc1476d169ad1333d1
>>  %if %{release} == %{nil}
>>  %define release 1
>>  %endif
>>
>> This worked successfully and the build ran fine. I am going to upload the entire python pycli project on the github in a couple of hours.
>>
>> The main part of the python script that did the work is here:
>>
>> '''import re
>>
>> def replace(oldstring, newstring, infile, dryrun=False):
>>     '''
>>     Sed-like Replace
>>     Usage: pycli <Old string>  <Replacement String> <Text File>
>>     Example: pycli 'xyz' 'XYZ' '/path/to/file.txt'
>>     '''
>>     linelist = []
>>     with open(infile) as f:
>>         for item in f:
>>             newitem = re.sub(oldstring, newstring, item)
>>             linelist.append(newitem)
>>     if dryrun == False:
>>         with open(infile, "w") as f:
>>             f.truncate()
>>             for line in linelist: f.writelines(line)
>>     elif dryrun == True:
>>         for line in linelist: print(line, end='')
>>     else:
>>         exit("Unknown option specified to 'dryrun' argument, Usage: dryrun=<True|False>.")'''
>>
>> Please do give the feedback if this has been done in the right way?
>>
>> Thanks
>> Mritunjay Sharma
>>
>>
>>
>> On Fri, Aug 14, 2020 at 3:13 AM Mritunjay Sharma <mritunjaysharma394 at gmail.com> wrote:
>>>
>>> Hello everyone,
>>>
>>> Based on the input received by Gedare, I have started working
>>> on finding a 'sed' alternative to be used in the RSB recipes to
>>> do streamline text replacements.
>>>
>>> I have gone through the Python Development Guidelines (https://docs.rtems.org/branches/master/eng/python-devel.html)
>>> and have installed the things suggested in this for the development
>>> environment.
>>>
>>> To start with the research work, I found this interesting
>>> the discussion here https://unix.stackexchange.com/questions/13711/differences-between-sed-on-mac-osx-and-other-standard-sed that gave me an idea
>>> on how sed differs in BSD and UNIX platforms.
>>>
>>> I learnt about differences like:
>>> `OS X's sed uses -E for ERE and GNU sed uses -r. -E is an alias for -r in GNU sed (added in 4.2, not documented until 4.3). Newer versions of FreeBSD and NetBSD sed support both -E and -r. OpenBSD sed only supports -E.
>>> `
>>>
>>> I also found a similar project https://github.com/chmln/sd which can help us in
>>> developing the Python code needed in our case.
>>>
>>> I would request the mentors to guide me on how and where to begin from
>>> in writing this alternative?
>>> Where do I have to write it? Is it the rtems-tools project (https://github.com/RTEMS/rtems-tools) where I have to contribute to for building this alternative?
>>>
>>> Thanks
>>> Mritunjay Sharma