Fwd: Identify 3rd party source in spec?

[Gedare Bloom]

On Fri, Nov 4, 2022 at 8:58 AM Sebastian Huber
<sebastian.huber at embedded-brains.de> wrote:
>
> On 04/11/2022 15:38, Gedare Bloom wrote:
> > On Fri, Nov 4, 2022 at 2:59 AM Sebastian Huber
> > <sebastian.huber at embedded-brains.de>  wrote:
> >> On 31/10/2022 20:01, Gedare Bloom wrote:
> >>> I would like to float the idea of managing 3rd party source tracking
> >>> through the build system spec files. I believe this would be the most
> >>> efficient way to maintain this information, and we can leverage the
> >>> existing build system code for tasks such as automatic format checks,
> >>> generating lists of third-party code, etc.
> >>>
> >>> This will require refactoring some spec files to pull 3rd party code
> >>> out to a separate .yml file that gets linked. Once that is done, then
> >>> we could add another attribute for this tracking purpose. I would like
> >>> to keep it simple as a boolean, maybe just "third-party: true/false"
> >>> Attached is an example patch showing how this might work for the
> >>> dtc/libfdt code as a build objects item type 'obj', and for zlib
> >>> library as a build library item type 'lib' with some proof-of-concept
> >>> code for generating a listing of third party source files.
> >> In the future we could expand this third-party attribute to a dictionary
> >> to provide more detailed information about the third-party sources, for
> >> example the upstream Git repository, the import commit, path translation
> >> mappings, post and pre import actions. This could be used to
> >> automatically generate updates from upstream changes.
> >>
> > Nice. I don't see an explicit "dictionary" type for attributes, but I
> > do see that dict-like behavior could be achieved using a 'list' of
> > attributes nested below an attribute. I have the time allocated that i
> > could try to learn how to do this with a simple example, for example
> > to add a "upstream" and an "import-commit" field. Then, the
> > third-party indicator will change from
> > third-party: false   ->
> >      third-party: []
> > third-party: true    ->
> >       third-party:
> >           - upstream: URL
> >           - import-commit: git-sha
> > I would guess. Would this kind of structure "just work" in the current
> > support of spec/build or will it require some integration with
> > rtems-central?  Or, am I following the wrong ideas,
>
> third-party: null
>
> would indicate that this object/library doesn't have an upstream project.
>
> If there is an upstream project, then we could describe this through
> additional attributes, for example:
>
> third-party:
>    type: git
>    commit: abc
>    repository: def
>    file-map:
>    - replace regex 1
>    - replace regex 2
>

OK, that makes sense. I think I can make a first pass just to put in
the framework for extending this later. Taking DTC as an example,
maybe something like the following makes sense?

third-party:
  - type: git
  - repository: git://git.kernel.org/pub/scm/utils/dtc/dtc.git
  - commits:
    - upstream: c0c2e115f82ed3bc5f9d3f9e5380f0f7e81a1c21
    - import: 175263ec91f00f6668ae7f28f385850a1c77bf6b
  - update: cherry-pick


Where we will have to define some update methods such as applying
patches from the upstream (cherry-pick).

As POC I can process this with wscript and obtain
[{'type': 'git'}, {'repository':
'git://git.kernel.org/pub/scm/utils/dtc/dtc.git'}, {'commits':
[{'upstream': 'c0c2e115f82ed3bc5f9d3f9e5380f0f7e81a1c21'}, {'import':
'175263ec91f00f6668ae7f28f385850a1c77bf6b'}]}, {'update':
'cherry-pick'}, {'source': ['cpukit/dtc/libfdt/fdt.c',
'cpukit/dtc/libfdt/fdt_addresses.c',
'cpukit/dtc/libfdt/fdt_empty_tree.c', 'cpukit/dtc/libfdt/fdt_ro.c',
'cpukit/dtc/libfdt/fdt_rw.c', 'cpukit/dtc/libfdt/fdt_strerror.c',
'cpukit/dtc/libfdt/fdt_sw.c', 'cpukit/dtc/libfdt/fdt_wip.c']}]

If this looks reasonable, I can try to add these fields as part of my
sweep through the cpukit tagging third-party sources.

> --
> embedded brains GmbH
> Herr Sebastian HUBER
> Dornierstr. 4
> 82178 Puchheim
> Germany
> email: sebastian.huber at embedded-brains.de
> phone: +49-89-18 94 741 - 16
> fax:   +49-89-18 94 741 - 08
>
> Registergericht: Amtsgericht München
> Registernummer: HRB 157899
> Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
> Unsere Datenschutzerklärung finden Sie hier:
> https://embedded-brains.de/datenschutzerklaerung/