Fwd: Identify 3rd party source in spec?

[Gedare Bloom]

On Fri, Nov 4, 2022 at 9:55 AM Gedare Bloom <gedare at rtems.org> wrote:
>
> On Fri, Nov 4, 2022 at 8:58 AM Sebastian Huber
> <sebastian.huber at embedded-brains.de> wrote:
> >
> > On 04/11/2022 15:38, Gedare Bloom wrote:
> > > On Fri, Nov 4, 2022 at 2:59 AM Sebastian Huber
> > > <sebastian.huber at embedded-brains.de>  wrote:
> > >> On 31/10/2022 20:01, Gedare Bloom wrote:
> > >>> I would like to float the idea of managing 3rd party source tracking
> > >>> through the build system spec files. I believe this would be the most
> > >>> efficient way to maintain this information, and we can leverage the
> > >>> existing build system code for tasks such as automatic format checks,
> > >>> generating lists of third-party code, etc.
> > >>>
> > >>> This will require refactoring some spec files to pull 3rd party code
> > >>> out to a separate .yml file that gets linked. Once that is done, then
> > >>> we could add another attribute for this tracking purpose. I would like
> > >>> to keep it simple as a boolean, maybe just "third-party: true/false"
> > >>> Attached is an example patch showing how this might work for the
> > >>> dtc/libfdt code as a build objects item type 'obj', and for zlib
> > >>> library as a build library item type 'lib' with some proof-of-concept
> > >>> code for generating a listing of third party source files.
> > >> In the future we could expand this third-party attribute to a dictionary
> > >> to provide more detailed information about the third-party sources, for
> > >> example the upstream Git repository, the import commit, path translation
> > >> mappings, post and pre import actions. This could be used to
> > >> automatically generate updates from upstream changes.
> > >>
> > > Nice. I don't see an explicit "dictionary" type for attributes, but I
> > > do see that dict-like behavior could be achieved using a 'list' of
> > > attributes nested below an attribute. I have the time allocated that i
> > > could try to learn how to do this with a simple example, for example
> > > to add a "upstream" and an "import-commit" field. Then, the
> > > third-party indicator will change from
> > > third-party: false   ->
> > >      third-party: []
> > > third-party: true    ->
> > >       third-party:
> > >           - upstream: URL
> > >           - import-commit: git-sha
> > > I would guess. Would this kind of structure "just work" in the current
> > > support of spec/build or will it require some integration with
> > > rtems-central?  Or, am I following the wrong ideas,
> >
> > third-party: null
> >
> > would indicate that this object/library doesn't have an upstream project.
> >
> > If there is an upstream project, then we could describe this through
> > additional attributes, for example:
> >
> > third-party:
> >    type: git
> >    commit: abc
> >    repository: def
> >    file-map:
> >    - replace regex 1
> >    - replace regex 2
> >
>
> OK, that makes sense. I think I can make a first pass just to put in
> the framework for extending this later. Taking DTC as an example,
> maybe something like the following makes sense?
>
> third-party:
>   - type: git
>   - repository: git://git.kernel.org/pub/scm/utils/dtc/dtc.git
>   - commits:
>     - upstream: c0c2e115f82ed3bc5f9d3f9e5380f0f7e81a1c21
>     - import: 175263ec91f00f6668ae7f28f385850a1c77bf6b
>   - update: cherry-pick
>
>
> Where we will have to define some update methods such as applying
> patches from the upstream (cherry-pick).
>
> As POC I can process this with wscript and obtain
> [{'type': 'git'}, {'repository':
> 'git://git.kernel.org/pub/scm/utils/dtc/dtc.git'}, {'commits':
> [{'upstream': 'c0c2e115f82ed3bc5f9d3f9e5380f0f7e81a1c21'}, {'import':
> '175263ec91f00f6668ae7f28f385850a1c77bf6b'}]}, {'update':
> 'cherry-pick'}, {'source': ['cpukit/dtc/libfdt/fdt.c',
> 'cpukit/dtc/libfdt/fdt_addresses.c',
> 'cpukit/dtc/libfdt/fdt_empty_tree.c', 'cpukit/dtc/libfdt/fdt_ro.c',
> 'cpukit/dtc/libfdt/fdt_rw.c', 'cpukit/dtc/libfdt/fdt_strerror.c',
> 'cpukit/dtc/libfdt/fdt_sw.c', 'cpukit/dtc/libfdt/fdt_wip.c']}]
>
> If this looks reasonable, I can try to add these fields as part of my
> sweep through the cpukit tagging third-party sources.
>

Given the complexity of this tagging, I'm going to start with just the
true/false approach to categorize third-party sources. We can do
something like the above in the future.