[PATCH rtems-libbsd.nfs 1/4] Implement portable kernel symbol namespace tool

Gedare Bloom gedare at rtems.org
Tue Jul 27 21:16:59 UTC 2021


On Tue, Jul 27, 2021 at 2:09 AM <chrisj at rtems.org> wrote:
>
> From: Chris Johns <chrisj at rtems.org>
>
> - The script's use of sort proved to not be portable
>
> - No need to check the commits as symbols are only added
>
> - Regenerated kernel header to reset the sort order
>
> Update #4475
> ---
>  CONTRIBUTING.md                               |    72 +-
>  create-kernel-namespace.sh                    |   133 -
>  rtems-kern-symbols                            |   502 +
>  .../machine/rtems-bsd-kernel-namespace.h      | 11418 ++++++++--------
>  4 files changed, 6283 insertions(+), 5842 deletions(-)
>  delete mode 100755 create-kernel-namespace.sh
>  create mode 100755 rtems-kern-symbols
>
> diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
> index 20dc938b..5dae62f8 100644
> --- a/CONTRIBUTING.md
> +++ b/CONTRIBUTING.md
> @@ -121,6 +121,76 @@ $ git log freebsd-org
>  ```
>  to figure out the current FreeBSD baseline.
>
> +Updates to FreeBSD or RTEMS Kernel Support
> +------------------------------------------
> +
> +If you update code or change any defines that effect code in the following paths:
> +
> +* `freebsd/sys/*.[ch]`
> +* `rtemsbsd/rtems/rtems-kernel-*.c`
> +
> +you need to see if any new kernel symbols have been generated for
s/for/or

> +exposed. The tool `rtems-kern-symbols` command support this operation.
s/support/supports
> +
> +The public (global) kernel symbosl need to reside in a private
s/symbosl/symbols

> +namespace to avoid clashing with symbols in the user space code or
> +applications. The FreeBSD kernel names functions and variables
> +assuming a private kernel only symbols space. RTEMS builds FreeBSD
> +kernel and user space code in the same symbols sapce so there can be
s/sapce/space

> +clashes. We manage this by maintained a header file that maps the
s/maintained/maintaining

> +global kernel symbols to a private namespace. For example `malloc` is
> +mapped to `_bsd_malloc`.
> +
> +The set of symbols to map is not easy to obtain because symbols may be
> +the result of complex preprocessing of the source, the code is
> +specific to a BSP or the code is controlled by a buildset.
> +
> +The approach is not remove symbols unless you are certain the symbols
s/is/is to  ?

> +has been removed from the source. This is a manual operation.
s/has/have
s/source/libbsd source   ?

> +
> +You are required to check symbols with a 32bit and 64bit
> +architecture.
> +
> +If you are working a specific BSP and related drivers please make sure
s/working/working on

> +the kernel symbols are checked. It is too much to ask every developer
> +to build all BSPs and check.
> +
> +RTEMS Kernel Symbols Tool
> +~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +The python tool `rtems-kern-symbols` can read a kernel header load the existing of headers.

"to load" or "load of"   ?

> +
> +The kernel namespace header can be regenerated from the original
> +header. This checks the kernel header is already sorted. If you think
> +there is a sorting issue in the existing header please regenerate
> +without adding new symbols.
> +
> +```
> +$ ./rtems-kern-symbols --regenerate --output=tmp.h
> +```
> +
> +The tools needs access to your built RTEMS tools. You can set your
"tool needs"

> +environment `PATH` variable or you can specify the top level path as an argument:
> +```
> +$ ./rtems-kern-symbols --tools=/opt/work/rtems/6
> +```

This is inconsistent with other ecosystem support, use --rtems-tools instead?

> +
> +Options:
> +
> +* You can provide a different kernel header using the `--kern-header`
> +argument. The default is the LibbSD header.
> +
> +* The `--report` option provides a report.
> +
> +* The `--diff` option pvoides a unified diff of any changes.
provides

> +
> +* The `--write` option is need to write any changes
needed

> +
> +* The `--output` option lets you write to different file
different from what?

> +
> +The tool manages a number of exlcuded symbols. These are symbols in
excluded

> +the kernel space that are not mapped to the RTEMS kernal namespace.
kernel

> +
>  How to Import Code from FreeBSD
>  -------------------------------
>
> @@ -134,7 +204,7 @@ How to Import Code from FreeBSD
>  * Immediately check in the imported files without the changes to `libbsd.py` and the buildsets.  Do not touch the imported files yourself at this point.
>  * Port the imported files to RTEMS.  See 'Rules for Modifying FreeBSD Source'.
>  * Add a test to the testsuite if possible.
> -* Run `./create-kernel-namespace.sh` if you imported kernel space headers.  Add only your new defines via `git add -p rtemsbsd/include/machine/rtems-bsd-kernel-namespace.h`.
> +* Run `./rtems-kern-symbols` as discussed above
>  * Create one commit from this.
>
>  The -S or --stats option generates reports the changes we have made to
> diff --git a/create-kernel-namespace.sh b/create-kernel-namespace.sh
> deleted file mode 100755
> index ed4efa10..00000000
> --- a/create-kernel-namespace.sh
> +++ /dev/null
> @@ -1,133 +0,0 @@
> -#!/bin/sh -x
> -
> -#
> -# Copyright (c) 2016 embedded brains GmbH.  All rights reserved.
> -#
> -#  embedded brains GmbH
> -#  Dornierstr. 4
> -#  82178 Puchheim
> -#  Germany
> -#  <rtems at embedded-brains.de>
> -#
> -# Redistribution and use in source and binary forms, with or without
> -# modification, are permitted provided that the following conditions
> -# are met:
> -# 1. Redistributions of source code must retain the above copyright
> -#    notice, this list of conditions and the following disclaimer.
> -# 2. Redistributions in binary form must reproduce the above copyright
> -#    notice, this list of conditions and the following disclaimer in the
> -#    documentation and/or other materials provided with the distribution.
> -#
> -# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
> -# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
> -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
> -# ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
> -# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
> -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
> -# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
> -# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
> -# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
> -# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
> -# SUCH DAMAGE.
> -#
> -
> -#
> -# This script generates the kernel namespace header file
> -# <machine/rtems-bsd-kernel-namespace.h>.
> -#
> -
> -objdump --syms `for i in build/*rtems* ; do \
> -       find $i/freebsd/sys/ -name '*.o' ; \
> -       echo $i/rtemsbsd/rtems/rtems-kernel-*.o ; \
> -       done` \
> -       | awk '/^[0-9a-f]+[[:blank:]]+g/ {print $6}' \
> -       | sed 's/^_bsd_//' \
> -       | sed '/^accept$/d' \
> -       | sed '/^arc4random$/d' \
> -       | sed '/^bind$/d' \
> -       | sed '/^blackhole$/d' \
> -       | sed '/^bpf_filter$/d' \
> -       | sed '/^bpf_jitter$/d' \
> -       | sed '/^bpf_jitter_enable$/d' \
> -       | sed '/^bpf_validate$/d' \
> -       | sed '/^connect$/d' \
> -       | sed '/^drop_redirect$/d' \
> -       | sed '/^drop_synfin$/d' \
> -       | sed '/^free$/d' \
> -       | sed '/^getentropy$/d' \
> -       | sed '/^getpeername$/d' \
> -       | sed '/^getsockname$/d' \
> -       | sed '/^getsockopt$/d' \
> -       | sed '/^ifqmaxlen$/d' \
> -       | sed '/^in6addr_any$/d' \
> -       | sed '/^in6addr_loopback$/d' \
> -       | sed '/^in6addr_nodelocal_allnodes$/d' \
> -       | sed '/^in6addr_linklocal_allnodes$/d' \
> -       | sed '/^kevent$/d' \
> -       | sed '/^kqueue$/d' \
> -       | sed '/^listen$/d' \
> -       | sed '/^malloc$/d' \
> -       | sed '/^max_datalen$/d' \
> -       | sed '/^max_hdr$/d' \
> -       | sed '/^max_linkhdr$/d' \
> -       | sed '/^max_protohdr$/d' \
> -       | sed '/^maxsockets$/d' \
> -       | sed '/^nd6_debug$/d' \
> -       | sed '/^nd6_delay$/d' \
> -       | sed '/^nd6_gctimer$/d' \
> -       | sed '/^nd6_maxnudhint$/d' \
> -       | sed '/^nd6_mmaxtries$/d' \
> -       | sed '/^nd6_onlink_ns_rfc4861$/d' \
> -       | sed '/^nd6_prune$/d' \
> -       | sed '/^nd6_umaxtries$/d' \
> -       | sed '/^nd6_useloopback$/d' \
> -       | sed '/^nmbclusters$/d' \
> -       | sed '/^nmbjumbo16$/d' \
> -       | sed '/^nmbjumbo9$/d' \
> -       | sed '/^nmbjumbop$/d' \
> -       | sed '/^nmbufs$/d' \
> -       | sed '/^nolocaltimewait$/d' \
> -       | sed '/^path_mtu_discovery$/d' \
> -       | sed '/^pause$/d' \
> -       | sed '/^pf_osfp_entry_pl$/d' \
> -       | sed '/^pf_osfp_pl$/d' \
> -       | sed '/^pipe$/d' \
> -       | sed '/^poll$/d' \
> -       | sed '/^random$/d' \
> -       | sed '/^realloc$/d' \
> -       | sed '/^reallocf$/d' \
> -       | sed '/^recvfrom$/d' \
> -       | sed '/^recvmsg$/d' \
> -       | sed '/^rtems/d' \
> -       | sed '/^select$/d' \
> -       | sed '/^sendmsg$/d' \
> -       | sed '/^sendto$/d' \
> -       | sed '/^setfib$/d' \
> -       | sed '/^setsockopt$/d' \
> -       | sed '/^shutdown$/d' \
> -       | sed '/^socket$/d' \
> -       | sed '/^socketpair$/d' \
> -       | sed '/^soreceive_stream$/d' \
> -       | sed '/^srandom$/d' \
> -       | sed '/^strdup$/d' \
> -       | sed '/^sysctlbyname$/d' \
> -       | sed '/^sysctl$/d' \
> -       | sed '/^sysctlnametomib$/d' \
> -       | sed '/sys_init/d' \
> -       | sed '/^taskqueue_/d' \
> -       | sed '/^tcp_offload_listen_start$/d' \
> -       | sed '/^tcp_offload_listen_stop$/d' \
> -       | sed '/^ticks$/d' \
> -       | sed '/^useloopback$/d' \
> -       | sed '/^_Watchdog_Ticks_since_boot$/d' \
> -       | sort -u > symbols.txt
> -
> -cat << eof > rtems-bsd-kernel-namespace.h
> -#ifndef _RTEMS_BSD_MACHINE_RTEMS_BSD_KERNEL_SPACE_H_
> -#error "the header file <machine/rtems-bsd-kernel-space.h> must be included first"
> -#endif
> -
> -eof
> -sed 's/^\(.*\)/#define \1 _bsd_\1/' < symbols.txt >> rtems-bsd-kernel-namespace.h
> -rm symbols.txt
> -mv rtems-bsd-kernel-namespace.h rtemsbsd/include/machine/rtems-bsd-kernel-namespace.h
> diff --git a/rtems-kern-symbols b/rtems-kern-symbols
> new file mode 100755
> index 00000000..e34a14a7
> --- /dev/null
> +++ b/rtems-kern-symbols
> @@ -0,0 +1,502 @@
> +#! /usr/bin/env python
> +
> +# SPDX-License-Identifier: BSD-2-Clause
> +"""RTEMS LibBBSD Kernel Symbols
> +
> +Generate the symbols for the kernel headers and merge in any new ones
> +"""
> +
> +#
> +# Copyright (C) 2021 Chris Johns <chrisj at rtems.org>, All rights reserved.
> +#
> +# Redistribution and use in source and binary forms, with or without
> +# modification, are permitted provided that the following conditions
> +# are met:
> +# 1. Redistributions of source code must retain the above copyright
> +#    notice, this list of conditions and the following disclaimer.
> +# 2. Redistributions in binary form must reproduce the above copyright
> +#    notice, this list of conditions and the following disclaimer in the
> +#    documentation and/or other materials provided with the distribution.
> +#
> +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
> +# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
> +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
> +# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
> +# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
> +# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
> +# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
> +# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
> +# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
> +# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
> +# POSSIBILITY OF SUCH DAMAGE.
> +#
> +
> +from __future__ import print_function
> +
> +import argparse
> +import os
> +import re
> +import sys
> +
> +version = "1.0"
> +
> +kern_objects = [
> +    ( 'freebsd/sys', '.*\.o' ),
> +    ( 'rtemsbsd/rtems', 'rtems-kernel-.*\.o' )
> +] # yapf: disable
> +
> +kern_excludes = [
> +    '^_bsd_' ,
> +    '^rtems_',
> +    '^accept$',
> +    '^arc4random$',
> +    '^bind$',
> +    '^blackhole$',
> +    '^bootverbose$',
> +    '^bpf_filter$',
> +    '^bpf_jitter$',
> +    '^bpf_jitter_enable$',
> +    '^bpf_validate$',
> +    '^cache_enter$',
> +    '^connect$',
> +    '^drop_redirect$',
> +    '^drop_synfin$',
> +    '^free$',
> +    '^getentropy$',
> +    '^getpeername$',
> +    '^getsockname$',
> +    '^getsockopt$',
> +    '^global_epoch$',
> +    '^global_epoch_preempt$',
> +    '^ifqmaxlen$',
> +    '^in6addr_any$',
> +    '^in6addr_linklocal_allnodes$',
> +    '^in6addr_loopback$',
> +    '^in6addr_nodelocal_allnodes$',
> +    '^in_epoch$',
> +    '^kevent$',
> +    '^kqueue$',
> +    '^listen$',
> +    '^malloc$',
> +    '^max_datalen$',
> +    '^max_hdr$',
> +    '^max_linkhdr$',
> +    '^max_protohdr$',
> +    '^maxsockets$',
> +    '^nd6_debug$',
> +    '^nd6_delay$',
> +    '^nd6_gctimer$',
> +    '^nd6_maxnudhint$',
> +    '^nd6_mmaxtries$',
> +    '^nd6_onlink_ns_rfc4861$',
> +    '^nd6_prune$',
> +    '^nd6_umaxtries$',
> +    '^nd6_useloopback$',
> +    '^net_epoch$',
> +    '^net_epoch_preempt$',
> +    '^nmbclusters$',
> +    '^nmbjumbo16$',
> +    '^nmbjumbo9$',
> +    '^nmbjumbop$',
> +    '^nmbufs$',
> +    '^nolocaltimewait$',
> +    '^path_mtu_discovery$',
> +    '^pause$',
> +    '^pf_osfp_entry_pl$',
> +    '^pf_osfp_pl$',
> +    '^pipe$',
> +    '^poll$',
> +    '^pselect$',
> +    '^random$',
> +    '^realloc$',
> +    '^reallocf$',
> +    '^recvfrom$',
> +    '^recvmsg$',
> +    '^rtems',
> +    '^select$',
> +    '^sendmsg$',
> +    '^sendto$',
> +    '^setfib$',
> +    '^setsockopt$',
> +    '^shutdown$',
> +    '^socket$',
> +    '^socketpair$',
> +    '^soreceive_stream$',
> +    '^srandom$',
> +    '^strdup$',
> +    '^sysctlbyname$',
> +    '^sysctl$',
> +    '^sysctlnametomib$',
> +    'sys_init',
> +    '^taskqueue_',
> +    '^tcp_offload_listen_start$',
> +    '^tcp_offload_listen_stop$',
> +    '^ticks$',
> +    '^useloopback$',
> +    '^_Watchdog_Ticks_since_boot$'
> +] # yapf: disable
> +
> +kern_header = 'rtemsbsd/include/machine/rtems-bsd-kernel-namespace.h'
Not that it worked before, but paths should avoid using explicitly '/' (or '\').

> +
> +
> +class exit(Exception):
> +    """Base class for exceptions."""
> +    def __init__(self, code):
> +        self.code = code
> +
> +
> +class error(Exception):
> +    """Base class for exceptions."""
> +    def set_output(self, msg):
> +        self.msg = msg
> +
> +    def __str__(self):
> +        return self.msg
> +
> +
> +class general_error(error):
> +    """Raise for a general error."""
> +    def __init__(self, what):
> +        self.set_output('error: ' + str(what))
> +
> +
> +class command:
> +    def __init__(self, cmd, cwd='.'):
> +        self.exit_code = 0
> +        self.output = None
> +        self.cmd = cmd
> +        self.cwd = cwd
> +        self.result = None
> +
> +    def run(self):
> +
> +        import subprocess
> +
> +        #
> +        # Support Python 2.6
> +        #
> +        if "check_output" not in dir(subprocess):
> +
> +            def f(*popenargs, **kwargs):
> +                if 'stdout' in kwargs:
> +                    raise ValueError(
> +                        'stdout argument not allowed, it will be overridden.')
> +                process = subprocess.Popen(stdout=subprocess.PIPE,
> +                                           *popenargs,
> +                                           **kwargs)
> +                output, unused_err = process.communicate()
> +                retcode = process.poll()
> +                if retcode:
> +                    cmd = kwargs.get("args")
> +                    if cmd is None:
> +                        cmd = popenargs[0]
> +                    raise subprocess.CalledProcessError(retcode, cmd)
> +                return output
> +
> +            subprocess.check_output = f
> +
> +        self.exit_code = 0
> +        try:
> +            if os.name == 'nt':
> +                cmd = ['sh', '-c'] + self.cmd
> +            else:
> +                cmd = self.cmd
> +                output = subprocess.check_output(cmd,
> +                                                 cwd=self.cwd).decode("utf-8")
> +                self.output = output.split(os.linesep)
> +        except subprocess.CalledProcessError as cpe:
> +            self.exit_code = cpe.returncode
> +            output = cpe.output.decode("utf-8")
> +            self.output = output.split(os.linesep)
> +        except OSError as ose:
> +            cs = ' '.join(cmd)
> +            if len(cs) > 80:
> +                cs = cs[:80] + '...'
> +            raise general_error('bootstrap failed: %s in %s: %s' % \
> +                                (cs, self.cwd, (str(ose))))
> +        except KeyboardInterrupt:
> +            pass
> +        except:
> +            raise
> +
> +
> +class kernel_symbols:
> +    def __init__(self, excludes):
> +        self.excludes = [re.compile(exc) for exc in excludes]
> +        self.bsps = {}
> +        self.header = {'source': [], 'symbols': []}
> +        self.output = {'source': [], 'symbols': []}
> +        self.analysis = {'unmapped': [], 'new': []}
> +
> +    @staticmethod
> +    def _find(base, spec):
> +        found = []
> +        filter = re.compile(spec[1])
> +        for root, dirs, files in os.walk(os.path.join(base, spec[0]),
> +                                         topdown=True):
> +            for f in files:
> +                if filter.match(f):
> +                    found += [os.path.join(root, f)]
> +        return found
> +
> +    @staticmethod
> +    def _find_bsps(build):
> +        bsps = []
> +        filter = re.compile('^.*-rtems[0-9].*-.*')
> +        with os.scandir(build) as sdir:
> +            for entry in sdir:
> +                if entry.is_dir() and filter.match(entry.name) != None:
> +                    bsps += [entry.name]
> +        return bsps
> +
> +    @staticmethod
> +    def bsp_arch(bsp):
> +        bs = bsp.split('-')
> +        return bs[0] + '-' + bs[1]
> +
> +    def _clean(self, symbols):
> +        syms = []
> +        for sym in symbols:
> +            add = True
> +            for exclude in self.excludes:
> +                if exclude.match(sym) is not None:
> +                    add = False
> +                    break
> +            if add:
> +                syms += [sym]
> +        return sorted(list(set(syms)))
> +
> +    def load_header(self, header):
> +        with open(header, 'r') as h:
> +            self.header['source'] = h.read().splitlines()
> +        filter = re.compile('^#define\s')
> +        for line in self.header['source']:
> +            if filter.match(line) != None:
> +                ls = line.split()
> +                if len(ls) == 3:
> +                    self.header['symbols'] += [ls[1]]
> +
> +    def load_symbols(self, specs, excludes, build, tools):
> +        bsps = self._find_bsps(build)
> +        for bsp in bsps:
> +            self.bsps[bsp] = {'output': [], 'objects': [], 'symbols': []}
> +            for spec in specs:
> +                self.bsps[bsp]['objects'] += self._find(
> +                    os.path.join(build, bsp), spec)
> +            arch = self.bsp_arch(bsp)
> +            if tools is not None:
> +                cmd = os.path.join(tools, 'bin', arch + '-nm')
> +            else:
> +                cmd = arch + '-nm'
> +            nm = command([cmd] + self.bsps[bsp]['objects'])
> +            nm.run()
> +            self.bsps[bsp]['output'] = nm.output
> +            object = '-'
> +            syms = []
> +            for line in nm.output:
> +                if len(line) == 0:
> +                    continue
> +                if line[-1] == ':':
> +                    object = os.path.basename(line[:-1])
> +                    continue
> +                ls = line.split()
> +                if len(ls) == 3:
> +                    ls = ls[1:]
> +                if ls[0] in ['A', 'B', 'C', 'D', 'R', 'T', 'W']:
> +                    if ls[1] in syms:
> +                        print('warning: duplicate symbol: %s:%s: %s' %
> +                              (bsp, object, ls[1]))
> +                    syms += [ls[1]]
> +            self.bsps[bsp]['symbols'] += syms
> +
> +    def generate_header(self):
> +        out = [
> +            '/*', '* RTEMS Libbsd, this file is generated. Do not edit.', '*/',
> +            '#ifndef _RTEMS_BSD_MACHINE_RTEMS_BSD_KERNEL_SPACE_H_',
> +            '#error "the header file <machine/rtems-bsd-kernel-space.h> must be included first"',
> +            '#endif', ''
> +        ]
> +        for sym in self.output['symbols']:
> +            out += ['#define %s _bsd_%s' % (sym, sym)]
> +        self.output['source'] = out
> +
> +    def write_header(self, header):
> +        with open(header, 'wb') as o:
> +            o.write(
> +                os.linesep.join(self.output['source'] + ['']).encode("utf-8"))
> +
> +    def write_sym_data(self):
> +        for bsp in self.bsps:
> +            arch = self.bsp_arch(bsp)
> +            with open('sym-data-' + arch + '.txt', 'w') as o:
> +                o.writelines(os.linesep.join(ks.bsps[bsp]['output']))
> +
> +    def merge(self, symbols):
> +        self.output['symbols'] = \
> +            self._clean(self.output['symbols'] + symbols)
> +
> +    def merge_bsp(self):
> +        for bsp in self.bsps:
> +            self.merge(self.bsps[bsp]['symbols'])
> +
> +    def analyse(self):
> +        for bsp in self.bsps:
> +            for sym in self.bsps[bsp]['symbols']:
> +                if sym in self.header['symbols']:
> +                    key = 'unmapped'
> +                else:
> +                    key = 'new'
> +                self.analysis[key] += [sym]
> +        for key in self.analysis:
> +            self.analysis[key] = self._clean(self.analysis[key])
> +
> +    def diff(self):
> +        import difflib
> +        return list(
> +            difflib.unified_diff(self.header['source'], self.output['source']))
> +
> +    def report(self):
> +        out = [
> +            'Symbols:',
> +            ' mapped   : %d' % (len(self.header['symbols'])),
> +            ' unmapped : %d' % (len(self.analysis['unmapped'])),
> +            ' new      : %d' % (len(self.analysis['new']))
> +        ]
> +        max_len = 0
> +        for bsp in self.bsps:
> +            if max_len < len(bsp):
> +                max_len = len(bsp)
> +        out += ['BSPs: %*s Unmapped Total' % (max_len - 4, ' ')]
> +        for bsp in self.bsps:
> +            unmapped = len(self._clean(self.bsps[bsp]['symbols']))
> +            total = len(self.bsps[bsp]['symbols'])
> +            out += [' %-*s: %-8d %d' % (max_len, bsp, unmapped, total)]
> +        out += ['Unmapped:'] + [' ' + sym for sym in self.analysis['unmapped']]
> +        out += ['New:'] + [' ' + sym for sym in self.analysis['new']]
> +        return out
> +
> +
> +def run(args):
> +    try:
> +        argsp = argparse.ArgumentParser(
> +            prog='rtems-kern-symbols',
> +            description="RTEMS LibBSD Kernel Symbols")
> +        argsp.add_argument('-t',
> +                           '--tools',
> +                           help='RTEMS Tools (default: %(default)s).',
> +                           type=str,
> +                           default=None)
> +        argsp.add_argument(
> +            '-w',
> +            '--write',
> +            action='store_true',
> +            help=
> +            'Write the header to the output file name (default: %(default)s).')
> +        argsp.add_argument(
> +            '-d',
> +            '--diff',
> +            action='store_true',
> +            help='Show a diff if the header has changed (default: %(default)s).'
> +        )
> +        argsp.add_argument(
> +            '-o',
> +            '--output',
> +            type=str,
> +            default=kern_header,
> +            help='output path to the write the header (default: %(default)s).')
> +        argsp.add_argument(
> +            '-b',
> +            '--build',
> +            type=str,
> +            default='build',
> +            help='path to the rtems libbsd build output (default: %(default)s).'
> +        )
> +        argsp.add_argument(
> +            '-K',
> +            '--kern-header',
> +            type=str,
> +            default=kern_header,
> +            help=
> +            'kernel header to load existing symbols from(default: %(default)s).'
> +        )
> +        argsp.add_argument(
> +            '-S',
> +            '--sym-data',
> +            action="store_true",
> +            help=
> +            'Write the BSP symbol data that is parsed (default: %(default)s).')
> +        argsp.add_argument(
> +            '-r',
> +            '--regenerate',
> +            action="store_true",
> +            help=
> +            'Regenerate the header file from the symbols in it, write option ignored (default: %(default)s).'
> +        )
> +        argsp.add_argument('-R',
> +                           '--report',
> +                           action="store_true",
> +                           help='Generate a report (default: %(default)s).')
> +        argopts = argsp.parse_args(args[1:])
> +
> +        print('RTEMS LibBSD Kernel Symbols, %s' % (version))
> +
> +        if not os.path.exists(argopts.build):
> +            raise general_error('path does not exist: %s' % (argopts.build))
> +
> +        ks = kernel_symbols(kern_excludes)
> +
> +        ks.load_header(argopts.kern_header)
> +
> +        if argopts.regenerate:
> +            ks.merge(ks.header['symbols'])
> +            print('Regenerating: symbols: %d: %s' %
> +                  (len(ks.output['symbols']), argopts.output))
> +            ks.generate_header()
> +            diff = ks.diff()
> +            if len(diff) == 0:
> +                print('info: no changes; header not updated')
> +            else:
> +                print('info: writing: %s' % (argopts.output))
> +                ks.write_header(argopts.output)
> +            raise exit(0)
> +
> +        ks.load_symbols(kern_objects, kern_excludes, argopts.build,
> +                        argopts.tools)
> +
> +        if argopts.sym_data:
> +            ks.write_sym_data()
> +
> +        ks.analyse()
> +        ks.merge(ks.header['symbols'])
> +        ks.merge_bsp()
> +        ks.generate_header()
> +
> +        diff = ks.diff()
> +        if argopts.write:
> +            if len(diff) == 0:
> +                print('info: no changes; header not updated')
> +            else:
> +                print('info: writing: %s' % (argopts.output))
> +                ks.write_header(argopts.output)
> +
> +        if argopts.report:
> +            print(os.linesep.join(ks.report()))
> +
> +        if argopts.diff:
> +            print('Diff: %d' % (len(diff)))
> +            print(os.linesep.join(diff))
> +
> +    except general_error as gerr:
> +        print(gerr)
> +        print('RTEMS Kernel Symbols FAILED', file=sys.stderr)
> +        sys.exit(1)
> +    except KeyboardInterrupt:
> +        log.notice('abort: user terminated')
> +        sys.exit(1)
> +    except exit as ec:
> +        sys.exit(ec.code)
> +    sys.exit(0)
> +
> +
> +if __name__ == "__main__":
> +    run(sys.argv)
> diff --git a/rtemsbsd/include/machine/rtems-bsd-kernel-namespace.h b/rtemsbsd/include/machine/rtems-bsd-kernel-namespace.h
> index 3300b3fd..108709ce 100644
> --- a/rtemsbsd/include/machine/rtems-bsd-kernel-namespace.h
> +++ b/rtemsbsd/include/machine/rtems-bsd-kernel-namespace.h
> @@ -1,5712 +1,5714 @@
> +/*
> +* RTEMS Libbsd, this file is generated. Do not edit.
> +*/
>  #ifndef _RTEMS_BSD_MACHINE_RTEMS_BSD_KERNEL_SPACE_H_
>  #error "the header file <machine/rtems-bsd-kernel-space.h> must be included first"
>  #endif
>


More information about the devel mailing list