[GSOC] : Patches for merging different syslog.h
Joel Sherrill
joel.sherrill at OARcorp.com
Sun Apr 14 18:40:57 UTC 2013
On 4/14/2013 11:50 AM, Rempel, Cynthia wrote:
> Hi Vipul Nayyar,
>
> Thanks for doing this research! This is really encouraging!
Indeed! This is a hard and important project. Unfortunately it is also
tedious
and we will be capturing the existing practice rules for .h files as we
go and
fixing violations.
It is likely best viewed as a mountain of toothpicks. You will end up
focusing
on a single file at a time say "XXX.h" can be installed" or "can be
private" or
"XXX.h appears to be installed under the wrong subdirectory". Then after a
review, you will be given permission to move it, fix the Makefiles, etc.
Hopefully you will automate the rule checking. :)
> Wow, there are a lot of header files... Looks like we might be identifying which headers to not install more than merging, in either case, verifying each change to the installed headers won't break something will probably not be possible over the summer, so one of the primary goals of this project would be to outline how to identify which installed headers should be private (and not install them instead of merging them), and then outline how to merge the headers together by name.
Good point. Not installing a conflicting .h file is even better. :)
If you only need <uart.h> to build your BSP. Leave it in the source
tree. This IS TESTABLE
because the scope of the change is limited.
I am assuming that for chip specific .h files, this is possible but for
bus oriented ones
which might be used to access cards/devices added by the user, this is
not possible.
For example, user applications might include device drivers for VMEBus,
CompactPCI,
PCI, and AMBA bus devices.
But I think you would quickly get a set of rules as to reliably
determine if it could be
private (e.g. not installed) or must be installed. You could start with
a "must install"
set per BSP (e.g. bsp.h is one) and what they directly include. That
defines the set
currently in the "must install" set. If it is only included by a .c file
in the BSP, then
it is a first tier candidate for not installing.
And.. preinstall.am is your friend. Write scripts to get the list of
installed .h files and analyze them. Then let the script know the rules
for recommendations based on the written rules (that we will all help
you develop).
Poking around as I reply to your emails, I see a few interesting cases.
+ multiple uart.h files
+ tm27.h - installed by every BSP but only used in one Timing Test
+ bootcard.h - I would have thought it was a private to the build only
but one test includes it.
Again.. thinking out load.. capturing the decision process for
private/public
presence of the .h and writing a script to check that would be helpful.
There
are lots of cases.
Any .h installed by libcpu, libchip, and libbsp would have to be put through
the criteria tested by the script which implements the written rules.
This may sound scary but I believe that a lot of BSP specific files will
turn
out to be private. If there name is unique to a single BSP or not
universally
guaranteed to be available, then it is likely to be private.
Exception again being capabilities like buses and interrupt APIs needed to
write device drivers in user space. Files like <erc32.h> which have been
installed in the past and define system on chip addresses will probably
require
discussion but I would tend to try to make them private. We could err on
the side of private and publish them given a user problem. That is a
community
decision though.
> I suspect the following headers should be verified as private /machine/, and /bsp/ (and not to be installed)
> I have no idea if /score/ headers should be public or private, but they might be private, (in which case we could ask they not be installed instead of merged)... further research of the rtems documentation is needed
/machine/ is likely an implementation detail of libc and/or FreeBSD. I
don't know
that this convention is used by any code owned by the RTEMS Project. I would
consider those off-limits by default.
c/src/libbsp, libcpu, and libchip are where the focus must be.
/score/ very likely could be private to a large extent. EXCEPT that we
allow a few
APIs to creep across the border .. chain, interrupt, scheduler plugins,
come to mind.
cpukit/rtems/include/rtems/rtems/*.h need attention to separate private
implementation
details from public. This is needed.
cpukit/posix is better about separating public from private.
> Not merging some of the /sys/ headers may be okay (depending on opengroup guidance).
/sys/ tends to walk into POSIX and C standard code or
*BSD and libc code which we don't own.
> Part of this project could be to outline how we can verify not installing those headers won't break building external applications, such as example-v2, then to outline what's left to do (hopefully we could get this detailed out to the point we could take care of this using Google Code In 2013/2014 assistance, as the remaining work lends itself to micro-projects).
The meat is in the BSPs where there is the least review and most
variation in coding
style. If he can develop a script to check a set of rules and make a
recommendation,
then I think we could crank through the BSP specific .h files which
should be private
in this project.
> I suspect one approach to verifying we can get by with not installing private headers is to export the rtems/testsuite to a stand-alone git, so it's an external application, and build the testsuite again on a fresh rtems install for each removed header. We would probably also want to include script to check the output of the testsuite for breakage. Running the script would have to be documented in the README. We would also want a to keep the exported testsuite as in sync with the internal testsuite as feasible... to export the testsuite means dealing with a small amount of issues with the build system... but this would also require assistance from someone with committer access...
Would it be possible to simply know what .h files are directly included
by the test code
and support files?
If they aren't directly included, then the change shouldn't impact the
tests.
We do have to build to be sure but I think a script which checks a
particular
<XXX.h> against the known set of include files needed by the tests. I have
attached a shell script that looks at all the direct #include's in the
testsuites,
then eliminates those files (1) in the test suite directory, then (2) in
libnetworking
and then in (3) libc. A few more filters for various directories in cpukit
and you will have something close to the answer. :)
A good thing to know is whether any code in libcpu, libchip or libbsp
installs
a .h file so that it is included as <rtems/XXX.h>. I wouldn't say it is
immediately
wrong but it needs questioning.
> Once that's completed, the outline could be improved by using it to identify which private headers we can get rid of, and submitting patches to rtems-devel to make them private, and verifying all the steps for the outline are there... it may be interesting to see what headers still need merging after not installing them (if any)...
I think this is a good thing to focus on. Make them private. Then they
do not pollute the
public "include namespace". End of story for that file.
>
> If it helps, typically I update the preinstall.am by runing ./bootstrap -c && ./bootstrap -p && ./bootstrap .
And you can do that from any directory with a configure.ac. When hacking
at the BSP
level, you can do it in a specific BSP directory and it runs much quicker.
> Over the summer, we typically have patches reviewed on rtems-devel to verify we don't accidentally break something... We'll probably need to identify which committer(s) will commit your patches, although we may be getting build-bot at some point, but I'm not sure how soon that will happen...
This definitely is a project with a lot of small projects and they will
need to be submitted
incrementally.
> Hope this helps! Thanks again for the script, seeing the big picture sure puts things into perspective...
This is a big and important project. It can be overwhelming. But if you
focus on one
file at a time. Script the rules why it is public/private, etc. Then as
you go on, you will
get faster and faster at the analysis. And we will have written rules
developed on
.h files naming and installation.
Don't be intimidated. I personally think the GSOC projects which are
mountains
of toothpicks are easier for a student if they can focus on one thing at
a time and
not get overwhelmed. Your changes will be integrated as a long sequence of
small patches.
> Cynthia Rempel
> ________________________________________
> From: Vipul Nayyar [nayyar_vipul at yahoo.com]
> Sent: Sunday, April 14, 2013 7:27 AM
> To: Rempel, Cynthia
> Cc: rtems-devel at rtems.org
> Subject: [GSOC] : Patches for merging different syslog.h
>
> Hello Cynthia Rempel,
>
> As advised by you, I tried to merge header files with same name getting installed into different places. For this task, I chose to merge two 'syslog.h' files present in libnetworking directory , as a starting point. After merging them and editing other files which required this header file, I re-compiled RTEMS successfully. I've attached two patches with this email. Please check them if they're ok and good to be applied.
>
> To have an idea that how many duplicate header files are actually present in /opt/rtems-4.11/sparc-rtems4.11 , I wrote a shell script for that. The output of that bash script is also attached with this email. As you can see in that text file, there are quite many duplicate header files present . Please tell me any plan or steps that I should use to merge so many files, if possible.
>
> Also please explain the procedure to obtain committer access as written here : http://rtems.org/wiki/index.php/Git_Committers.
> Thanks for your support in advance.
>
> Regards
> Vipul Nayyar
> Computer Engineering
> Jamia Millia Islamia
> New Delhi
>
>
>
>
>
>
> _______________________________________________
> rtems-devel mailing list
> rtems-devel at rtems.org
> http://www.rtems.org/mailman/listinfo/rtems-devel
--
Joel Sherrill, Ph.D. Director of Research & Development
joel.sherrill at OARcorp.com On-Line Applications Research
Ask me about RTEMS: a free RTOS Huntsville AL 35805
Support Available (256) 722-9985
-------------- next part --------------
#! /bin/bash
#
# If duped in output, then "XXX.h" and <XXX.h" were found. FIX
if [ ! -r PROBLEMS ] ; then
echo "Are you in rtems/testsuites?"
exit 1
fi
target=sparc-rtems4.11
tname=`type ${target}-gcc | cut -d' ' -f3 2>/dev/null`
if [ $? -ne 0 ] ; then
echo "Did not find ${target}-gcc"
exit 1
fi
echo $tname
libcdir=`dirname $tname`/../${target}/include
find . -name "*.[ch]" | xargs -e grep "^#include" | \
cut -d' ' -f2 | sort -u | \
while read fraw
do
f=`echo $fraw | tr -d "<>\""`
#echo === $f
bf=`basename $f`
count=`find . -name "$bf" | wc -l 2>/dev/null`
# provided by test suite, OK
if [ $count -ne 0 ] ; then
continue
fi
# Was it part of the C library?
# If yes, then ignore
if [ -r ${libcdir}/${f} ] ; then
#echo "in libc/${f}"
continue
fi
# ... more tests to see if it came from somewhere we don't control
if [ -r ../cpukit/libnetworking/${f} ] ; then
#echo "in libnetworking/${f}"
continue
fi
echo $f
done
More information about the devel
mailing list