Build failures w/RSB

Sun Mar 22 22:49:54 UTC 2015

Hi Chris,

Thanks for your quick reply.  My host specs were hidden in the original
msg: Linux Mint Debian Edition (all up-to-date: basically, Debian, similar
to Ubuntu, but without all the Ubuntu-only mods) on x86_64; Intel i7,
quad-core, 8G RAM.  My target is a TI Stellaris Launchpad TM4C123GLX board
(the LM4F120 bsp should be fine): that's an ARM M4 core.

Re docs page: yes, that page is the same as what I found; your new link is
better.

Re RSB design: didn't mean to sound so critical, sorry.  What you've done
is amazing and when I first read about it, that convinced me to try the RSB
option instead of working through the RTEMS configure/compile issues I
originally encountered.  I do understand that it can be very difficult to
create a package like RSB and just as difficult to assure all possible
execution paths through it will lead to success (or comprehensible
errors).  I wish I had spare time to look through it in more detail, maybe
I could offer patches -- just not right now.

I too would prefer more RTEMS feature/function/bug fixing instead of RSB
features/flexibility.

Yes, I've been poring over the log and error files.  Just can't tell what
commands follow the gdb configure success (the one immediately after
appears to be the failing command, the one that doesn't find python2.7).  I
didn't go over the RBS source deeply enough to resolve.

I'm now moving into compiling RTEMS directly from git.  Thanks!

Doug

On Sun, Mar 22, 2015 at 10:20 PM, Chris Johns <chrisj at rtems.org> wrote:

> On 23/03/2015 6:50 am, Doug Hawkins wrote:
>
>> Hi,
>>
>> I put my questions at the bottom.  Intervening is "reference" material.
>>
>> I recently read about RTEMS and it seems like a great addition to my
>> project (on a TI LaunchPad), but I'm having trouble building it.  I'm
>> running Linux Mint Debian Edition 64bit.
>>
>> I cloned the git repository for RTEMS and tried to configure it (but
>> automake was a newer version, so there were many warnings). Then I
>> noticed the comments about RTEMS Source Builder and thought that might
>> be a useful way to proceed, since it integrates many of the other
>> packages I'm using (newlib, gdb, etc.).
>>
>> My first question that I could not find in the documentation
>> (https://ftp.rtems.org/pub/rtems/people/chrisj/source-
>> builder/source-builder.html):
>> How can I take advantage of the hundreds of MB's I've already
>> downloaded?  Too late for me, but an answer might help someone else.
>>
>
> The documentation is now located at https://docs.rtems.org/rsb. I need to
> add a redirect.
>
>
>> I ran sb-check and everything was fine.  Then gritted my teeth and ran
>> sb-set-builder and re-downloaded everything. But it failed at an
>> "#include <zlib.h>" while compiling gcc-4.9.2: lto_compress.c... I found
>> the header in the gcc tree (should have been <zlib/zlib.h> and re-ran
>> "sb-set-builder".
>>
>
> The package in question does not check for the header during an early
> configure stage so the build proceeds for a while before it fails. The RSB
> is only wrapping the build process.
>
>
>> Since everything seems to be reset, my <zlib.h> code change was cast
>> out, so the failure occurred again. I finally decided to try installing
>> "zlib1g-dev" package on my system -- that got me through this pickle,
>> though I thought it should look for the local file.
>>
>>
> What host operating system ?
>
>  I also tried the --no-clean and --no-download options, to see if that
>> would speed up this re-cycling.
>>
>
> A bug was fixed with these options on the weekend. The
> --with-*/--without-* option parser ate an extra option and that broke the
> builds.
>
>  But then I learned that not everything
>> was already downloaded, so the next failure occurred when the gdb source
>> package wasn't found.  Each "build" cycle costs me 25 minutes on my
>> four-core i7 machine.
>>
>
> If you hit the --with-download bug then this could happen.
>
>  Re-starting without the --no-download & --no-clean options (I've given
>> up on "optimizing" this process) -- next failure is telling me that
>> python2.7 doesn't exist (it is installed on my system).  I've spent the
>> last couple hours searching through configure code & run logs to try to
>> understand what part of python2.7 it needs.
>>
>> I did find the gdb configure call just before the failure, but when I
>> re-ran it, it completed successfully, duplicating the log messages up
>> to, but except the last 40 lines or so.  The start of that sequence
>> (which seemed too generic to locate) was:
>>
>> checking whether NLS is requested... no
>> checking whether makeinfo --split-size=5000000 supports @click... yes
>> checking for default auto-load directory... $debugdir:$datadir/auto-load
>> checking for default auto-load safe-path... $debugdir:$datadir/auto-load
>>
>> I've finally given up!
>>
>
> Which host ? What architecture ?
>
>   Here are the remaining questions running through
>> my mind.
>>
>> 1) Is there any way I can optimize the compile process (not rebuild
>> everything when a failure occurs)?  Specifically: how do I "resume"
>> sb-set-builder?  It appears that I have to check all repositories &
>> rebuild everything each time I run it.
>>
>
> Sorry there is nothing currently that can do this, and my time is limited.
> It is a repeated question and worthy of a full answer. If there is
> something you can add to the documentation in the doc directory of the RSB
> for your host that helps please provide a patch.
>
> The complexity of adding this is the side effects it beings. The main
> focus of the RSB is to provide a user with a tool set that consistent and
> repeated. Add such a feature would require I make sure there are no side
> effects of the build process which could result in a partial or mixed build
> that may appear ok but is not. The feature would require a hash of the
> contents of each stage would need to be made and verified. This means you
> cannot cheat and alter a completed build stage and doing so would restart
> that stage. At the moment the RSB makes no assumptions about files or data
> in the build tree and a restart feature would require adding 'cached' data
> and state information and my experiences with 'cached' data and state is
> the complexity rises.
>
> Looking at the timely builds such as a tool set the autotools is fast, and
> binutils is fast, so it is basically gcc that is taking the time. I do not
> have the figures in front of me but lets say is it 75% of the total time
> and it is also the main reason a build fails, is adding the feature and the
> complexity worth while. From where I sit having to implement it I can not
> as sure given there are other things that RTEMS could have before this.
>
> The other approach is to teach the RSB to look for libraries. The RSB has
> no integration with host package management system and I think this is a
> good thing. This means we would need to look for a different approach to
> solving this problem. An idea I have been considering is to create a series
> of waf scripts that check for various libraries. Each package in a build
> set can define a list of checks to be made and the RSB collects these when
> the configuration files are read and then executables the tests before
> running. This would expose any system libraries or tools not present early.
> The only restriction is not adding a dependency to packages built before
> hand.
>
> Finally, this RSB has been having a lot of churn recently as a range of
> changes for the 4.11 have been applied. My hope is this will slow down once
> we have gdb-7.9 sorted, and the recent rtems-tools waf issue has been
> resolved.
>
>
>> 2) When a failure occurs, how can I locate the "real" problem and find
>> information on how to resolve it?  It seems that the source-builder is
>> too complex & doesn't have enough documentation for me to dig deep
>> enough into the sorts of problems I'm encountering.
>>
>
> If you have suggestions for the documentation in the 'doc' directory
> please make them.
>
>  On other
>> "meta-builders" I've worked with, there is usually a large header pushed
>> into the log file for each execution segment, with all the necessary
>> options to re-run that segment manually, but I couldn't find anything
>> like that in the log file.
>>
>
> There was no .log file in the directory you ran the command in ?
>
> If you get an issue it is best to post the rsb-report-* file. Each package
> can have different configure and build systems and each results in a
> different way to find errors. For autoconf type configure you need to find
> the config.log file and then know how to find the error in it.
>
>
>> I had a look through some of set-builder's python code, but I'd need
>> more documentation there to understand some of the structures (I was
>> hoping to redirect the sources to my already existing repositories).
>>
>> My log file is over 90MB now, the error log files are each about 280kB.
>> I could compress them and send, if needed, but won't do that until
>> requested.
>>
>
> So you found the .log file and the error reports files ?
>
>
>> Any help that can be offered would be greatly appreciated.
>>
>>
> I hope this helps.
>
> Chris
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/users/attachments/20150322/86d19237/attachment-0002.html>