waf and Python 3.7

Wed Dec 19 09:20:56 UTC 2018

Am 19.12.18 um 06:20 schrieb Chris Johns:
> On 19/12/2018 14:10, Joel Sherrill wrote:
>> On Tue, Dec 18, 2018, 7:31 PM Chris Johns <chrisj at rtems.org
>> <mailto:chrisj at rtems.org> wrote:
>>     On 19/12/2018 11:49, Joel Sherrill wrote:
>>     > On Tue, Dec 18, 2018 at 5:33 PM Chris Johns <chrisj at rtems.org
>>     <mailto:chrisj at rtems.org>
>>     > <mailto:chrisj at rtems.org <mailto:chrisj at rtems.org>>> wrote:
>>     >     On 19/12/2018 06:12, Christian Mauderer wrote:
>>     >     > I updated waf in rtems-tools with a reference to the ticket. I'm still a
>>     >     > little unsure about our tickets: Should I create separate tickets for
>>     >     > that problem in rtems-docs, rtems-libbsd (both have the same problem)
>>     >     > and examples-v2 (I haven't tested that yet but I would expect the
>>     >     > problem there too).
>>     >
>>     >     No, the same ticket should be fine. Maybe update the ticket's details to
>>     >     highlight this is happening. Separate tickets are needed for separate
>>     branches
>>     >     because the timeline cannot be shared.
>>     >
>>     >     > Note that rtems-docs needs an extra patch to build with python 3. There
>>     >     > is an indentation error (mix of space and tab).
>>     >
>>     >     Yes I have tripped over this but I have not had time to push a fix.
>>     Building PDF
>>     >     is broken for me with the latest Sphinx. I am on a fully updated
>>     FreeBSD 11.2-p5
>>     >     and all packages are updated plus the latest Sphinx in a virtualend.
>>     The issue
>>     >     seems to be deep in Latex and relates to unicode characters in bib
>>     entries with
>>     >     hyperef and the other package and things being confused. I do not know
>>     if it is
>>     >     a bug in Sphinx or our configuration.
>>     >
>>     >
>>     > FWIW I am building html fine with Python 2 and Sphinx 1.6, 1.7, and 1.8.2
>>     on three
>>     > different CentOS computers. But PDF doesn't work on any of those platforms. 
>>
>>     The build on sync.rtems.org <http://sync.rtems.org> for the online manuals
>>     is failing with unicode errors:
>>
>>     $ file `find . -name \*.rst` | grep Unicode
>>     ./eng/users-manuals.rst:                          UTF-8 Unicode text
>>     ./eng/test-plan.rst:                              UTF-8 Unicode text
>>     ./eng/vc-users.rst:                               UTF-8 Unicode text
>>     ./eng/appendix-a.rst:                             UTF-8 Unicode text
>>     ./eng/vc-authors.rst:                             UTF-8 Unicode text
>>     ./eng/license-requirements.rst:                   UTF-8 Unicode text
>>     ./eng/prequalification.rst:                       UTF-8 Unicode text
>>     ./user/tracing/examples.rst:                      UTF-8 Unicode text
>>     ./user/tracing/introduction.rst:                  UTF-8 Unicode text
>>     ./bsp-howto/i2c.rst:                              UTF-8 Unicode text
>>
>> I can't try this tonight but Google reminded me that I did this on another
>> project with the iconv utility. 
>>
>> This happens when you copy from a website or MS-Word to text. 
>>
>> I can fix it tomorrow if you can't find the command.
> 
> Thanks.
> 
>>     This raises 2 issues, we need the docs to build so the listed files need to be
>>     cleaned and the second issue is being able to support unicode. That is a
>>     separate development effort that will need to be worked on.
>>
>> I should be able to fix those if you can't.
> 
> I have not been able to get to this today.
> 
>>
>> How about a git check for Unicode. :)
>>
> 
> I am considering a waf build check.
> 
> Chris
> 

Hello Chris and Joel,

I think you are discussing some other bug(s) than the one I originally
meant. First of all: I tried a pdf build. But I receive the same error
on a html build. See attached log for the complete output with python2
on the current HEAD (75f24638b3f).

What I meant is a simple indentation error in common/waf.py. Python 3
seems to be a little picky about these. It throws a "TabError:
inconsistent use of tabs and spaces in indentation". I'll post a patch
for review in a few minutes.

Regarding the ticket: I updated the title and added a comment to reflect
the fact that multiple repos are affected.

I'll update the waf in libbsd soon. I haven't tested examples-v2 yet so
this will need some more time. And I won't update rtems-docs till I'm
able to build it. I'll also post a patch for review that will update the
rtems-tools in RSB to use the new version.

Best regards

Christian
-------------- next part --------------
…ome/christian/rtems/rtems-docs 10:08 > python2 ./waf configure -v
Setting top to                           : /home/christian/archive/rtems-docs 
Setting out to                           : /home/christian/archive/rtems-docs/build 
Checking version                         : 5.0.0 (master) 
Checking for program 'sphinx-build'      : /usr/bin/sphinx-build 
Checking for program 'aspell'            : /usr/bin/aspell 
Checking if Sphinx is at least 1.3       : yes (1.8) 
Checking Sphinx Verbose                  : quiet 
Checking for 'sphinx.ext.autodoc'        : 10:08:09 runner ' /usr/bin/sphinx-build -b html -D extensions=sphinx.ext.autodoc -C . out '
found 
Checking for 'sphinx.ext.coverage'       : 10:08:10 runner ' /usr/bin/sphinx-build -b html -D extensions=sphinx.ext.coverage -C . out '
found 
Checking for 'sphinx.ext.doctest'        : 10:08:11 runner ' /usr/bin/sphinx-build -b html -D extensions=sphinx.ext.doctest -C . out '
found 
Checking for 'sphinx.ext.graphviz'       : 10:08:12 runner ' /usr/bin/sphinx-build -b html -D extensions=sphinx.ext.graphviz -C . out '
found 
Checking for 'sphinx.ext.intersphinx'    : 10:08:13 runner ' /usr/bin/sphinx-build -b html -D extensions=sphinx.ext.intersphinx -C . out '
found 
Checking for 'sphinx.ext.mathjax'        : 10:08:14 runner ' /usr/bin/sphinx-build -b html -D extensions=sphinx.ext.mathjax -C . out '
found 
Checking for 'sphinxcontrib.bibtex'      : 10:08:14 runner ' /usr/bin/sphinx-build -b html -D extensions=sphinxcontrib.bibtex -C . out '
found 
'configure' finished successfully (6.070s)
…ome/christian/rtems/rtems-docs 10:08 > python2 ./waf -v
Waf: Entering directory `/home/christian/archive/rtems-docs/build'
[1/4] Compiling common/_static/favicon.ico
[2/4] Compiling common/_static/logo.png
[3/4] Compiling common/_static/my-styles.css
[4/4] Compiling common/_static/style.css
[5/5] Compiling common/_templates/layout.html
[ 6/10] Processing /home/christian/archive/rtems-docs/build/user/html/index.html: user/apps/index.rst user/bsp/index.rst user/bsps/bsps-aarch64.rst user/bsps/bsps-arm.rst user/bsps/bsps-bfin.rst user/bsps/bsps-epiphany.rst user/bsps/bsps-i386.rst user/bsps/bsps-lm32.rst user/bsps/bsps-m68k.rst user/bsps/bsps-microblaze.rst user/bsps/bsps-mips.rst user/bsps/bsps-moxie.rst user/bsps/bsps-nios2.rst user/bsps/bsps-or1k.rst user/bsps/bsps-powerpc.rst user/bsps/bsps-riscv.rst user/bsps/bsps-sh.rst user/bsps/bsps-sparc.rst user/bsps/bsps-sparc64.rst user/bsps/bsps-v850.rst user/bsps/bsps-x86_64.rst user/bsps/index.rst user/ecosys/index.rst user/exe/debugging.rst user/exe/executables.rst user/exe/execution.rst user/exe/index.rst user/exe/initialization.rst user/glossary/index.rst user/hardware/architectures.rst user/hardware/bsps.rst user/hardware/index.rst user/hardware/targets.rst user/hardware/tiers.rst user/hosts/index.rst user/hosts/macos.rst user/hosts/os.rst user/hosts/posix.rst user/hosts/windows.rst user/index.rst user/installation/developer.rst user/installation/index.rst user/installation/kernel.rst user/installation/prefixes-sandboxing.rst user/installation/releases.rst user/overview/index.rst user/start/index.rst user/support/index.rst user/testing/configuration.rst user/testing/consoles.rst user/testing/gdb-jtag.rst user/testing/index.rst user/testing/simulation.rst user/testing/tests.rst user/testing/tftp.rst user/tools/bsp-builder.rst user/tools/exeinfo.rst user/tools/index.rst user/tools/linker.rst user/tools/symbols.rst user/tools/tester.rst user/tracing/captureengine.rst user/tracing/examples.rst user/tracing/index.rst user/tracing/introduction.rst user/tracing/tracelinker.rst -> build/user/html/index.html
10:08:20 runner ' /usr/bin/sphinx-build -Q -b html -c /home/christian/archive/rtems-docs/user -Drelease="5.0.0 (master)" -Dversion="5.0.0 (master)" -d /home/christian/archive/rtems-docs/build/user/doctrees/html  -D templates_path=/home/christian/archive/rtems-docs/build/user/html/_templates /home/christian/archive/rtems-docs/user /home/christian/archive/rtems-docs/build/user/html apps/index.rst bsp/index.rst bsps/bsps-aarch64.rst bsps/bsps-arm.rst bsps/bsps-bfin.rst bsps/bsps-epiphany.rst bsps/bsps-i386.rst bsps/bsps-lm32.rst bsps/bsps-m68k.rst bsps/bsps-microblaze.rst bsps/bsps-mips.rst bsps/bsps-moxie.rst bsps/bsps-nios2.rst bsps/bsps-or1k.rst bsps/bsps-powerpc.rst bsps/bsps-riscv.rst bsps/bsps-sh.rst bsps/bsps-sparc.rst bsps/bsps-sparc64.rst bsps/bsps-v850.rst bsps/bsps-x86_64.rst bsps/index.rst ecosys/index.rst exe/debugging.rst exe/executables.rst exe/execution.rst exe/index.rst exe/initialization.rst glossary/index.rst hardware/architectures.rst hardware/bsps.rst hardware/index.rst hardware/targets.rst hardware/tiers.rst hosts/index.rst hosts/macos.rst hosts/os.rst hosts/posix.rst hosts/windows.rst index.rst installation/developer.rst installation/index.rst installation/kernel.rst installation/prefixes-sandboxing.rst installation/releases.rst overview/index.rst start/index.rst support/index.rst testing/configuration.rst testing/consoles.rst testing/gdb-jtag.rst testing/index.rst testing/simulation.rst testing/tests.rst testing/tftp.rst tools/bsp-builder.rst tools/exeinfo.rst tools/index.rst tools/linker.rst tools/symbols.rst tools/tester.rst tracing/captureengine.rst tracing/examples.rst tracing/index.rst tracing/introduction.rst tracing/tracelinker.rst '
[ 7/10] Compiling common/_static/favicon.ico
[ 8/10] Compiling common/_static/logo.png
[ 9/10] Compiling common/_static/my-styles.css
[10/10] Compiling common/_static/style.css

Sphinx error:
master file /home/christian/archive/rtems-docs/user/contents.rst not found

Waf: Leaving directory `/home/christian/archive/rtems-docs/build'
Build failed
 -> task in '/home/christian/archive/rtems-docs/build/user/html/index.html' failed with exit status 2: 
        {task 139804728900256: /home/christian/archive/rtems-docs/build/user/html/index.html index.rst,index.rst,bsps-aarch64.rst,bsps-arm.rst,bsps-bfin.rst,bsps-epiphany.rst,bsps-i386.rst,bsps-lm32.rst,bsps-m68k.rst,bsps-microblaze.rst,bsps-mips.rst,bsps-moxie.rst,bsps-nios2.rst,bsps-or1k.rst,bsps-powerpc.rst,bsps-riscv.rst,bsps-sh.rst,bsps-sparc.rst,bsps-sparc64.rst,bsps-v850.rst,bsps-x86_64.rst,index.rst,index.rst,debugging.rst,executables.rst,execution.rst,index.rst,initialization.rst,index.rst,architectures.rst,bsps.rst,index.rst,targets.rst,tiers.rst,index.rst,macos.rst,os.rst,posix.rst,windows.rst,index.rst,developer.rst,index.rst,kernel.rst,prefixes-sandboxing.rst,releases.rst,index.rst,index.rst,index.rst,configuration.rst,consoles.rst,gdb-jtag.rst,index.rst,simulation.rst,tests.rst,tftp.rst,bsp-builder.rst,exeinfo.rst,index.rst,linker.rst,symbols.rst,tester.rst,captureengine.rst,examples.rst,index.rst,introduction.rst,tracelinker.rst -> index.html}
' /usr/bin/sphinx-build -Q -b html -c /home/christian/archive/rtems-docs/user -Drelease="5.0.0 (master)" -Dversion="5.0.0 (master)" -d /home/christian/archive/rtems-docs/build/user/doctrees/html  -D templates_path=/home/christian/archive/rtems-docs/build/user/html/_templates /home/christian/archive/rtems-docs/user /home/christian/archive/rtems-docs/build/user/html apps/index.rst bsp/index.rst bsps/bsps-aarch64.rst bsps/bsps-arm.rst bsps/bsps-bfin.rst bsps/bsps-epiphany.rst bsps/bsps-i386.rst bsps/bsps-lm32.rst bsps/bsps-m68k.rst bsps/bsps-microblaze.rst bsps/bsps-mips.rst bsps/bsps-moxie.rst bsps/bsps-nios2.rst bsps/bsps-or1k.rst bsps/bsps-powerpc.rst bsps/bsps-riscv.rst bsps/bsps-sh.rst bsps/bsps-sparc.rst bsps/bsps-sparc64.rst bsps/bsps-v850.rst bsps/bsps-x86_64.rst bsps/index.rst ecosys/index.rst exe/debugging.rst exe/executables.rst exe/execution.rst exe/index.rst exe/initialization.rst glossary/index.rst hardware/architectures.rst hardware/bsps.rst hardware/index.rst hardware/targets.rst hardware/tiers.rst hosts/index.rst hosts/macos.rst hosts/os.rst hosts/posix.rst hosts/windows.rst index.rst installation/developer.rst installation/index.rst installation/kernel.rst installation/prefixes-sandboxing.rst installation/releases.rst overview/index.rst start/index.rst support/index.rst testing/configuration.rst testing/consoles.rst testing/gdb-jtag.rst testing/index.rst testing/simulation.rst testing/tests.rst testing/tftp.rst tools/bsp-builder.rst tools/exeinfo.rst tools/index.rst tools/linker.rst tools/symbols.rst tools/tester.rst tracing/captureengine.rst tracing/examples.rst tracing/index.rst tracing/introduction.rst tracing/tracelinker.rst '
…ome/christian/rtems/rtems-docs 10:08 >