RTEMS Source Builder newlib/iconv issues on OpenBSD/Solaris

[Chris Johns]

On 22/06/2015 11:46 pm, Karel Gardas wrote:
> On 06/22/15 02:35 PM, Chris Johns wrote:
>> On 22/06/2015 6:30 am, Karel Gardas wrote:
>>>
>>> Hello,
>>>
>>> on both Solaris 11 and OpenBSD 5.7 I do have an issue with source
>>> builder which fails on newlib compilation. This shows as:
>>>
>>> checking for sparc-rtems4.11-ar... sparc-rtems4.11-ar
>>> checking for sparc-rtems4.11-ranlib... sparc-rtems4.11-ranlib
>>> checking for sparc-rtems4.11-readelf... sparc-rtems4.11-readelf
>>> checking whether to enable maintainer-specific portions of
>>> Makefiles... no
>>> checking for a sed that does not truncate output...
>>> /tmp/rtems-sparc-toolchain/bin//sed
>>> checking how to print strings... print -r
>>> checking for gawk... (cached) gawk
>>> configure: error: big5 is not supported - see
>>> ../../../gcc-4.9.2/newlib/libc/iconv/encoding.aliases file for the list
>>> of available encodings
>>> gmake[1]: *** [configure-target-newlib] Error 1
>>> gmake[1]: Leaving directory
>>> `/export/home/karel/vcs/rtems-source-builder/rtems/build/sparc-rtems4.11-gcc-4.9.2-newlib-2.2.0.20150423-i386-pc-solaris2-1/build'
>>>
>>>
>>> gmake: *** [all] Error 2
>>>
>>>
>>> the problem is in newlib's configure file which strictly requires GNU
>>> sed/grep combo. The problem is the configure file itself invokes just
>>> "sed" and "grep". So there are two solutions to this issue:
>>>
>>> Either:
>>>
>>> - disable iconv as I did for OpenBSD in provided patch in source builder
>>> platform file
>>>
>>> or
>>>
>>> - put GNU sed and grep (perhaps grep is enough, but I would need to
>>> retest) into PATH in front of original system's grep/sed. This makes
>>> newlib configure happy.
>>>
>>> The general question is what's better and how to solve this issue?
>>
>> We should solve the problem. I build on FreeBSD without GNU sed so I am
>> interested why OpenBSD would break this way.
> 
> My guess is that the issue is in this code:
> 
>     iconv_encodings=`echo "${iconv_encodings}" | sed -e 's/,/ /g' -e
> 's/-/_/g' -e 'y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/'`
>   iconv_to_encodings=`echo "${iconv_to_encodings}" | sed -e 's/,/ /g' -e
> 's/-/_/g' -e 'y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/'`
>   iconv_from_encodings=`echo "${iconv_from_encodings}" | sed -e 's/,/
> /g' -e 's/-/_/g' -e
> 'y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/'`
> 
>       available_encodings=`cat "${srcdir}/libc/iconv/encoding.aliases" |
> sed -e '/^#.*/d'`
> 
> 
>     iconv_encodings1="${iconv_encodings}"
>   iconv_encodings=""
>   for encoding in ${iconv_encodings1}; do
>       result=`echo "${available_encodings}" | grep -e "\(^\|
> \)${encoding}\( \|\$\)"`
>       if test $? != "0"; then
>           as_fn_error $? "${encoding} is not supported - see
> ${srcdir}/libc/iconv/encoding.aliases file for the list of available
> encodings" "$LINENO" 5
>       fi
>       encoding1=`echo "${result}" | sed -e 's/\(^[^ ]*\).*$/\1/'`
>       iconv_encodings="${iconv_encodings} ${encoding1}"
>   done
> 

I cannot see anything here that is a GNU specific thing then again it is
never easy to spot.

Do you have the ${iconv_encodings} value ?

> 
> both sed and grep are used there. Honestly I'm still not able to debug
> that since for me source builder is black box which needs to run from
> the start to end and I'm not probably not allowed to hack some debugging
> messages there to see if the grep or sed is actually the culprit here.
>

This question is not often asked so it is something I have no
documentation for. What I do is cd into the build tree and find the
specific build that is failing. Often there is a config.status or
config.log which contains the command line and run things by hand until
I figure out what is happening. The only downside is the environment may
not be set up as it is inside the RSB and that can be a pain.

> Anyway, "sed -e" rings the bell here since I've already seen quite some
> issue with it on Solaris 11 in GHC project, E.g.
> https://phabricator.haskell.org/D740 -- but this is just a tip of the
> iceberg.
> 
> Hence my "solution" to use GNU sed/grep solely for this...
> 

It is a solution but it just means the standard has moved and that is a
shame.

> BTW: I'm talking about Solaris 11 since symptomps are the same on
> Solaris 11 and on OpenBSD... Debugging may actually reveals different
> reason...

Sure.

Chris