<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Aug 1, 2018 at 8:56 AM, Gedare Bloom <span dir="ltr"><<a href="mailto:gedare@rtems.org" target="_blank">gedare@rtems.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On Tue, Jul 31, 2018 at 11:50 PM, Dannie Huang <<a href="mailto:danxue.huang@gmail.com">danxue.huang@gmail.com</a>> wrote:<br>
> Hi,<br>
><br>
> This is the initial implementation commit of converting newlib markup to<br>
> rst, the example .rst file looks like this photo (see attached photo<br>
> please).<br>
><br>
<br>
</span>Can we see a side-by-side comparison with how newlib's documentation looks?<br></blockquote><div><br></div><div>Sure, see attached photo please. Newlib's documentation is on the left and the rst output is on the right.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><div class="h5"><br>
> Code is pushed on my github:<br>
><br>
> <a href="https://github.com/dh0072/NewlibMarkup2SphinxConverter" rel="noreferrer" target="_blank">https://github.com/dh0072/<wbr>NewlibMarkup2SphinxConverter</a><br>
><br>
> Best,<br>
> Dannie<br>
><br>
> On Tue, Jul 31, 2018 at 10:42 PM, Dannie Huang <<a href="mailto:danxue.huang@gmail.com">danxue.huang@gmail.com</a>><br>
> wrote:<br>
>><br>
>> From: Danxue Huang <<a href="mailto:36866155%2Bdh0072@users.noreply.github.com">36866155+dh0072@users.<wbr>noreply.github.com</a>><br>
>><br>
>> ---<br>
>> .gitignore | 2 +<br>
>> README.md | 2 +<br>
>> gen_rst_from_makedoc.py | 125<br>
>> ++++++++++++++++++++++++++++++<wbr>++++++++++++++++++<br>
>> rst.py | 104 ++++++++++++++++++++++++++++++<wbr>++++++++++<br>
>> strcmp.rst | 47 ++++++++++++++++++<br>
>> 5 files changed, 280 insertions(+)<br>
>> create mode 100644 .gitignore<br>
>> create mode 100644 README.md<br>
>> create mode 100755 gen_rst_from_makedoc.py<br>
>> create mode 100644 rst.py<br>
>> create mode 100644 strcmp.rst<br>
>><br>
>> diff --git a/.gitignore b/.gitignore<br>
>> new file mode 100644<br>
>> index 0000000..9f90354<br>
>> --- /dev/null<br>
>> +++ b/.gitignore<br>
>> @@ -0,0 +1,2 @@<br>
>> +.idea/*<br>
>> +__pycache__/*<br>
>> diff --git a/README.md b/README.md<br>
>> new file mode 100644<br>
>> index 0000000..8ebb224<br>
>> --- /dev/null<br>
>> +++ b/README.md<br>
>> @@ -0,0 +1,2 @@<br>
>> +# NewlibMarkup2SphinxConvertorPr<wbr>ivate<br>
>> +(PRIVATE) This repo contains code for<br>
>> NewlibMarkup2SphinxConvertorPr<wbr>ivate.<br>
>> diff --git a/gen_rst_from_makedoc.py b/gen_rst_from_makedoc.py<br>
>> new file mode 100755<br>
>> index 0000000..da69c80<br>
>> --- /dev/null<br>
>> +++ b/gen_rst_from_makedoc.py<br>
>> @@ -0,0 +1,125 @@<br>
>> +#!/usr/bin/env python<br>
>> +#<br>
>> +# RTEMS Tools Project (<a href="http://www.rtems.org/" rel="noreferrer" target="_blank">http://www.rtems.org/</a>)<br>
>> +# Copyright 2018 Danxue Huang (<a href="mailto:danxue.huang@gmail.com">danxue.huang@gmail.com</a>)<br>
>> +# All rights reserved.<br>
>> +#<br>
>> +# This file is part of the RTEMS Tools package in 'rtems-tools'.<br>
>> +#<br>
>> +# Redistribution and use in source and binary forms, with or without<br>
>> +# modification, are permitted provided that the following conditions are<br>
>> met:<br>
>> +#<br>
>> +# 1. Redistributions of source code must retain the above copyright<br>
>> notice,<br>
>> +# this list of conditions and the following disclaimer.<br>
>> +#<br>
>> +# 2. Redistributions in binary form must reproduce the above copyright<br>
>> notice,<br>
>> +# this list of conditions and the following disclaimer in the<br>
>> documentation<br>
>> +# and/or other materials provided with the distribution.<br>
>> +#<br>
>> +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS<br>
>> IS"<br>
>> +# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,<br>
>> THE<br>
>> +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR<br>
>> PURPOSE<br>
>> +# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS<br>
>> BE<br>
>> +# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR<br>
>> +# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF<br>
>> +# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR<br>
>> BUSINESS<br>
>> +# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN<br>
>> +# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)<br>
>> +# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF<br>
>> THE<br>
>> +# POSSIBILITY OF SUCH DAMAGE.<br>
>> +#<br>
>> +<br>
>> +<br>
>> +import argparse<br>
>> +import re<br>
>> +import rst<br>
>> +<br>
>> +<br>
>> +def is_command(s):<br>
>> + """<br>
>> + A command is a single word of at least 3 characters, all uppercase<br>
>> + :param s: input string<br>
>> + :return: True if s is a single command, otherwise False<br>
>> + """<br>
>> + return True if re.match('^[A-Z_]{3,}\s*$', s) else False<br>
>> +<br>
>> +<br>
>> +def extract_comments(content):<br>
>> + """<br>
>> + Extract content inside of /* */<br>
>> + :param content: input file content<br>
>> + :return: extracted comments<br>
>> + """<br>
>> + comments = ''<br>
>> + comments_match = re.match('/\*(\*(?!/)|[^*])*\*<wbr>/', content)<br>
>> + if comments_match:<br>
>> + wrapped_comments = comments_match.group()<br>
>> + comments =<br>
>> wrapped_comments.lstrip('/*').<wbr>rstrip('*/').lstrip().rstrip()<br>
>> + return comments<br>
>> +<br>
>> +<br>
>> +def extract_command_and_text(<wbr>content):<br>
>> + """<br>
>> + Extract command and text from input string content<br>
>> + :param content: input string containing command and text<br>
>> + :return: a tuple containing command and text<br>
>> + """<br>
>> + command = ''<br>
>> + text = ''<br>
>> + command_text_dict = {}<br>
>> + for line in content.splitlines():<br>
>> + if is_command(line):<br>
>> + if command and text:<br>
>> + command_text_dict[command] = text<br>
>> + command = line.rstrip()<br>
>> + text = ''<br>
>> + else:<br>
>> + text += line + '\n'<br>
>> + return command_text_dict<br>
>> +<br>
>> +<br>
>> +def generate_rst(command_text_<wbr>dict):<br>
>> + rst_str = ''<br>
>> + for command, text in command_text_dict.items():<br>
>> + rst_str += rst.get_command_processor(<wbr>command)(command, text)<br>
>> + return rst_str<br>
>> +<br>
>> +<br>
>> +def get_parser():<br>
>> + parser = argparse.ArgumentParser(<br>
>> + description='Convert newlib style markup to rst markup'<br>
>> + )<br>
>> + parser.add_argument(<br>
>> + '-s',<br>
>> + '--source_file_dir',<br>
>> + type=str,<br>
>> + help='Source file directory with newlib style comments',<br>
>> + )<br>
>> + parser.add_argument(<br>
>> + '-d',<br>
>> + '--dest_file_dir',<br>
>> + type=str,<br>
>> + help='Destination directory for converted rst markup file',<br>
>> + )<br>
>> + return parser<br>
>> +<br>
>> +<br>
>> +def main(source_file_dir, dest_file_dir):<br>
>> + with open(source_file_dir, 'r') as source_file, open(dest_file_dir,<br>
>> 'w') as dest_file:<br>
>> + file_content = source_file.read()<br>
>> +<br>
>> + # Get comments inside of /* */<br>
>> + comments = extract_comments(file_content)<br>
>> +<br>
>> + # Parse comments<br>
>> + command_text_dict = extract_command_and_text(<wbr>comments)<br>
>> +<br>
>> + # Process text based on command type<br>
>> + rst_str = generate_rst(command_text_<wbr>dict)<br>
>> +<br>
>> + dest_file.write(rst_str)<br>
>> +<br>
>> +<br>
>> +if __name__ == '__main__':<br>
>> + args = get_parser().parse_args()<br>
>> + main(args.source_file_dir, args.dest_file_dir)<br>
>> diff --git a/rst.py b/rst.py<br>
>> new file mode 100644<br>
>> index 0000000..2531f46<br>
>> --- /dev/null<br>
>> +++ b/rst.py<br>
>> @@ -0,0 +1,104 @@<br>
>> +#<br>
>> +# RTEMS Tools Project (<a href="http://www.rtems.org/" rel="noreferrer" target="_blank">http://www.rtems.org/</a>)<br>
>> +# Copyright 2018 Danxue Huang (<a href="mailto:danxue.huang@gmail.com">danxue.huang@gmail.com</a>)<br>
>> +# All rights reserved.<br>
>> +#<br>
>> +# This file is part of the RTEMS Tools package in 'rtems-tools'.<br>
>> +#<br>
>> +# Redistribution and use in source and binary forms, with or without<br>
>> +# modification, are permitted provided that the following conditions are<br>
>> met:<br>
>> +#<br>
>> +# 1. Redistributions of source code must retain the above copyright<br>
>> notice,<br>
>> +# this list of conditions and the following disclaimer.<br>
>> +#<br>
>> +# 2. Redistributions in binary form must reproduce the above copyright<br>
>> notice,<br>
>> +# this list of conditions and the following disclaimer in the<br>
>> documentation<br>
>> +# and/or other materials provided with the distribution.<br>
>> +#<br>
>> +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS<br>
>> IS"<br>
>> +# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,<br>
>> THE<br>
>> +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR<br>
>> PURPOSE<br>
>> +# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS<br>
>> BE<br>
>> +# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR<br>
>> +# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF<br>
>> +# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR<br>
>> BUSINESS<br>
>> +# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN<br>
>> +# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)<br>
>> +# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF<br>
>> THE<br>
>> +# POSSIBILITY OF SUCH DAMAGE.<br>
>> +#<br>
>> +<br>
>> +<br>
>> +import re<br>
>> +<br>
>> +<br>
>> +def register_processor_command():<br>
>> + return {<br>
>> + 'FUNCTION': gen_function_summary,<br>
>> + 'SYNOPSIS': gen_code_block,<br>
>> + 'ANSI_SYNOPSIS': gen_code_block,<br>
>> + 'TRAD_SYNOPSIS': gen_code_block,<br>
>> + 'TYPEDEF': gen_code_block,<br>
>> + 'DESCRIPTION': gen_custom_directives,<br>
>> + 'INDEX': gen_custom_directives,<br>
>> + 'RETURNS': gen_custom_directives,<br>
>> + 'PORTABILITY': gen_custom_directives,<br>
>> + 'NOTES': gen_custom_directives,<br>
>> + 'ERRORS': gen_custom_directives,<br>
>> + 'BUGS': gen_custom_directives,<br>
>> + 'WARNINGS': gen_custom_directives,<br>
>> + 'QUICKREF': gen_nothing,<br>
>> + 'MATHREF': gen_nothing,<br>
>> + 'NEWPAGE': gen_nothing,<br>
>> + 'START': gen_nothing,<br>
>> + 'END': gen_nothing,<br>
>> + 'SEEALSO': gen_custom_directives,<br>
>> + }<br>
>> +<br>
>> +<br>
>> +def get_command_processor(command)<wbr>:<br>
>> + command_processor_dict = register_processor_command()<br>
>> + if command in command_processor_dict:<br>
>> + return command_processor_dict[<wbr>command]<br>
>> + else:<br>
>> + print('Command {c} is not recognized, skip it'.format(c=command))<br>
>> + return gen_nothing<br>
>> +<br>
>> +<br>
>> +def gen_function_summary(command, text):<br>
>> + function_name = extract_function_name(text)<br>
>> + summary = extract_summary(text)<br>
>> +<br>
>> + title = '.. {f}:\n\n{f} - {s}\n'.format(<br>
>> + f=function_name,<br>
>> + s=summary.capitalize()<br>
>> + )<br>
>> + dashes = '-' * len(text) + '\n'<br>
>> + func_name_index = '.. index:: {name}\n'.format(name=<wbr>function_name)<br>
>> + summary_index = '.. index:: {summary}\n\n'.format(summary=<wbr>summary)<br>
>> + return title + dashes + func_name_index + summary_index<br>
>> +<br>
>> +<br>
>> +def extract_function_name(text):<br>
>> + function_name = ''<br>
>> + function_name_match = re.match('\s+(<<(>(?!>)|[^>])*<wbr>>>)', text)<br>
>> + if function_name_match:<br>
>> + function_name =<br>
>> function_name_match.group(1).<wbr>lstrip('<<').rstrip('>>')<br>
>> + return function_name<br>
>> +<br>
>> +<br>
>> +def extract_summary(text):<br>
>> + return text.split('---')[-1].rstrip()<br>
>> +<br>
>> +<br>
>> +def gen_code_block(command, text):<br>
>> + return '**{c}:**\n\n.. code-block:: c\n\n{t}\n\n'.format(c = command,<br>
>> t=text)<br>
>> +<br>
>> +<br>
>> +def gen_nothing(command, text):<br>
>> + return '\n\n'<br>
>> +<br>
>> +<br>
>> +def gen_custom_directives(command, text):<br>
>> + return '**{c}:**\n\n{t}\n\n'.format(<wbr>c=command, t=text)<br>
>> +<br>
>> diff --git a/strcmp.rst b/strcmp.rst<br>
>> new file mode 100644<br>
>> index 0000000..c1dc543<br>
>> --- /dev/null<br>
>> +++ b/strcmp.rst<br>
>> @@ -0,0 +1,47 @@<br>
>> +.. strcmp:<br>
>> +<br>
>> +strcmp - Character string compare<br>
>> +-----------------------------<wbr>------------<br>
>> +.. index:: strcmp<br>
>> +.. index:: character string compare<br>
>> +<br>
>> +**INDEX:**<br>
>> +<br>
>> + strcmp<br>
>> +<br>
>> +<br>
>> +<br>
>> +**SYNOPSIS:**<br>
>> +<br>
>> +.. code-block:: c<br>
>> +<br>
>> + #include <string.h><br>
>> + int strcmp(const char *<[a]>, const char *<[b]>);<br>
>> +<br>
>> +<br>
>> +<br>
>> +**DESCRIPTION:**<br>
>> +<br>
>> + <<strcmp>> compares the string at <[a]> to<br>
>> + the string at <[b]>.<br>
>> +<br>
>> +<br>
>> +<br>
>> +**RETURNS:**<br>
>> +<br>
>> + If <<*<[a]>>> sorts lexicographically after <<*<[b]>>>,<br>
>> + <<strcmp>> returns a number greater than zero. If the two<br>
>> + strings match, <<strcmp>> returns zero. If <<*<[a]>>><br>
>> + sorts lexicographically before <<*<[b]>>>, <<strcmp>> returns a<br>
>> + number less than zero.<br>
>> +<br>
>> +<br>
>> +<br>
>> +**PORTABILITY:**<br>
>> +<br>
>> +<<strcmp>> is ANSI C.<br>
>> +<br>
>> +<<strcmp>> requires no supporting OS subroutines.<br>
>> +<br>
>> +<br>
>> +<br>
>> --<br>
>> 2.7.4<br>
>><br>
><br>
><br>
</div></div>> ______________________________<wbr>_________________<br>
> devel mailing list<br>
> <a href="mailto:devel@rtems.org">devel@rtems.org</a><br>
> <a href="http://lists.rtems.org/mailman/listinfo/devel" rel="noreferrer" target="_blank">http://lists.rtems.org/<wbr>mailman/listinfo/devel</a><br>
</blockquote></div><br></div></div>