[PATCH] Initial implemantation commit
Dannie Huang
danxue.huang at gmail.com
Thu Aug 2 05:03:48 UTC 2018
On Wed, Aug 1, 2018 at 8:56 AM, Gedare Bloom <gedare at rtems.org> wrote:
> On Tue, Jul 31, 2018 at 11:50 PM, Dannie Huang <danxue.huang at gmail.com>
> wrote:
> > Hi,
> >
> > This is the initial implementation commit of converting newlib markup to
> > rst, the example .rst file looks like this photo (see attached photo
> > please).
> >
>
> Can we see a side-by-side comparison with how newlib's documentation looks?
>
Sure, see attached photo please. Newlib's documentation is on the left and
the rst output is on the right.
>
> > Code is pushed on my github:
> >
> > https://github.com/dh0072/NewlibMarkup2SphinxConverter
> >
> > Best,
> > Dannie
> >
> > On Tue, Jul 31, 2018 at 10:42 PM, Dannie Huang <danxue.huang at gmail.com>
> > wrote:
> >>
> >> From: Danxue Huang <36866155+dh0072 at users.noreply.github.com>
> >>
> >> ---
> >> .gitignore | 2 +
> >> README.md | 2 +
> >> gen_rst_from_makedoc.py | 125
> >> ++++++++++++++++++++++++++++++++++++++++++++++++
> >> rst.py | 104 ++++++++++++++++++++++++++++++++++++++++
> >> strcmp.rst | 47 ++++++++++++++++++
> >> 5 files changed, 280 insertions(+)
> >> create mode 100644 .gitignore
> >> create mode 100644 README.md
> >> create mode 100755 gen_rst_from_makedoc.py
> >> create mode 100644 rst.py
> >> create mode 100644 strcmp.rst
> >>
> >> diff --git a/.gitignore b/.gitignore
> >> new file mode 100644
> >> index 0000000..9f90354
> >> --- /dev/null
> >> +++ b/.gitignore
> >> @@ -0,0 +1,2 @@
> >> +.idea/*
> >> +__pycache__/*
> >> diff --git a/README.md b/README.md
> >> new file mode 100644
> >> index 0000000..8ebb224
> >> --- /dev/null
> >> +++ b/README.md
> >> @@ -0,0 +1,2 @@
> >> +# NewlibMarkup2SphinxConvertorPrivate
> >> +(PRIVATE) This repo contains code for
> >> NewlibMarkup2SphinxConvertorPrivate.
> >> diff --git a/gen_rst_from_makedoc.py b/gen_rst_from_makedoc.py
> >> new file mode 100755
> >> index 0000000..da69c80
> >> --- /dev/null
> >> +++ b/gen_rst_from_makedoc.py
> >> @@ -0,0 +1,125 @@
> >> +#!/usr/bin/env python
> >> +#
> >> +# RTEMS Tools Project (http://www.rtems.org/)
> >> +# Copyright 2018 Danxue Huang (danxue.huang at gmail.com)
> >> +# All rights reserved.
> >> +#
> >> +# This file is part of the RTEMS Tools package in 'rtems-tools'.
> >> +#
> >> +# Redistribution and use in source and binary forms, with or without
> >> +# modification, are permitted provided that the following conditions
> are
> >> met:
> >> +#
> >> +# 1. Redistributions of source code must retain the above copyright
> >> notice,
> >> +# this list of conditions and the following disclaimer.
> >> +#
> >> +# 2. Redistributions in binary form must reproduce the above copyright
> >> notice,
> >> +# this list of conditions and the following disclaimer in the
> >> documentation
> >> +# and/or other materials provided with the distribution.
> >> +#
> >> +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> "AS
> >> IS"
> >> +# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
> >> THE
> >> +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
> >> PURPOSE
> >> +# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
> CONTRIBUTORS
> >> BE
> >> +# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
> >> +# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
> >> +# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
> >> BUSINESS
> >> +# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
> IN
> >> +# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
> OTHERWISE)
> >> +# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
> OF
> >> THE
> >> +# POSSIBILITY OF SUCH DAMAGE.
> >> +#
> >> +
> >> +
> >> +import argparse
> >> +import re
> >> +import rst
> >> +
> >> +
> >> +def is_command(s):
> >> + """
> >> + A command is a single word of at least 3 characters, all uppercase
> >> + :param s: input string
> >> + :return: True if s is a single command, otherwise False
> >> + """
> >> + return True if re.match('^[A-Z_]{3,}\s*$', s) else False
> >> +
> >> +
> >> +def extract_comments(content):
> >> + """
> >> + Extract content inside of /* */
> >> + :param content: input file content
> >> + :return: extracted comments
> >> + """
> >> + comments = ''
> >> + comments_match = re.match('/\*(\*(?!/)|[^*])*\*/', content)
> >> + if comments_match:
> >> + wrapped_comments = comments_match.group()
> >> + comments =
> >> wrapped_comments.lstrip('/*').rstrip('*/').lstrip().rstrip()
> >> + return comments
> >> +
> >> +
> >> +def extract_command_and_text(content):
> >> + """
> >> + Extract command and text from input string content
> >> + :param content: input string containing command and text
> >> + :return: a tuple containing command and text
> >> + """
> >> + command = ''
> >> + text = ''
> >> + command_text_dict = {}
> >> + for line in content.splitlines():
> >> + if is_command(line):
> >> + if command and text:
> >> + command_text_dict[command] = text
> >> + command = line.rstrip()
> >> + text = ''
> >> + else:
> >> + text += line + '\n'
> >> + return command_text_dict
> >> +
> >> +
> >> +def generate_rst(command_text_dict):
> >> + rst_str = ''
> >> + for command, text in command_text_dict.items():
> >> + rst_str += rst.get_command_processor(command)(command, text)
> >> + return rst_str
> >> +
> >> +
> >> +def get_parser():
> >> + parser = argparse.ArgumentParser(
> >> + description='Convert newlib style markup to rst markup'
> >> + )
> >> + parser.add_argument(
> >> + '-s',
> >> + '--source_file_dir',
> >> + type=str,
> >> + help='Source file directory with newlib style comments',
> >> + )
> >> + parser.add_argument(
> >> + '-d',
> >> + '--dest_file_dir',
> >> + type=str,
> >> + help='Destination directory for converted rst markup file',
> >> + )
> >> + return parser
> >> +
> >> +
> >> +def main(source_file_dir, dest_file_dir):
> >> + with open(source_file_dir, 'r') as source_file, open(dest_file_dir,
> >> 'w') as dest_file:
> >> + file_content = source_file.read()
> >> +
> >> + # Get comments inside of /* */
> >> + comments = extract_comments(file_content)
> >> +
> >> + # Parse comments
> >> + command_text_dict = extract_command_and_text(comments)
> >> +
> >> + # Process text based on command type
> >> + rst_str = generate_rst(command_text_dict)
> >> +
> >> + dest_file.write(rst_str)
> >> +
> >> +
> >> +if __name__ == '__main__':
> >> + args = get_parser().parse_args()
> >> + main(args.source_file_dir, args.dest_file_dir)
> >> diff --git a/rst.py b/rst.py
> >> new file mode 100644
> >> index 0000000..2531f46
> >> --- /dev/null
> >> +++ b/rst.py
> >> @@ -0,0 +1,104 @@
> >> +#
> >> +# RTEMS Tools Project (http://www.rtems.org/)
> >> +# Copyright 2018 Danxue Huang (danxue.huang at gmail.com)
> >> +# All rights reserved.
> >> +#
> >> +# This file is part of the RTEMS Tools package in 'rtems-tools'.
> >> +#
> >> +# Redistribution and use in source and binary forms, with or without
> >> +# modification, are permitted provided that the following conditions
> are
> >> met:
> >> +#
> >> +# 1. Redistributions of source code must retain the above copyright
> >> notice,
> >> +# this list of conditions and the following disclaimer.
> >> +#
> >> +# 2. Redistributions in binary form must reproduce the above copyright
> >> notice,
> >> +# this list of conditions and the following disclaimer in the
> >> documentation
> >> +# and/or other materials provided with the distribution.
> >> +#
> >> +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> "AS
> >> IS"
> >> +# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
> >> THE
> >> +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
> >> PURPOSE
> >> +# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
> CONTRIBUTORS
> >> BE
> >> +# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
> >> +# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
> >> +# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
> >> BUSINESS
> >> +# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
> IN
> >> +# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
> OTHERWISE)
> >> +# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
> OF
> >> THE
> >> +# POSSIBILITY OF SUCH DAMAGE.
> >> +#
> >> +
> >> +
> >> +import re
> >> +
> >> +
> >> +def register_processor_command():
> >> + return {
> >> + 'FUNCTION': gen_function_summary,
> >> + 'SYNOPSIS': gen_code_block,
> >> + 'ANSI_SYNOPSIS': gen_code_block,
> >> + 'TRAD_SYNOPSIS': gen_code_block,
> >> + 'TYPEDEF': gen_code_block,
> >> + 'DESCRIPTION': gen_custom_directives,
> >> + 'INDEX': gen_custom_directives,
> >> + 'RETURNS': gen_custom_directives,
> >> + 'PORTABILITY': gen_custom_directives,
> >> + 'NOTES': gen_custom_directives,
> >> + 'ERRORS': gen_custom_directives,
> >> + 'BUGS': gen_custom_directives,
> >> + 'WARNINGS': gen_custom_directives,
> >> + 'QUICKREF': gen_nothing,
> >> + 'MATHREF': gen_nothing,
> >> + 'NEWPAGE': gen_nothing,
> >> + 'START': gen_nothing,
> >> + 'END': gen_nothing,
> >> + 'SEEALSO': gen_custom_directives,
> >> + }
> >> +
> >> +
> >> +def get_command_processor(command):
> >> + command_processor_dict = register_processor_command()
> >> + if command in command_processor_dict:
> >> + return command_processor_dict[command]
> >> + else:
> >> + print('Command {c} is not recognized, skip
> it'.format(c=command))
> >> + return gen_nothing
> >> +
> >> +
> >> +def gen_function_summary(command, text):
> >> + function_name = extract_function_name(text)
> >> + summary = extract_summary(text)
> >> +
> >> + title = '.. {f}:\n\n{f} - {s}\n'.format(
> >> + f=function_name,
> >> + s=summary.capitalize()
> >> + )
> >> + dashes = '-' * len(text) + '\n'
> >> + func_name_index = '.. index:: {name}\n'.format(name=function_name)
> >> + summary_index = '.. index:: {summary}\n\n'.format(summary=summary)
> >> + return title + dashes + func_name_index + summary_index
> >> +
> >> +
> >> +def extract_function_name(text):
> >> + function_name = ''
> >> + function_name_match = re.match('\s+(<<(>(?!>)|[^>])*>>)', text)
> >> + if function_name_match:
> >> + function_name =
> >> function_name_match.group(1).lstrip('<<').rstrip('>>')
> >> + return function_name
> >> +
> >> +
> >> +def extract_summary(text):
> >> + return text.split('---')[-1].rstrip()
> >> +
> >> +
> >> +def gen_code_block(command, text):
> >> + return '**{c}:**\n\n.. code-block:: c\n\n{t}\n\n'.format(c =
> command,
> >> t=text)
> >> +
> >> +
> >> +def gen_nothing(command, text):
> >> + return '\n\n'
> >> +
> >> +
> >> +def gen_custom_directives(command, text):
> >> + return '**{c}:**\n\n{t}\n\n'.format(c=command, t=text)
> >> +
> >> diff --git a/strcmp.rst b/strcmp.rst
> >> new file mode 100644
> >> index 0000000..c1dc543
> >> --- /dev/null
> >> +++ b/strcmp.rst
> >> @@ -0,0 +1,47 @@
> >> +.. strcmp:
> >> +
> >> +strcmp - Character string compare
> >> +-----------------------------------------
> >> +.. index:: strcmp
> >> +.. index:: character string compare
> >> +
> >> +**INDEX:**
> >> +
> >> + strcmp
> >> +
> >> +
> >> +
> >> +**SYNOPSIS:**
> >> +
> >> +.. code-block:: c
> >> +
> >> + #include <string.h>
> >> + int strcmp(const char *<[a]>, const char *<[b]>);
> >> +
> >> +
> >> +
> >> +**DESCRIPTION:**
> >> +
> >> + <<strcmp>> compares the string at <[a]> to
> >> + the string at <[b]>.
> >> +
> >> +
> >> +
> >> +**RETURNS:**
> >> +
> >> + If <<*<[a]>>> sorts lexicographically after <<*<[b]>>>,
> >> + <<strcmp>> returns a number greater than zero. If the two
> >> + strings match, <<strcmp>> returns zero. If <<*<[a]>>>
> >> + sorts lexicographically before <<*<[b]>>>, <<strcmp>> returns a
> >> + number less than zero.
> >> +
> >> +
> >> +
> >> +**PORTABILITY:**
> >> +
> >> +<<strcmp>> is ANSI C.
> >> +
> >> +<<strcmp>> requires no supporting OS subroutines.
> >> +
> >> +
> >> +
> >> --
> >> 2.7.4
> >>
> >
> >
> > _______________________________________________
> > devel mailing list
> > devel at rtems.org
> > http://lists.rtems.org/mailman/listinfo/devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20180802/61419dad/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: text and rst.JPG
Type: image/jpeg
Size: 78630 bytes
Desc: not available
URL: <http://lists.rtems.org/pipermail/devel/attachments/20180802/61419dad/attachment-0002.jpe>
More information about the devel
mailing list