GSOC project: #4334 Replace Mongoose with Civetweb

Fri Apr 2 04:13:38 UTC 2021

Respected Sir,

1. I successfully build and installed the tool suit in rtems6
(../source-builder/sb-set-builder --prefix=$HOME/quick-start/rtems/6
devel/qemu.bset) , this time all executable related to qemu
(qemu-system-arm) were installed in 6/bin but the gcc file
(arm-rtems6-gcc) and other arm-rtems- related files weren't installed
which is causing problem in building xilinx_zynq_a9_qemu bsp  (Could
not find the program ['arm-rtems6-gcc'] ). This problem didn't occur
and I was able to build the bsp when I installed tools in rtems5 ( $
../source-builder/sb-set-builder --prefix="$HOME/quick-start/rtems/5"
5/rtems-arm ) but there qemu-system-arm and other qemu- files weren't
installed . I don't why is the problem is occurring but very
enthusiast in solving this if you can please provide a little help.

2, I had drawn my conclusion regarding placement of civetweb package
after reading some of the initial discussion regarding replacing
"mongoose with civetweb" on devel mailing list
(https://lists.rtems.org/pipermail/devel/2016-April/042581.html) but
after going through entire discussion I am getting confused weather to
provide it as an external 3rd party package via the RSB (which I now
think is a good idea) or place it in a separate repository. Also after
reading through discussion I think one other task of project would be
to 1.make build configuration for civetweb as it uses make Cmake and
Makefile build system where as rtems now uses waf build system and 2.
to integrate tests provided by civetweb into rtems.
Sir, I would be highly grateful if you can provide some documentation
to get more information regarding  the current architecture and build
configuration of web-server in rtems as most of the discussions are
pretty old and I wanted to start writing proposal for this project.
thanks , ayushman

On Wed, Mar 31, 2021 at 9:28 PM Gedare Bloom <gedare at rtems.org> wrote:
>
> On Wed, Mar 31, 2021 at 3:18 AM Ayushman Mishra
> <ayushvidushi01 at gmail.com> wrote:
> >
> > Sir,
> > I am trying to integrate mongoose web-server in simple rtems
> > application (https://devel.rtems.org/wiki/Developer/Mongoose_Web_Server),
> > for that I have to configure networking in rtems. I am using
> > xilinx_zynq_a9_qemu for simulation bsp and following steps given here
> > https://git.rtems.org/rtems-libbsd/tree/README.md but after doing
> > ./waf install , i was not able to find qemu-system-arm in any
> > directory , I did sudo apt-get install qemu but then also
> > qemu-system-arm was not found . I discussed in discord channel and
> > started using rsb recipe devel/qemu4 (
> > $ ../source-builder/sb-set-builder --prefix=<TOOLCHAIN_DIR>
> > devel/qemu4.bset ) present in branch 5 of rsb ( since master branch
> > doesn't have qemu4.bset ) but after building for few minutes it gave
> > BUILD FAILED error
>
> bare/devel/qemu.bset should get you a working version on qemu v5, I
> would guess that should work.
>
> > ( Last lines of error report:
> >  ../../glib-2.46.2/gio/gdbusauth.c: In function '_g_dbus_auth_run_server':
> > ../../glib-2.46.2/gio/gdbusauth.c:1295:11: error: '%s' directive
> > argument is null [-Werror=format-overflow=]
> >  1295 |           debug_print ("SERVER: WaitingForBegin, read '%s'", line);
> >       |           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > cc1: some warnings being treated as errors
> > make[4]: *** [Makefile:3624: libgio_2_0_la-gdbusauth.lo] Error 1
> > make[4]: *** Waiting for unfinished jobs....
> > make[4]: Leaving directory
> > '/home/ayush/quick-start1/src/rsb/rtems/build/glib-2.46.2-x86_64-linux-gnu-1/build/gio'
> > make[3]: *** [Makefile:4094: all-recursive] Error 1
> > make[3]: Leaving directory
> > '/home/ayush/quick-start1/src/rsb/rtems/build/glib-2.46.2-x86_64-linux-gnu-1/build/gio'
> > make[2]: *** [Makefile:1861: all] Error 2
> > make[2]: Leaving directory
> > '/home/ayush/quick-start1/src/rsb/rtems/build/glib-2.46.2-x86_64-linux-gnu-1/build/gio'
> > make[1]: *** [Makefile:1212: all-recursive] Error 1
> > make[1]: Leaving directory
> > '/home/ayush/quick-start1/src/rsb/rtems/build/glib-2.46.2-x86_64-linux-gnu-1/build'
> > make: *** [Makefile:854: all] Error 2
> > shell cmd failed: /bin/sh -ex
> > /home/ayush/quick-start1/src/rsb/rtems/build/glib-2.46.2-x86_64-linux-gnu-1/do-build
> > error: building glib-2.46.2-x86_64-linux-gnu-1
> > )
> I have the same problem trying to build qemu4, technically this is a
> problem building glib-2.46.2.
>
> > I think i am really stuck here and I would be highly obliged if you
> > can please provide some guidance in configuring networking and
> > integrating mongoose with simple rtems application.
> >
> > Also I think creating a separate repository like rtems-civetweb would
> > make the tree more clearer to understand and easy to build and install
> > , since integrating it with rsb like a package may sometimes cause
> > build failed due to bugs in rsb.
>
> This reasoning doesn't make too much sense to me. Do you have a
> technical reason for a separate repo?
>
> > thanks , Ayushman
> >
> > On Tue, Mar 30, 2021 at 12:16 PM Christian MAUDERER
> > <christian.mauderer at embedded-brains.de> wrote:
> > >
> > > Hello Ayushman, hello Gedare,
> > >
> > > Am 29.03.21 um 18:50 schrieb Gedare Bloom:
> > > > CC: Christian
> > > >
> > > > On Mon, Mar 29, 2021 at 10:34 AM Ayushman Mishra
> > > > <ayushvidushi01 at gmail.com> wrote:
> > > >>
> > > >> Sir ,
> > > >> It would be very helpful to know about potential mentors of (Replace mongoose with civetweb) https://devel.rtems.org/ticket/4334 this project.
> > > >>
> > > > I think Christian is the most likely potential mentor.
> > > >
> > >
> > > Possible for the civetweb part. I have done quite a bit with mongoose or
> > > later civetweb. So most likely I should at least help on this project.
> > > But I don't think that building civetweb will be the critical part here.
> > > That is quite straightforward.
> > >
> > > The more tricky part of the project will be to find a way to make the
> > > configuration options available to the users. That means either creating
> > > a repo (similar to for example rtems-littlevgl) or some integration into
> > > RSB (no idea how to add options there). We should try to get someone
> > > with more experience in that area into this project too.
> > >
> > > >> On Mon, Mar 29, 2021, 9:39 PM Gedare Bloom <gedare at rtems.org> wrote:
> > > >>>
> > > >>> On Mon, Mar 29, 2021 at 9:49 AM Ayushman Mishra
> > > >>> <ayushvidushi01 at gmail.com> wrote:
> > > >>>>
> > > >>>> I have installed libbsd package (made
> > > >>>> RTEMS_POSIX_API=True,RTEMS_NETWORKING=True in config.ini file to
> > > >>>> enable POSIX and networking) but I am still confused about the work
> > > >>> That doesn't look right. RTEMS_NETWORKING=True enables the legacy
> > > >>> network stack. If you want to build libbsd, you need to check out
> > > >>> https://git.rtems.org/rtems-libbsd/
> > > >>>
> > > >>>> already done in this project in past. Also after interacting on
> > > >>>> discord I think completely replacing  mongoose with another web-server
> > > >>>> is not a good option since it may rise some user complaints who are
> > > >>>> already using old system and may find it difficult to instantly switch
> > > >>>> over to new system , instead civetweb can be added as an extra new
> > > >>>> web-server where user has a choice to either use old mongoose or the
> > > >>>> new web server both having all dependencies and functionalities
> > > >>>> available.
> > > >>>>
> > > >>> No, we will replace mghttpd with civetweb. Anyone who still wants the
> > > >>> frozen version of mghttpd can revive it, but we want to replace it
> > > >>> moving forward.
> > >
> > > Agreed. We maybe can leave some hints where to get the current version
> > > of mghttpd (basically a commit number, a path and current build
> > > options). If someone really want's to continue using mghttpd, he can
> > > then copy the files into his application build. But the old version we
> > > have is now a lot behind the master and most likely has a lot of bugs
> > > and quite some security holes.
> > >
> > > >>>
> > > >>>> Also I still don't know whether the project #4334 is long enough for
> > > >>>> summer of code project and after going through docs I think
> > > >>> There are other activities involved in getting civetweb to work well
> > > >>> with RTEMS. It should be sufficient, but you would need to flesh out
> > > >>> the proposal details with mentor assistance. To be clear, it will be
> > > >>> C/network programming.
> > > >>>
> > >
> > > Civetweb also has a lot of different features. If building with basic
> > > features is not enough for the project, we can pile different
> > > configurations on top of it ;-)
> > >
> > > I think that we should target at least two configurations:
> > >
> > > 1. Minimal functionality as http server.
> > >
> > > 2. Integration with OpenSSL (most likely the version from libbsd) so
> > > that we can use https.
> > >
> > > Having two configurations makes it necessary to create a build
> > > environment that allows having configurations. That is a good start for
> > > further extensions.
> > >
> > > Best regards
> > >
> > > Christian
> > >
> > > >>>> https://devel.rtems.org/ticket/4272 is also a very interesting project
> > > >>>> since now I think I know a little bit about BSPs and I am good in
> > > >>>> python and know shell-scripting. I would be very grateful to know more
> > > >>>> about this project.
> > > >>> You should start a new thread to discuss this project. I thought
> > > >>> someone else may have inquired about this, maybe it was you? I didn't
> > > >>> see any response yet, but probably because it was buried in other
> > > >>> content (like this comment) so likely the potential mentor didn't even
> > > >>> notice it, especially since all you mention here is the ticket number.
> > > >>>
> > > >>> Gedare
> > > >>>
> > > >>>> Thanks.
> > > >>>>
> > > >>>> On Sat, Mar 27, 2021 at 5:36 PM Ayushman Mishra
> > > >>>> <ayushvidushi01 at gmail.com> wrote:
> > > >>>>>
> > > >>>>> Greetings to all the respected mentors,
> > > >>>>> 1. I saw there has been a lot of discussion regarding replacing
> > > >>>>> mongoose with civetweb (
> > > >>>>> https://lists.rtems.org/pipermail/devel/2016-April/014661.html ).
> > > >>>>> I think the basic outline of the project is like this
> > > >>>>> a. Completely removing mongoose and replace it with civetweb and make
> > > >>>>> it configurable in rsb
> > > >>>>> b. check the parameters and options available in civetweb and make it
> > > >>>>> usable for users in rtems.
> > > >>>>> c. making test-cases for civetweb (should be similar to mongoose)
> > > >>>>> d. documentation of using civetweb and web-server in general.
> > > >>>>> I would be very grateful to know what is the actual current status of
> > > >>>>> this project and the coding involved in it enough for a summer of code
> > > >>>>> project or some extra things should be added in it.
> > > >>>>>
> > > >>>>> Also,
> > > >>>>> 2. I have build bsp on rtems6 and a simple application is working
> > > >>>>> correct, but while trying to build libbsd package in rtems6
> > > >>>>> (../source-builder/sb-set-builder --prefix=$HOME/quick-start/rtems/6 \
> > > >>>>>    --host=sparc-rtems6 --with-rtems-bsp=erc32 6/rtems-libbsd ) I am
> > > >>>>> constantly getting build failed
> > > >>>>> RTEMS Source Builder - Set Builder, 6 (ade089253e70)
> > > >>>>> Build Set: 6/rtems-libbsd
> > > >>>>> config: tools/rtems-libbsd-6.cfg
> > > >>>>> error: rtems-bsp.cfg:155: invalid %if operator:  -mcpu=cypress
> > > >>>>> -I/home/ayush/quick-start/rtems/6/sparc-rtems6/erc32/lib/include ==
> > > >>>>> Build FAILED
> > > >>>>> Build Set: Time 0:00:00.045392
> > > >>>>> Build FAILED
> > > >>>>>
> > > >>>>> I also tried building all packages for bsp erc32
> > > >>>>> (../source-builder/sb-set-builder --prefix=$HOME/quick-start/rtems/6 \
> > > >>>>>      --with-rtems-tests=yes bsps/erc32) but after building for more
> > > >>>>> than an hour it also gave build failed error.
> > > >>>>> building: sparc-rtems6-kernel-erc32-1
> > > >>>>> error: building sparc-rtems6-kernel-erc32-1
> > > >>>>> Build FAILED
> > > >>>>>    See error report: rsb-report-sparc-rtems6-kernel-erc32-1.txt
> > > >>>>> error: building sparc-rtems6-kernel-erc32-1
> > > >>>>> Build Set: Time 0:23:14.195494
> > > >>>>> error: building sparc-rtems6-kernel-erc32-1
> > > >>>>> Build Set: Time 3:55:05.826612
> > > >>>>> Build FAILED
> > > >>>>>
> > > >>>>> Thanks, Ayushman
> > > >>>>>
> > > >>>>> On Mon, Mar 22, 2021 at 6:37 AM Ayushman Mishra
> > > >>>>> <ayushvidushi01 at gmail.com> wrote:
> > > >>>>>>
> > > >>>>>> Thanks a lot for help and information , actually i am trying to setup
> > > >>>>>> mongoose on simple rtems application
> > > >>>>>> (https://devel.rtems.org/wiki/Developer/Mongoose_Web_Server) and for
> > > >>>>>> that right now I am trying to configure networking and file-system in
> > > >>>>>> the application.
> > > >>>>>>
> > > >>>>>> On Sun, Mar 21, 2021 at 8:54 PM Joel Sherrill <joel at rtems.org> wrote:
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> On Sun, Mar 21, 2021, 4:45 AM Christian Mauderer <oss at c-mauderer.de> wrote:
> > > >>>>>>>>
> > > >>>>>>>> Hello Ayushman,
> > > >>>>>>>>
> > > >>>>>>>> On 21/03/2021 04:15, Ayushman Mishra wrote:
> > > >>>>>>>>> Ayushman
> > > >>>>>>>>> Hello everyone , I am very much interested in taking
> > > >>>>>>>>> https://devel.rtems.org/ticket/4334
> > > >>>>>>>>> as a GSOC 2021 project. I know some basic networking concepts and would like to
> > > >>>>>>>>> learn more about it and how its applied to OS like RTEMS , regarding
> > > >>>>>>>>> this I have some questions.
> > > >>>>>>>>
> > > >>>>>>>> Note that the ticket will be more about integrating civetweb into a
> > > >>>>>>>> RTEMS Source Builder (RSB) recipe and finding a way to make it
> > > >>>>>>>> configurable there. Alternative could be some kind of stand alone repo
> > > >>>>>>>> like for littlevgl.
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> Making the civitweb configure options available to RTEMS user is a key point.
> > > >>>>>>>
> > > >>>>>>> It may make sense to do a repo with waf build system and config.ini that maps to their settings.
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> civetweb builds on RTEMS nearly out of the box. So don't expect too much
> > > >>>>>>>> C-Code.
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> I set it up recently on Linux for embedding in their application. As you turned on options, it had more dependencies. This would have to be managed with RTEMS.
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> I'm not yet sure how much work will be on that ticket. If it is too few
> > > >>>>>>>> for a whole GSoC, you might want to think about reviving the discussion
> > > >>>>>>>> about some useful civetweb parameters (for an embedded system) here:
> > > >>>>>>>>
> > > >>>>>>>>       https://github.com/civetweb/civetweb/pull/297
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> +1
> > > >>>>>>>
> > > >>>>>>> How to present the options to the user and manage their dependencies would be a key part of this.
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>> 1. After building a simple hello world application how and where should i write
> > > >>>>>>>>> configurations of
> > > >>>>>>>>> https://docs.rtems.org/branches/master/networking/using_networking_rtems_app.html
> > > >>>>>>>>> OR  https://docs.rtems.org/branches/master/user/migration/v4_11-to-v5.html#networking
> > > >>>>>>>>> to start using networking stack in RTEMS .
> > > >>>>>>>>
> > > >>>>>>>> The documentation is currently mostly for the legacy stack. Please
> > > >>>>>>>> ignore most of that. You should focus on a BSP that uses libbsd. The
> > > >>>>>>>> legacy stack and it's documentation will be removed from the main repos
> > > >>>>>>>> soon.
> > > >>>>>>>>
> > > >>>>>>>>> Also I think a simple shell has to be spawned to use networking modules in RTEMS
> > > >>>>>>>>> and for getting it this
> > > >>>>>>>>> https://docs.rtems.org/branches/master/shell/configuration_and_init.html#attached-to-a-serial-port
> > > >>>>>>>>> I think could be a simple method and for doing so ( like executing
> > > >>>>>>>>> rtems_shell_init with parameters )
> > > >>>>>>>>> do i have to run the specific test in testsuite or re-run the application
> > > >>>>>>>>
> > > >>>>>>>> There is a "mghttpd01" test in rtems-libbsd which is used for the
> > > >>>>>>>> current fixed version of mongoose that is integrated in RTEMS. I would
> > > >>>>>>>> suggest to use that as a starting point.
> > > >>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>> 2. Is there any specific device or bsp needed for running networking
> > > >>>>>>>>> or shell configurations
> > > >>>>>>>>> with file-system in RTEMS
> > > >>>>>>>>
> > > >>>>>>>> Basically you can use every BSP that has network support. A good
> > > >>>>>>>> simulation BSP where you don't need any hardware is xilinx_zynq_a9_qemu:
> > > >>>>>>>>
> > > >>>>>>>> https://docs.rtems.org/branches/master/user/bsps/bsps-arm.html#debugging-with-xilinx-zynq-a9-qemu
> > > >>>>>>>>
> > > >>>>>>>> Best Regards
> > > >>>>>>>>
> > > >>>>>>>> Christian
> > > >>>>>>>>
> > > >>>>>>>>> I would be very thankful if someone can please clarify my doubts and
> > > >>>>>>>>> guide me further
> > > >>>>>>>>> with this project as I am getting lost and don't know how to proceed
> > > >>>>>>>>> after building a simple application.
> > > >>>>>>>>> Thank-you
> > > >>>>>>>>> _______________________________________________
> > > >>>>>>>>> devel mailing list
> > > >>>>>>>>> devel at rtems.org
> > > >>>>>>>>> http://lists.rtems.org/mailman/listinfo/devel
> > > >>>>>>>>>
> > > >>>>>>>> _______________________________________________
> > > >>>>>>>> devel mailing list
> > > >>>>>>>> devel at rtems.org
> > > >>>>>>>> http://lists.rtems.org/mailman/listinfo/devel
> > > >>>> _______________________________________________
> > > >>>> devel mailing list
> > > >>>> devel at rtems.org
> > > >>>> http://lists.rtems.org/mailman/listinfo/devel
> > >
> > > --
> > > --------------------------------------------
> > > embedded brains GmbH
> > > Herr Christian MAUDERER
> > > Dornierstr. 4
> > > 82178 Puchheim
> > > Germany
> > > email: christian.mauderer at embedded-brains.de
> > > phone: +49-89-18 94 741 - 18
> > > fax:   +49-89-18 94 741 - 08
> > >
> > > Registergericht: Amtsgericht München
> > > Registernummer: HRB 157899
> > > Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
> > > Unsere Datenschutzerklärung finden Sie hier:
> > > https://embedded-brains.de/datenschutzerklaerung/