GSOC project: #4334 Replace Mongoose with Civetweb

Wed Mar 31 15:58:19 UTC 2021

On Wed, Mar 31, 2021 at 3:18 AM Ayushman Mishra
<ayushvidushi01 at gmail.com> wrote:
>
> Sir,
> I am trying to integrate mongoose web-server in simple rtems
> application (https://devel.rtems.org/wiki/Developer/Mongoose_Web_Server),
> for that I have to configure networking in rtems. I am using
> xilinx_zynq_a9_qemu for simulation bsp and following steps given here
> https://git.rtems.org/rtems-libbsd/tree/README.md but after doing
> ./waf install , i was not able to find qemu-system-arm in any
> directory , I did sudo apt-get install qemu but then also
> qemu-system-arm was not found . I discussed in discord channel and
> started using rsb recipe devel/qemu4 (
> $ ../source-builder/sb-set-builder --prefix=<TOOLCHAIN_DIR>
> devel/qemu4.bset ) present in branch 5 of rsb ( since master branch
> doesn't have qemu4.bset ) but after building for few minutes it gave
> BUILD FAILED error

bare/devel/qemu.bset should get you a working version on qemu v5, I
would guess that should work.

> ( Last lines of error report:
>  ../../glib-2.46.2/gio/gdbusauth.c: In function '_g_dbus_auth_run_server':
> ../../glib-2.46.2/gio/gdbusauth.c:1295:11: error: '%s' directive
> argument is null [-Werror=format-overflow=]
>  1295 |           debug_print ("SERVER: WaitingForBegin, read '%s'", line);
>       |           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> cc1: some warnings being treated as errors
> make[4]: *** [Makefile:3624: libgio_2_0_la-gdbusauth.lo] Error 1
> make[4]: *** Waiting for unfinished jobs....
> make[4]: Leaving directory
> '/home/ayush/quick-start1/src/rsb/rtems/build/glib-2.46.2-x86_64-linux-gnu-1/build/gio'
> make[3]: *** [Makefile:4094: all-recursive] Error 1
> make[3]: Leaving directory
> '/home/ayush/quick-start1/src/rsb/rtems/build/glib-2.46.2-x86_64-linux-gnu-1/build/gio'
> make[2]: *** [Makefile:1861: all] Error 2
> make[2]: Leaving directory
> '/home/ayush/quick-start1/src/rsb/rtems/build/glib-2.46.2-x86_64-linux-gnu-1/build/gio'
> make[1]: *** [Makefile:1212: all-recursive] Error 1
> make[1]: Leaving directory
> '/home/ayush/quick-start1/src/rsb/rtems/build/glib-2.46.2-x86_64-linux-gnu-1/build'
> make: *** [Makefile:854: all] Error 2
> shell cmd failed: /bin/sh -ex
> /home/ayush/quick-start1/src/rsb/rtems/build/glib-2.46.2-x86_64-linux-gnu-1/do-build
> error: building glib-2.46.2-x86_64-linux-gnu-1
> )
I have the same problem trying to build qemu4, technically this is a
problem building glib-2.46.2.

> I think i am really stuck here and I would be highly obliged if you
> can please provide some guidance in configuring networking and
> integrating mongoose with simple rtems application.
>
> Also I think creating a separate repository like rtems-civetweb would
> make the tree more clearer to understand and easy to build and install
> , since integrating it with rsb like a package may sometimes cause
> build failed due to bugs in rsb.

This reasoning doesn't make too much sense to me. Do you have a
technical reason for a separate repo?

> thanks , Ayushman
>
> On Tue, Mar 30, 2021 at 12:16 PM Christian MAUDERER
> <christian.mauderer at embedded-brains.de> wrote:
> >
> > Hello Ayushman, hello Gedare,
> >
> > Am 29.03.21 um 18:50 schrieb Gedare Bloom:
> > > CC: Christian
> > >
> > > On Mon, Mar 29, 2021 at 10:34 AM Ayushman Mishra
> > > <ayushvidushi01 at gmail.com> wrote:
> > >>
> > >> Sir ,
> > >> It would be very helpful to know about potential mentors of (Replace mongoose with civetweb) https://devel.rtems.org/ticket/4334 this project.
> > >>
> > > I think Christian is the most likely potential mentor.
> > >
> >
> > Possible for the civetweb part. I have done quite a bit with mongoose or
> > later civetweb. So most likely I should at least help on this project.
> > But I don't think that building civetweb will be the critical part here.
> > That is quite straightforward.
> >
> > The more tricky part of the project will be to find a way to make the
> > configuration options available to the users. That means either creating
> > a repo (similar to for example rtems-littlevgl) or some integration into
> > RSB (no idea how to add options there). We should try to get someone
> > with more experience in that area into this project too.
> >
> > >> On Mon, Mar 29, 2021, 9:39 PM Gedare Bloom <gedare at rtems.org> wrote:
> > >>>
> > >>> On Mon, Mar 29, 2021 at 9:49 AM Ayushman Mishra
> > >>> <ayushvidushi01 at gmail.com> wrote:
> > >>>>
> > >>>> I have installed libbsd package (made
> > >>>> RTEMS_POSIX_API=True,RTEMS_NETWORKING=True in config.ini file to
> > >>>> enable POSIX and networking) but I am still confused about the work
> > >>> That doesn't look right. RTEMS_NETWORKING=True enables the legacy
> > >>> network stack. If you want to build libbsd, you need to check out
> > >>> https://git.rtems.org/rtems-libbsd/
> > >>>
> > >>>> already done in this project in past. Also after interacting on
> > >>>> discord I think completely replacing  mongoose with another web-server
> > >>>> is not a good option since it may rise some user complaints who are
> > >>>> already using old system and may find it difficult to instantly switch
> > >>>> over to new system , instead civetweb can be added as an extra new
> > >>>> web-server where user has a choice to either use old mongoose or the
> > >>>> new web server both having all dependencies and functionalities
> > >>>> available.
> > >>>>
> > >>> No, we will replace mghttpd with civetweb. Anyone who still wants the
> > >>> frozen version of mghttpd can revive it, but we want to replace it
> > >>> moving forward.
> >
> > Agreed. We maybe can leave some hints where to get the current version
> > of mghttpd (basically a commit number, a path and current build
> > options). If someone really want's to continue using mghttpd, he can
> > then copy the files into his application build. But the old version we
> > have is now a lot behind the master and most likely has a lot of bugs
> > and quite some security holes.
> >
> > >>>
> > >>>> Also I still don't know whether the project #4334 is long enough for
> > >>>> summer of code project and after going through docs I think
> > >>> There are other activities involved in getting civetweb to work well
> > >>> with RTEMS. It should be sufficient, but you would need to flesh out
> > >>> the proposal details with mentor assistance. To be clear, it will be
> > >>> C/network programming.
> > >>>
> >
> > Civetweb also has a lot of different features. If building with basic
> > features is not enough for the project, we can pile different
> > configurations on top of it ;-)
> >
> > I think that we should target at least two configurations:
> >
> > 1. Minimal functionality as http server.
> >
> > 2. Integration with OpenSSL (most likely the version from libbsd) so
> > that we can use https.
> >
> > Having two configurations makes it necessary to create a build
> > environment that allows having configurations. That is a good start for
> > further extensions.
> >
> > Best regards
> >
> > Christian
> >
> > >>>> https://devel.rtems.org/ticket/4272 is also a very interesting project
> > >>>> since now I think I know a little bit about BSPs and I am good in
> > >>>> python and know shell-scripting. I would be very grateful to know more
> > >>>> about this project.
> > >>> You should start a new thread to discuss this project. I thought
> > >>> someone else may have inquired about this, maybe it was you? I didn't
> > >>> see any response yet, but probably because it was buried in other
> > >>> content (like this comment) so likely the potential mentor didn't even
> > >>> notice it, especially since all you mention here is the ticket number.
> > >>>
> > >>> Gedare
> > >>>
> > >>>> Thanks.
> > >>>>
> > >>>> On Sat, Mar 27, 2021 at 5:36 PM Ayushman Mishra
> > >>>> <ayushvidushi01 at gmail.com> wrote:
> > >>>>>
> > >>>>> Greetings to all the respected mentors,
> > >>>>> 1. I saw there has been a lot of discussion regarding replacing
> > >>>>> mongoose with civetweb (
> > >>>>> https://lists.rtems.org/pipermail/devel/2016-April/014661.html ).
> > >>>>> I think the basic outline of the project is like this
> > >>>>> a. Completely removing mongoose and replace it with civetweb and make
> > >>>>> it configurable in rsb
> > >>>>> b. check the parameters and options available in civetweb and make it
> > >>>>> usable for users in rtems.
> > >>>>> c. making test-cases for civetweb (should be similar to mongoose)
> > >>>>> d. documentation of using civetweb and web-server in general.
> > >>>>> I would be very grateful to know what is the actual current status of
> > >>>>> this project and the coding involved in it enough for a summer of code
> > >>>>> project or some extra things should be added in it.
> > >>>>>
> > >>>>> Also,
> > >>>>> 2. I have build bsp on rtems6 and a simple application is working
> > >>>>> correct, but while trying to build libbsd package in rtems6
> > >>>>> (../source-builder/sb-set-builder --prefix=$HOME/quick-start/rtems/6 \
> > >>>>>    --host=sparc-rtems6 --with-rtems-bsp=erc32 6/rtems-libbsd ) I am
> > >>>>> constantly getting build failed
> > >>>>> RTEMS Source Builder - Set Builder, 6 (ade089253e70)
> > >>>>> Build Set: 6/rtems-libbsd
> > >>>>> config: tools/rtems-libbsd-6.cfg
> > >>>>> error: rtems-bsp.cfg:155: invalid %if operator:  -mcpu=cypress
> > >>>>> -I/home/ayush/quick-start/rtems/6/sparc-rtems6/erc32/lib/include ==
> > >>>>> Build FAILED
> > >>>>> Build Set: Time 0:00:00.045392
> > >>>>> Build FAILED
> > >>>>>
> > >>>>> I also tried building all packages for bsp erc32
> > >>>>> (../source-builder/sb-set-builder --prefix=$HOME/quick-start/rtems/6 \
> > >>>>>      --with-rtems-tests=yes bsps/erc32) but after building for more
> > >>>>> than an hour it also gave build failed error.
> > >>>>> building: sparc-rtems6-kernel-erc32-1
> > >>>>> error: building sparc-rtems6-kernel-erc32-1
> > >>>>> Build FAILED
> > >>>>>    See error report: rsb-report-sparc-rtems6-kernel-erc32-1.txt
> > >>>>> error: building sparc-rtems6-kernel-erc32-1
> > >>>>> Build Set: Time 0:23:14.195494
> > >>>>> error: building sparc-rtems6-kernel-erc32-1
> > >>>>> Build Set: Time 3:55:05.826612
> > >>>>> Build FAILED
> > >>>>>
> > >>>>> Thanks, Ayushman
> > >>>>>
> > >>>>> On Mon, Mar 22, 2021 at 6:37 AM Ayushman Mishra
> > >>>>> <ayushvidushi01 at gmail.com> wrote:
> > >>>>>>
> > >>>>>> Thanks a lot for help and information , actually i am trying to setup
> > >>>>>> mongoose on simple rtems application
> > >>>>>> (https://devel.rtems.org/wiki/Developer/Mongoose_Web_Server) and for
> > >>>>>> that right now I am trying to configure networking and file-system in
> > >>>>>> the application.
> > >>>>>>
> > >>>>>> On Sun, Mar 21, 2021 at 8:54 PM Joel Sherrill <joel at rtems.org> wrote:
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> On Sun, Mar 21, 2021, 4:45 AM Christian Mauderer <oss at c-mauderer.de> wrote:
> > >>>>>>>>
> > >>>>>>>> Hello Ayushman,
> > >>>>>>>>
> > >>>>>>>> On 21/03/2021 04:15, Ayushman Mishra wrote:
> > >>>>>>>>> Ayushman
> > >>>>>>>>> Hello everyone , I am very much interested in taking
> > >>>>>>>>> https://devel.rtems.org/ticket/4334
> > >>>>>>>>> as a GSOC 2021 project. I know some basic networking concepts and would like to
> > >>>>>>>>> learn more about it and how its applied to OS like RTEMS , regarding
> > >>>>>>>>> this I have some questions.
> > >>>>>>>>
> > >>>>>>>> Note that the ticket will be more about integrating civetweb into a
> > >>>>>>>> RTEMS Source Builder (RSB) recipe and finding a way to make it
> > >>>>>>>> configurable there. Alternative could be some kind of stand alone repo
> > >>>>>>>> like for littlevgl.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> Making the civitweb configure options available to RTEMS user is a key point.
> > >>>>>>>
> > >>>>>>> It may make sense to do a repo with waf build system and config.ini that maps to their settings.
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>> civetweb builds on RTEMS nearly out of the box. So don't expect too much
> > >>>>>>>> C-Code.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> I set it up recently on Linux for embedding in their application. As you turned on options, it had more dependencies. This would have to be managed with RTEMS.
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>> I'm not yet sure how much work will be on that ticket. If it is too few
> > >>>>>>>> for a whole GSoC, you might want to think about reviving the discussion
> > >>>>>>>> about some useful civetweb parameters (for an embedded system) here:
> > >>>>>>>>
> > >>>>>>>>       https://github.com/civetweb/civetweb/pull/297
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> +1
> > >>>>>>>
> > >>>>>>> How to present the options to the user and manage their dependencies would be a key part of this.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> 1. After building a simple hello world application how and where should i write
> > >>>>>>>>> configurations of
> > >>>>>>>>> https://docs.rtems.org/branches/master/networking/using_networking_rtems_app.html
> > >>>>>>>>> OR  https://docs.rtems.org/branches/master/user/migration/v4_11-to-v5.html#networking
> > >>>>>>>>> to start using networking stack in RTEMS .
> > >>>>>>>>
> > >>>>>>>> The documentation is currently mostly for the legacy stack. Please
> > >>>>>>>> ignore most of that. You should focus on a BSP that uses libbsd. The
> > >>>>>>>> legacy stack and it's documentation will be removed from the main repos
> > >>>>>>>> soon.
> > >>>>>>>>
> > >>>>>>>>> Also I think a simple shell has to be spawned to use networking modules in RTEMS
> > >>>>>>>>> and for getting it this
> > >>>>>>>>> https://docs.rtems.org/branches/master/shell/configuration_and_init.html#attached-to-a-serial-port
> > >>>>>>>>> I think could be a simple method and for doing so ( like executing
> > >>>>>>>>> rtems_shell_init with parameters )
> > >>>>>>>>> do i have to run the specific test in testsuite or re-run the application
> > >>>>>>>>
> > >>>>>>>> There is a "mghttpd01" test in rtems-libbsd which is used for the
> > >>>>>>>> current fixed version of mongoose that is integrated in RTEMS. I would
> > >>>>>>>> suggest to use that as a starting point.
> > >>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> 2. Is there any specific device or bsp needed for running networking
> > >>>>>>>>> or shell configurations
> > >>>>>>>>> with file-system in RTEMS
> > >>>>>>>>
> > >>>>>>>> Basically you can use every BSP that has network support. A good
> > >>>>>>>> simulation BSP where you don't need any hardware is xilinx_zynq_a9_qemu:
> > >>>>>>>>
> > >>>>>>>> https://docs.rtems.org/branches/master/user/bsps/bsps-arm.html#debugging-with-xilinx-zynq-a9-qemu
> > >>>>>>>>
> > >>>>>>>> Best Regards
> > >>>>>>>>
> > >>>>>>>> Christian
> > >>>>>>>>
> > >>>>>>>>> I would be very thankful if someone can please clarify my doubts and
> > >>>>>>>>> guide me further
> > >>>>>>>>> with this project as I am getting lost and don't know how to proceed
> > >>>>>>>>> after building a simple application.
> > >>>>>>>>> Thank-you
> > >>>>>>>>> _______________________________________________
> > >>>>>>>>> devel mailing list
> > >>>>>>>>> devel at rtems.org
> > >>>>>>>>> http://lists.rtems.org/mailman/listinfo/devel
> > >>>>>>>>>
> > >>>>>>>> _______________________________________________
> > >>>>>>>> devel mailing list
> > >>>>>>>> devel at rtems.org
> > >>>>>>>> http://lists.rtems.org/mailman/listinfo/devel
> > >>>> _______________________________________________
> > >>>> devel mailing list
> > >>>> devel at rtems.org
> > >>>> http://lists.rtems.org/mailman/listinfo/devel
> >
> > --
> > --------------------------------------------
> > embedded brains GmbH
> > Herr Christian MAUDERER
> > Dornierstr. 4
> > 82178 Puchheim
> > Germany
> > email: christian.mauderer at embedded-brains.de
> > phone: +49-89-18 94 741 - 18
> > fax:   +49-89-18 94 741 - 08
> >
> > Registergericht: Amtsgericht München
> > Registernummer: HRB 157899
> > Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
> > Unsere Datenschutzerklärung finden Sie hier:
> > https://embedded-brains.de/datenschutzerklaerung/