[PATCH 00/13] Replace mongoose with civetweb.
Christian Mauderer
christian.mauderer at embedded-brains.de
Tue Apr 26 12:22:28 UTC 2016
Am 26.04.2016 um 04:51 schrieb Chris Johns:
> On 26/04/2016 07:22, Joel Sherrill wrote:
>> Agreed 100%!
>>
>> NTP has an RSB recipe and there are others for various libraries which
>> should be good examples. Shouldn't take long to do the recipe. Just ask
>> questions rather than getting stuck.
>>
>> On Apr 25, 2016 7:03 PM, "Gedare Bloom" <gedare at rtems.org
>> <mailto:gedare at rtems.org>> wrote:
>>
>> On Mon, Apr 25, 2016 at 11:25 AM, Joel Sherrill <joel at rtems.org
>> <mailto:joel at rtems.org>> wrote:
>> >
>> >
>> > On Mon, Apr 25, 2016 at 4:22 AM, Christian Mauderer
>> > <christian.mauderer at embedded-brains.de
>> <mailto:christian.mauderer at embedded-brains.de>> wrote:
>> >>
>> >> Essentially I agree that it would be nice to build civetweb as an
>> >> external library especially with the different network stacks in
>> mind.
>> >> But there are some points that keep me from doing it:
>> >>
>> >> 1. I have really no Idea what would be necessary to build it
>> as an
>> >> upstream project using RSB. If that are only something like
>> three simple
>> >> steps, it should be possible to squeeze it in the little time
>> budget
>> >> that is left for the project. If I first have to analyze and
>> change a
>> >> lot of RSB, it would be challenging.
>> >>
>> > FWIW I think this can be a secondary goal. Switching over to
>> the real
>> > upstream project with proper licensing is important.
>
> I see both as equally important and once the switch is made we have an
> on going liability in the project to keep the packaging maintained so
> considering it a bit more I suppose the long term issue is actually more
> important to me.
There is still some work left before we can use the upstream project. I
more or less took over most of the patches that we applied to mongoose.
Most of them should just need some clean up and don't really add any
maintenance effort for the civetweb maintainers.
I already contacted the civetweb author regarding two of the patches.
One was a new one (civetweb now uses a timegm function that we don't
have: https://github.com/civetweb/civetweb/pull/298).
The second one was a more complex one that _could_ lead to maintenance
effort: We (to be exact: I) added some paramters to mongoose to
influence stack size and scheduling. The maintainer of civetweb wasn't
really enthusiastic about the way of the integration. He would prefer
another interface. I tried to invite the RTEMS mailinglist into the
discussion but till now I didn't get any responses:
https://lists.rtems.org/pipermail/devel/2016-April/014621.html
https://groups.google.com/forum/#!topic/civetweb/_Mul9PxgpBE
>
>> >
>> > When we do this, we incur some burden of "retraining" users to
>> consider
>> > a webserver an external addon. But that is the direction I think
>> we want
>> > to head with httpd, telnetd, ftpd, etc. So it is a good long term
>> goal.
>
> Agreed.
>
>> >
>> >>
>> >> Chris, I think you know the RSB best: What steps do you think
>> would be
>> >> necessary?
>> >>
>> >
>> > Chris...
>> >
>
> There are a number of examples that show how to do this. Ping me if you
> need specific help. Gedare has highlighted NTP and there is also NetNSMP.
>
Civetweb seems to have multiple build systems. One is a most likely
handwritten Makefile
(https://github.com/civetweb/civetweb/blob/master/Makefile). Another
would be cmake. Is there any experience, which build system would be
easier to integrate? Do we have any other non-automake projects?
>> >>
>> >> 2. Currently there is one test case for mghttpd
>> (libtests/mghttpd01).
>> >> This is one of the few tests that check some networking
>> functions in
>> >> RTEMS.
>
> I would be concerned if the only way we can do network testing is to
> consume a 3prd party package's source into RTEMS. :)
It's not the only test that tests network functions in RTEMS. But as far
as I know, there are not many. If we remove it, it will be one less.
>
>> Further we should not loose the ability to test software
>> when it
>> >> is build with RSB.
>
> I agree.
>
>> How would we handle tests for software in RSB?
>> >>
>> >
>> > That's a Chris question. We should aim to have tests which run
>> with
>> > either network stack for RSB built network services. This is a
>> good
>> > requirement on the plan to moving to separately built network
>> stacks
>> > and services.
>
> Network testing is an open topic as is vertically integrated packages.
> The testing issues exists with libbsd, lwip and the in-tree stack once
> we move out of the source tree. As we flesh out the details of
> vertically integrating packages we need to figure out now to manage
> testing.
>
> What tests does civetweb provide? If the test is important I would
> assume it is important to all users. Can civetweb build the test?
>
Civetweb has a number of tests. But I think that most would not work
with RTEMS without changing them. In RTEMS we currently have a
relatively simple testcase that just checks some basic functions.
I don't think that we would be able to run either of them with the
current RSB. And I don't have enough time to integrate a way for testing
into RSB within the current project. So moving civetweb out of RTEMS
currently means that it is not tested any longer with an RTEMS system
(or only if someone does it manually after changing something).
>> >
>> > Having thought about this, I am not opposed to seeing the patches
>> > merged as long as it is clear what is just moving to the new
>> upstream,
>> > what are patches, the patches are submitted upstream and we push
>> > to get them merged.
>> >
>> > I realize you need a practical goal. I would like concurrence
>> from Chris
>> > and Gedare on the short and long term plans. If you can make any
>> > progress on the long term plan, that's good but we should accept
>> > the short term needed improvement.
>> >
>> > Perfection is the enemy of the good enough.
>> >
>> We need to watch out for incurring (new) technical debt. This patch
>> set does not add new debt, but it does expose some that we had not
>> thought about yet. I would certainly encourage spending some of your
>> budget to consider the difficulty to provide the web server as a 3rd
>> party package build, and to open a ticket to track such an idea.
>
> Adding the patch set lowers the threshold to bringing further updates in
> and that makes the change to a separate package appear a higher cost and
> less likely to happen.
>
> We have the httpd code in the RTEMS source tree and in libbsd and we
> have a project in GSoC to add lwip. Do we add another copy into lwip to
> make sure we have equivalent functionality across the stacks and that is
> not even considering network testing? I hope we do not another copy.
>
>> Since
>> we now have examples of other 3rd party packages (e.g. graphics tool
>> kit), I would expect that a web server should not be too hard. Plus,
>> you get the advantage of not tying the civetweb build to the in-tree
>> net stack.
>
> I agree.
>
> Chris
>
>>
>> >>
>> >> Kind regards
>> >>
>> >> Christian Mauderer
>> >>
>> >> Am 23.04.2016 um 15:07 schrieb Joel Sherrill:
>> >> > I am really with Gedare and Chris that it would be better to
>> treat this
>> >> > as an upstream project. Use the RSB and track patches through
>> RTEMS
>> >> > tools.
>> >> >
>> >> > It would be a good case to push the model of a single network
>> service
>> >> > supporting both stacks and begin the process of removing
>> networking code
>> >> > from the base RTEMS git repo.
>> >> >
>> >> > It would also push us to figure out how to rest RSB built
>> packages.
>> >> >
>> >> > On Apr 22, 2016 12:05 AM, "Christian Mauderer"
>> >> > <christian.mauderer at embedded-brains.de
>> <mailto:christian.mauderer at embedded-brains.de>
>> >> > <mailto:christian.mauderer at embedded-brains.de
>> <mailto:christian.mauderer at embedded-brains.de>>> wrote:
>> >> >
>> >> > Yes that's right. 05/13 just adds the unchanged sources
>> from
>> >> > civetweb.
>> >> > Beneath civetweb.c and civetweb.h it also adds
>> handle_form.inl and
>> >> > md5.inl. The last two files are included into civetweb.c.
>> According
>> >> > to
>> >> > the documentation "The *INL* file extension represents
>> code that is
>> >> > statically included inline in a source file."
>> >> >
>> >> > And yes: The patch didn't get through. I have got a replay
>> that
>> >> > "Your
>> >> > message to devel awaits moderator approval". The
>> civetweb.c file is
>> >> > over
>> >> > 300k and the mailing list seems to have a maximum of 256k.
>> I hoped
>> >> > that
>> >> > one of the mail admins would approve the patch soon.
>> >> >
>> >> > Am 21.04.2016 um 22:49 schrieb Gedare Bloom:
>> >> > > I think patch 05/13 probably adds civetweb.c and
>> civetweb.h? But
>> >> > it
>> >> > > did not come through the mailman.
>> >> > >
>> >> > > On Thu, Apr 21, 2016 at 4:46 PM, Gedare Bloom
>> <gedare at rtems.org <mailto:gedare at rtems.org>
>> >> > <mailto:gedare at rtems.org <mailto:gedare at rtems.org>>> wrote:
>> >> > >> P.S. might be worth it to open a ticket related to
>> civetweb and
>> >> > >> #update it from these patches.
>> >> > >>
>> >> > >> On Thu, Apr 21, 2016 at 4:45 PM, Gedare Bloom
>> <gedare at rtems.org <mailto:gedare at rtems.org>
>> >> > <mailto:gedare at rtems.org <mailto:gedare at rtems.org>>> wrote:
>> >> > >>> Is the plan eventually to be able to use the upstream
>> civetweb?
>> >> > or to
>> >> > >>> track it with our own copy?
>> >> > >>>
>> >> > >>> On Thu, Apr 21, 2016 at 4:49 AM, Christian Mauderer
>> >> > >>> <christian.mauderer at embedded-brains.de
>> <mailto:christian.mauderer at embedded-brains.de>
>> >> > <mailto:christian.mauderer at embedded-brains.de
>> <mailto:christian.mauderer at embedded-brains.de>>> wrote:
>> >> > >>>> This patch series replaces the mongoose webserver by
>> its still
>> >> > MIT
>> >> > >>>> licensed fork civetweb.
>> >> > >>>>
>> >> > >>>> Please note that I try to get some (currently two)
>> of the
>> >> > patches
>> >> > >>>> directly into civetweb too. But I think that it might
>> need some
>> >> > time and
>> >> > >>>> adaption till they are accepted. So I thought that
>> adding them
>> >> > to RTEMS
>> >> > >>>> would still make sense as a working interim solution.
>> >> > >>>>
>> >> > >>>> _______________________________________________
>> >> > >>>> devel mailing list
>> >> > >>>> devel at rtems.org <mailto:devel at rtems.org>
>> <mailto:devel at rtems.org <mailto:devel at rtems.org>>
>> >> > >>>> http://lists.rtems.org/mailman/listinfo/devel
>> >> >
>> >> > --
>> >> > --------------------------------------------
>> >> > embedded brains GmbH
>> >> > Christian Mauderer
>> >> > Dornierstr. 4
>> >> > D-82178 Puchheim
>> >> > Germany
>> >> > email: christian.mauderer at embedded-brains.de
>> <mailto:christian.mauderer at embedded-brains.de>
>> >> > <mailto:christian.mauderer at embedded-brains.de
>> <mailto:christian.mauderer at embedded-brains.de>>
>> >> > Phone: +49-89-18 94 741 - 18
>> <tel:%2B49-89-18%2094%20741%20-%2018>
>> <tel:%2B49-89-18%2094%20741%20-%2018>
>> >> > Fax: +49-89-18 94 741 - 08
>> <tel:%2B49-89-18%2094%20741%20-%2008>
>> <tel:%2B49-89-18%2094%20741%20-%2008>
>> >> > PGP: Public key available on request.
>> >> >
>> >> > Diese Nachricht ist keine geschäftliche Mitteilung im
>> Sinne des
>> >> > EHUG.
>> >> > _______________________________________________
>> >> > devel mailing list
>> >> > devel at rtems.org <mailto:devel at rtems.org>
>> <mailto:devel at rtems.org <mailto:devel at rtems.org>>
>> >> > http://lists.rtems.org/mailman/listinfo/devel
>> >> >
>> >>
>> >> --
>> >> --------------------------------------------
>> >> embedded brains GmbH
>> >> Christian Mauderer
>> >> Dornierstr. 4
>> >> D-82178 Puchheim
>> >> Germany
>> >> email: christian.mauderer at embedded-brains.de
>> <mailto:christian.mauderer at embedded-brains.de>
>> >> Phone: +49-89-18 94 741 - 18
>> <tel:%2B49-89-18%2094%20741%20-%2018>
>> >> Fax: +49-89-18 94 741 - 08 <tel:%2B49-89-18%2094%20741%20-%2008>
>> >> PGP: Public key available on request.
>> >>
>> >> Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des
>> EHUG.
>> >
>> >
>>
--
--------------------------------------------
embedded brains GmbH
Christian Mauderer
Dornierstr. 4
D-82178 Puchheim
Germany
email: christian.mauderer at embedded-brains.de
Phone: +49-89-18 94 741 - 18
Fax: +49-89-18 94 741 - 08
PGP: Public key available on request.
Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.
More information about the devel
mailing list