GSoC 2017 -- initial proof

Sun Apr 2 17:33:39 UTC 2017

Hi Andy,

Joel and Gedare pointed me to the following:

https://devel.rtems.org/wiki/Developer/Projects/Open/PythonCoverageReporting
https://devel.rtems.org/ticket/2261
http://kmiesowicz.blogspot.in/p/esa-socis-2014.html

Last link is to the work done by a previous student.
Hope this helps.
My proposal does not take covoar into consideration. Though, I have
planned to produce xml reports from rtems-tester that can be
integrated into coverage testing. Also, I have planned to use this xml
data to get plots (for visualizing results) as a stretch goal (you can
look in the "Future Improvements" Section in my proposal).

Regards,
Tanu Hari Dixit.

On Sun, Apr 2, 2017 at 6:50 PM, Gedare Bloom <gedare at rtems.org> wrote:
> Joel's e-mail client messed up my reply thread. ;) Andy, have a read
> through https://gedare.github.io/pdf/FelShe15A.pdf if you haven't
> found it already.
>
> Getting qemu-couverture to build via RSB and then integrating the
> coverage scripts into rtems-test, would be a good start to a project.
> There is a student (Tanu) who has proposed rtems-test improvements, so
> you two would need to coordinate efforts for this piece to avoid too
> much duplicated work/waste. Improvements to the reports etc are also
> welcome, and then of course using the reports to guide code
> refactoring to improve coverage is somewhat open-ended.
>
> -Gedare
>
> On Sat, Apr 1, 2017 at 9:34 PM, Joel Sherrill <joel at rtems.org> wrote:
>>
>>
>> On Apr 1, 2017 7:59 PM, "Andy MacGregor" <amacgregor.2018 at comcast.net>
>> wrote:
>>
>> On 03/31/2017 09:52 PM, Gedare Bloom wrote:
>>
>> Don't forget the deadline is Monday to submit your Final
>> PDF and proof of enrollment. The "Final" PDF can be submitted multiple
>> times, so go ahead and submit it when you finish a first draft. Add
>> your draft proposal to the tracking table at
>> https://devel.rtems.org/wiki/GSoC/2017 also, to get some possible
>> feedback.
>>
>>
>> Thanks for the welcome! (I realize I am late to the GSOC application table).
>>
>> I'm set on improving code coverage of the testsuite, but I found some
>> questions as I'm starting to look into how its done.
>>
>> The link on the wiki to the current code status is currently broken, so I
>> tried to generate a sample coverage report for 4.12 using ./do_coverage,
>> ./run_coverage and ./coverage_cron as suggested on the wiki here but without
>> any luck. Is there an up to date coverage report available?
>> I modified the VERSIONS-COVERAGE file like the wiki said, and once that
>> didn't work dug around in the script but fix it myself quickly.
>>
>> Could updating these coverage analysis scripts be a viable component of a
>> GSOC proposal?
>>
>>
>> Yes. In fact, the idea was to rework the coverage scripts in shell to be
>> part of the rtems-tester package and in Python. There is some work on this
>> by a previous student which you can find or one of us can dig out. Starting
>> with it, making it work across multiple BSPs, independent of any hard coded
>> paths, improving the output (we did not get far enough that I recall
>> worrying about the quality of the output), etc could be a fair amount of
>> work.
>>
>> Also we have privately discussed switching the RSB qemu upstream source to
>> GNATCoverage at GitHub. This is a coverage and embedded focused downstream
>> which includes the coverage trace output I used for the qemu support in
>> covoar. I'm
>>
>> In addition, I'd like to actually make some improvements the code coverage
>> itself.
>> The most recent information I could find was these bar graphs from 2009.
>> If these graphs reflect the current coverage state, I think that improving
>> coverage for the i386/pc386,
>> would be interesting because there is room for improvement in both the
>> Baseline and Developmental groups (if these coverage statistics are up to
>> date).
>>
>>
>> It doesn't matter which BSP you choose for coverage since the code and tests
>> are the same.
>>
>> This also reminds me of the other improvements we had wanted. We wanted to
>> switch from Baseline and developmental to a more directory oriented approach
>> for the reports. For example, the reports should be at a granularity of say
>> the Dos file system or shell as reports. The way reports are generated now
>> it results in a skewed view when a large body of code is added to the
>> developmental set. Adding the Dos filesystem resulted in a 15-20% drop in
>> the coverage reported in the Developmental set. That is a bad way to report
>> and misleading. It detracts from the directories which have near 100%
>> coverage.
>>
>>
>> Does anyone know of a good reason to choose another BSP over i386/pc386?
>>
>>
>> Users fly the SPARC leon3 and not the pc386. :)
>>
>>
>> I've just posted a draft of my proposal to the GSOC page.
>> If there's any feedback I'll be sure to update my proposal in time.
>>
>>
>> Dinner time here. Hopefully my feedback makes sense and you can incorporate
>> it.
>>
>>
>>
>>
>> -Andy MacGregor
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> devel mailing list
>> devel at rtems.org
>> http://lists.rtems.org/mailman/listinfo/devel
>>
>>
>>
>> _______________________________________________
>> devel mailing list
>> devel at rtems.org
>> http://lists.rtems.org/mailman/listinfo/devel
> _______________________________________________
> devel mailing list
> devel at rtems.org
> http://lists.rtems.org/mailman/listinfo/devel