<div dir="auto"><div><br><br><div class="gmail_quote"><div dir="ltr">On Wed, 30 May 2018, 22:42 Cillian O'Donnell, <<a href="mailto:cpodonnell8@gmail.com">cpodonnell8@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto"><div><br><br><div class="gmail_quote"><div dir="ltr">On Wed, 30 May 2018, 16:58 Gedare Bloom, <<a href="mailto:gedare@rtems.org" target="_blank" rel="noreferrer">gedare@rtems.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Wed, May 30, 2018 at 11:32 AM, Vijay Kumar Banerjee<br>
<<a href="mailto:vijaykumar9597@gmail.com" rel="noreferrer noreferrer" target="_blank">vijaykumar9597@gmail.com</a>> wrote:<br>
> On 30 May 2018 at 20:18, Gedare Bloom <<a href="mailto:gedare@rtems.org" rel="noreferrer noreferrer" target="_blank">gedare@rtems.org</a>> wrote:<br>
>><br>
>> Hello Vijay,<br>
>><br>
>> Do you expect/need an answer to something in here?<br>
>><br>
>> gedare<br>
>><br>
> Hello,<br>
><br>
> I wanted to know if there were any plans on how covoar<br>
> should store the reports when running for multisets.<br>
> Earlier it used to be done by the coverage script,<br>
> after the recent changes covoar can process multi sets.<br>
><br>
> I think, covoar should store the reports into separate directories<br>
> for each set . eg. score/ , rtems/ . As the coverage can already read<br>
> from separate directories.<br></blockquote></div></div><div dir="auto"><br></div><div dir="auto">Sorry I thought all questions had been answered here. I think you have the right idea. Each set should be a sub-directory of coverage directory.</div><div dir="auto"><br></div><div dir="auto">By the way I tested your changes and everything seems fine. Still have to do a review of coverage.py to see how close we are to merging.</div></div></blockquote></div></div><div dir="auto">I have squashed everything and sent a patch to devel@. This will make it easy to go through all the changes. Please have a look.</div><div dir="auto"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto"><div dir="auto"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
><br>
> Any advice on how should it be approached ?<br>
<br>
It would help me if you could draw/write a diagram of what the<br>
filesystem tree might look like with separate directories, and what<br>
will go in each subdirectory.<br>
<br>
I don't have enough context to give any useful advice on this question.<br>
<br>
-Gedare<br>
<br>
>><br>
>> On Wed, May 30, 2018 at 10:29 AM, Vijay Kumar Banerjee<br>
>> <<a href="mailto:vijaykumar9597@gmail.com" rel="noreferrer noreferrer" target="_blank">vijaykumar9597@gmail.com</a>> wrote:<br>
>> > On 30 May 2018 at 00:54, Joel Sherrill <<a href="mailto:joel@rtems.org" rel="noreferrer noreferrer" target="_blank">joel@rtems.org</a>> wrote:<br>
>> >><br>
>> >><br>
>> >><br>
>> >> On Tue, May 29, 2018 at 11:27 AM, Vijay Kumar Banerjee<br>
>> >> <<a href="mailto:vijaykumar9597@gmail.com" rel="noreferrer noreferrer" target="_blank">vijaykumar9597@gmail.com</a>> wrote:<br>
>> >>><br>
>> >>><br>
>> >>><br>
>> >>> On Tue, 29 May 2018, 20:10 Joel Sherrill, <<a href="mailto:joel@rtems.org" rel="noreferrer noreferrer" target="_blank">joel@rtems.org</a>> wrote:<br>
>> >>>><br>
>> >>>><br>
>> >>>><br>
>> >>>> On Mon, May 28, 2018 at 11:08 PM, Chris Johns <<a href="mailto:chrisj@rtems.org" rel="noreferrer noreferrer" target="_blank">chrisj@rtems.org</a>><br>
>> >>>> wrote:<br>
>> >>>>><br>
>> >>>>> On 29/5/18 4:26 am, Joel Sherrill wrote:<br>
>> >>>>> > On Mon, May 28, 2018 at 5:43 AM, Vijay Kumar Banerjee<br>
>> >>>>> > <<a href="mailto:vijaykumar9597@gmail.com" rel="noreferrer noreferrer" target="_blank">vijaykumar9597@gmail.com</a><br>
>> >>>>> > <mailto:<a href="mailto:vijaykumar9597@gmail.com" rel="noreferrer noreferrer" target="_blank">vijaykumar9597@gmail.com</a>>> wrote:<br>
>> >>>>> ><br>
>> >>>>> > Hello,<br>
>> >>>>> ><br>
>> >>>>> > The coverage reports are now showing results.<br>
>> >>>>> > The current branch that holds all the changes is<br>
>> >>>>> > the cov-tester-support branch in my forked repo<br>
>> >>>>> ><br>
>> >>>>> ><br>
>> >>>>> > <a href="https://github.com/thelunatic/rtems-tools/tree/cov-tester-support" rel="noreferrer noreferrer noreferrer" target="_blank">https://github.com/thelunatic/rtems-tools/tree/cov-tester-support</a><br>
>> >>>>> ><br>
>> >>>>> ><br>
>> >>>>> > <<a href="https://github.com/thelunatic/rtems-tools/tree/cov-tester-support" rel="noreferrer noreferrer noreferrer" target="_blank">https://github.com/thelunatic/rtems-tools/tree/cov-tester-support</a>><br>
>> >>>>> ><br>
>> >>>>> > (Please have a look into the code and test it.)<br>
>> >>>>> ><br>
>> >>>>> > It is close to merging (hopefully). These are the issues<br>
>> >>>>> > that would need a fix before it can be merged :<br>
>> >>>>> ><br>
>> >>>>> > 1. I have added some #FIXME in the code (have a look)<br>
>> >>>>> > in coverage script. I have set the value of the targe to<br>
>> >>>>> > be<br>
>> >>>>> > sparc-rtems5, which makes it limited to sparc-rtems5 only.<br>
>> >>>>> > We<br>
>> >>>>> > can<br>
>> >>>>> > include the target in the bsp ini file, That would<br>
>> >>>>> > be a quick fix for this.<br>
>> >>>>> ><br>
>> >>>>> ><br>
>> >>>>> > Yes. This needs to be fixed.<br>
>> >>>>> ><br>
>> >>>>> > My hack to add 4 in ObjdumpProcessor.cc needs to be addressed<br>
>> >>>>> > also.<br>
>> >>>>> > I am thinking of adding a method to Target_*.cc to ask for the<br>
>> >>>>> > size<br>
>> >>>>> > of an<br>
>> >>>>> > instruction.<br>
>> >>>>> > Then pass it the last instruction. That way we can throw on other<br>
>> >>>>> > architectures for<br>
>> >>>>> > now. If Chris solves this with his changes before we try another<br>
>> >>>>> > architecture,<br>
>> >>>>> > great.<br>
>> >>>>> > If not, it will be easy to fix.<br>
>> >>>>><br>
>> >>>>> What is the overall requirement?<br>
>> >>>><br>
>> >>>><br>
>> >>>> To know the ending address of the function.<br>
>> >>>><br>
>> >>>> Technically there are three pieces of information:<br>
>> >>>><br>
>> >>>> + start address<br>
>> >>>> + end address<br>
>> >>>> + size<br>
>> >>>><br>
>> >>>> If you know two of those, you can compute the third.<br>
>> >>>><br>
>> >>>> I don't care if this comes from DWARF, ELF, or parsing the<br>
>> >>>> disassembly.<br>
>> >>>> It just needs to be reliable.<br>
>> >>>><br>
>> >>>> And.. I am not proposing my solution as permanent. Just to keep us<br>
>> >>>> moving. You want to change to an internal disassembler (which<br>
>> >>>> would also need to have the source interspersed) and DWARF. So<br>
>> >>>> this code would go away then.<br>
>> >>>><br>
>> >>>>><br>
>> >>>>><br>
>> >>>>> What defines the function and so size are attempting to get coverage<br>
>> >>>>> of? What if<br>
>> >>>>> that function calls an inline function and that function is inlined?<br>
>> >>>>> What if<br>
>> >>>>> that inlined function calls another inlined function?<br>
>> >>>><br>
>> >>>><br>
>> >>>> Then it is all inlined. It is done consistently now. I have never<br>
>> >>>> seen a<br>
>> >>>> case<br>
>> >>>> where two instances of a method differed as the assembly level. [1]<br>
>> >>>><br>
>> >>>> The actual body of the inlined method is evaluated at each expansion<br>
>> >>>> point.<br>
>> >>>> That is why a few years ago, I pushed hard to reduce the complexity<br>
>> >>>> of<br>
>> >>>> inline methods because we got test patch explosion when an inlined<br>
>> >>>> method<br>
>> >>>> included complex conditionals.<br>
>> >>>><br>
>> >>>> Similarly, I think it would be helpful to coverage and verification<br>
>> >>>> efforts to<br>
>> >>>> follow the **shudder** MISRA rule which want you to write simple<br>
>> >>>> conditional<br>
>> >>>> expressions rather than compound ones. I have taken to writing code<br>
>> >>>> this<br>
>> >>>> way as much as possible. But haven't pushed it as a coding rule.<br>
>> >>>><br>
>> >>>> if (a) {<br>
>> >>>> if (b) {<br>
>> >>>> do x;<br>
>> >>>> }<br>
>> >>>> }<br>
>> >>>><br>
>> >>>> Versus<br>
>> >>>> if (a && b) {<br>
>> >>>> do x;<br>
>> >>>> }<br>
>> >>>><br>
>> >>>> The reason is that the first is easier to analyse coverage on.<br>
>> >>>><br>
>> >>>> [1] We both expect LTO could change this.<br>
>> >>>><br>
>> >>>> [2] ESA did specifically mention this one though. Also in general<br>
>> >>>> terms,<br>
>> >>>> an RTEMS Project response to MISRA rules. Which ones we follow,<br>
>> >>>> reject, etc.. But I refuse to adopt hard rules which can't be<br>
>> >>>> enforced<br>
>> >>>> by free open source tools.<br>
>> >>>><br>
>> >>>>><br>
>> >>>>><br>
>> >>>>> The DWARF data provides details about the PC low and PC high of what<br>
>> >>>>> is<br>
>> >>>>> called<br>
>> >>>>> concrete functions and then it provides the details about inlines.<br>
>> >>>><br>
>> >>>><br>
>> >>>> We don't (currently) report on the inlines as independent methods.<br>
>> >>>><br>
>> >>>>><br>
>> >>>>><br>
>> >>>>> ><br>
>> >>>>> > 2. coverage used to feed ini file for each symbol to covoar<br>
>> >>>>> > in a loop and store the result in a separate directory<br>
>> >>>>> > for each symbol . Which is needed no more needed as<br>
>> >>>>> > covoar can now process multi sets from a<br>
>> >>>>> > single ini file. I think the results from covoar should<br>
>> >>>>> > be<br>
>> >>>>> > store in a separate directory for each symbol<br>
>> >>>>> > example :- score/<br>
>> >>>>> ><br>
>> >>>>> ><br>
>> >>>>> > A bit of history will help here. Originally covoar was run against<br>
>> >>>>> > a<br>
>> >>>>> > single set of<br>
>> >>>>> > code by the scripting framework. We would do coverage on either<br>
>> >>>>> > "core<br>
>> >>>>> > parts"<br>
>> >>>>> > or "developmental" (e.g. nearly all non-networked code). The<br>
>> >>>>> > optimization was<br>
>> >>>>> > either at -O2 or -Os. So there were four coverage variants.<br>
>> >>>>> ><br>
>> >>>>> > Turned out that when we added something to "all", the percentage<br>
>> >>>>> > would drop<br>
>> >>>>> > and reflect badly on the rest of the code. I remember adding the<br>
>> >>>>> > dosfs and<br>
>> >>>>> > the coverage dropped almost 20 percent.<br>
>> >>>>> ><br>
>> >>>>> > This led to the idea that we should report on a per<br>
>> >>>>> > directory/subsystem basis.<br>
>> >>>>> > The score, rtems, posix, sapi, and libcsupport should have high<br>
>> >>>>> > coverage now<br>
>> >>>>> > and the reports should reflect that independent of whether the<br>
>> >>>>> > dosfs<br>
>> >>>>> > needs a<br>
>> >>>>> > lot more tests.<br>
>> >>>>> ><br>
>> >>>>> > Before each directory/subsystem required a completely separate run<br>
>> >>>>> > of<br>
>> >>>>> > covoar.<br>
>> >>>>> > If we are headed to a solution where one analysis run of covoar<br>
>> >>>>> > generates different<br>
>> >>>>> > reports, that should speed up the processing time a lot!<br>
>> >>>>> ><br>
>> >>>>> > The other issue is what should the top directory look like/contain<br>
>> >>>>> > when a single<br>
>> >>>>> > run produces multiple subsystem reports. Seems like we would need<br>
>> >>>>> > at<br>
>> >>>>> > least a<br>
>> >>>>> > top level html and text file.<br>
>> >>>>> ><br>
>> >>>>> ><br>
>> >>>>> ><br>
>> >>>>> > 3. currently we are using the leon3-qemu-cov as the bsp.<br>
>> >>>>> > Are we going to have two ini file for each bsp ? ( one<br>
>> >>>>> > without coverage<br>
>> >>>>> > and one with coverage support)<br>
>> >>>>> ><br>
>> >>>>> > Earlier the approach was to include a section 'coverage'<br>
>> >>>>> > to the bsp ini to put the values we needed for coverage.<br>
>> >>>>> > I think that approach would be more "convenient" for the user.<br>
>> >>>>> ><br>
>> >>>>> ><br>
>> >>>>> > This was something Chris suggested. I think it was to avoid adding<br>
>> >>>>> > to<br>
>> >>>>> > the bsp ini file until the code was closer to merging.<br>
>> >>>>> ><br>
>> >>>>> > Chris.. what do you think? Long term, a section would be nice.<br>
>> >>>>> ><br>
>> >>>>><br>
>> >>>>> Sorry I cannot remember without looking it up and I am currently<br>
>> >>>>> buried<br>
>> >>>>> in<br>
>> >>>>> family issues.<br>
>> >>>><br>
>> >>>><br>
>> >>>> OK. Having the Python scripting loop over the sets or covoar looping<br>
>> >>>> over them<br>
>> >>>> is an open issue. Historically, looping over desired symbol sets was<br>
>> >>>> outside<br>
>> >>>> covoar. So looping inside covoar may take some work.<br>
>> >>><br>
>> >>><br>
>> >>> covoar can already loop over the<br>
>> >>> sets it seems, which is implemented<br>
>> >>> in DesiredSymbols. But it stores all the<br>
>> >>> reports generated from into the same directory.<br>
>> >><br>
>> >><br>
>> >> If there is an index that makes its possible to navigate through the<br>
>> >> different "subsystems", then that's the key thing. You don't want<br>
>> >> to think score is poorly covered due to dosfs.<br>
>> >><br>
>> >>><br>
>> >>><br>
>> >>> P.S:Sorry for the previous mail with no message<br>
>> >>>><br>
>> >>>><br>
>> >>>> --joel<br>
>> >>>>><br>
>> >>>>><br>
>> >>>>> Chris<br>
>> >>>><br>
>> >>>><br>
>> >><br>
>> ><br>
>> ><br>
>> > _______________________________________________<br>
>> > devel mailing list<br>
>> > <a href="mailto:devel@rtems.org" rel="noreferrer noreferrer" target="_blank">devel@rtems.org</a><br>
>> > <a href="http://lists.rtems.org/mailman/listinfo/devel" rel="noreferrer noreferrer noreferrer" target="_blank">http://lists.rtems.org/mailman/listinfo/devel</a><br>
><br>
><br>
_______________________________________________<br>
devel mailing list<br>
<a href="mailto:devel@rtems.org" rel="noreferrer noreferrer" target="_blank">devel@rtems.org</a><br>
<a href="http://lists.rtems.org/mailman/listinfo/devel" rel="noreferrer noreferrer noreferrer" target="_blank">http://lists.rtems.org/mailman/listinfo/devel</a><br>
</blockquote></div></div></div>
</blockquote></div></div></div>