Benchmark Apps for RTEMS

[Gedare Bloom]

On Thu, Apr 18, 2013 at 3:06 PM, Rempel, Cynthia
<cynt6007 at vandals.uidaho.edu> wrote:
>>________________________________________
>>From: gedare at gwmail.gwu.edu [gedare at gwmail.gwu.edu] on behalf of Gedare Bloom [gedare at rtems.org]
>>Sent: Thursday, April 18, 2013 6:12 AM
>>To: Rempel, Cynthia
>>Cc: Vivek Krishnamurthy; rtems-devel at rtems.org
>>Subject: Re: Benchmark Apps for RTEMS
>>
>>I would like to see a build system approach for the benchkit that is
>>easily maintained and flexible, probably something Waf-based.
> I agree it should be maintainable, and the current version of examples-v2 is maintainable...
>
> I think we need to pause and think about the implications of pushing something Waf-based on students...
>
> My understanding is we have been trying for months to get examples-v2 to be Waf-based, and because of the header problems, we have been unsuccessful so far.  Waf is a step up in complexity over a standard Makefile; if we go that route, we need the examples-v2 with a Waf build system publicly available as a reference first...
>
I don't know about the effort to port examples-v2 personally, so I
don't know what complications may exist. If the installed headers are
holding it up, and also would hold up any Waf-based kit
infrastructure, then the dependency and risk needs to be addressed.
But I would want to hear more detail about the problem before calling
it a blocker.

> Only a select few RTEMS developers know how to use Waf and we don't want this to be an elitist open-source project.  There needs to be a publically available tutorial for how to maintain a Waf-based build-system (I think its inadvisable that only a few people should have keys to the build-system kingdom)...
>
I don't have personal experience with Waf other than using it a couple
of times. I don't see Waf as an elitist project, but actually I think
it may be much more accessible than Make and auto* approaches. Waf has
quite good documentation about the Waf system and about how to build
custom build systems if we need such flexibility.

> I will go along with doing a Waf-based build system, I would like to hear your refutations, but I think it's inadvisable to do so until the above issues are addressed.
>
>>I have
>>ported a few benchmarks, e.g. mibench [1], in the past but not in a
>>flexible way--I integrated them directly to the RTEMS build for my
>>ease of use. I think this benchkit can be a benefit by providing an
>>external (to rtems.git) repository of benchmark packages that can be
>>individually maintained, perhaps in a git submodule, but uniformly
>>compiled and executed with the same build system.
>
> I agree it should be a separate git, I would suggest it remains being built independently, so we can build the benchkit with older versions of RTEMS as well. We could always tie the benchkit into RSB.
>
>>The work here is to define the "kit" framework, which is of general
>>interest for porting other software to RTEMS, and to apply the kit
>>framework to some benchmark suites.
> There is a separate "make package" open project to address this issue... but the student didn't select that one.
>
>>Aside from the build system, there
>>may be other issues to handle such as dealing with IO and
>>filing/fixing bugs that arise when running these benchmarks.
>>[1] http://code.google.com/p/rtemssparc64/source/browse/#svn%2Ftrunk%2Frtems%2Frtemscvs%2Ftestsuites%2Fmibench
> After selecting benchmarks this is what the majority of coding will entail...
>
>
>
>