Rhealstone detailed benchmark

[Joel Sherrill]

On Mon, May 29, 2017 at 3:50 PM, Gedare Bloom <gedare at rtems.org> wrote:

> On Mon, May 29, 2017 at 12:35 PM, Andreas Kölbl
> <andreas.koelbl at st.oth-regensburg.de> wrote:
> > Am 28.05.2017 um 18:24 schrieb Joel Sherrill:
> >> On Sun, May 28, 2017 at 6:43 AM, Andreas Kölbl <
> >> andreas.koelbl at st.oth-regensburg.de> wrote:
> >>
> >>> Hi guys,
> >>>
> >>> I've tried measuring the real time capability of a BSP using the
> >>> Rhealstone benchmark. This benchmark only tells about the average time
> >>> spent for e.g. task switching.
> >>> Is there interest in a more fine-grained Rhealstone benchmark, which
> >>> prints the Worst Case Execution Time (WCET) or the deviation of the
> >>> measured values?
> >>> Has there already been an attempt to modify Rhealstone in order to get
> >>> the WCET from each benchmark?
> >>>
> >> Rhealstone was a proposal from someone at Intel in 1989 that appeared
> >> in Dr Dobbs Journal.
> >>
> >> http://www.drdobbs.com/article/print?articleId=
> 184408081&siteSectionName=
> >>
> >> There was a follow up article a year later with some code which I didn't
> >> remember. I am not sure how closely the code written for RTEMS follows
> >> that code.
> >>
> >> http://www.drdobbs.com/article/print?articleId=
> 184408332&siteSectionName=cpp
> >>
> >> AFAIK there is no "standard" source for this benchmark -- just the
> >> definitions
> >> and some odd code for a few RTOSes.
> >>
> >> We would welcome a review of the code against the original articles
> >> and the code they published as well as improvements in the numbers
> >> gathered and their reporting. Statistical information and WCET would
> >> be valuable if the code accurately reflects the intended measurement.
> > I will try this for each benchmark and respond if I were successful.
> > The source of the benchmarks (drdobbs) seems to be similar to the ones
> > used in RTEMS.
> > So maybe there is a reason, like to get a more efficient implementation
> > (with caches) or more comparable results when measuring the hardware
> > this way?
>
> From what I remember from reading the original article, the Rhealstone
> is meant to measure the overhead of usual real-time operating system
> features.
>

Yep. In theory, all RTOSes provide some common services and
share common characteristics. This was intended to define standard
measurements. I think the originator was part of the old iRMX group
at Intel.

The original "tmtests"cases  came from published benchmarks from
pSOS+ (resold by Motorola as VMEExec). Motorola had proposed their
API as the RTEID standard which evolved to ORKID. The original
Classic API calls were based on that proposed standard from the
VMEBus Industry Trade Association (VITA). There is a copy of all
the versions of the documents I could find on the ftp site:

ftp://ftp.rtems.org/pub/rtems/people/joel/RTEID-ORKID/

At one point, I thought these would be useful for deriving the original
RTEMS requirements. Then I read them again and realized that
pSOS+ had really proposed their user manual as a standard. It
doesn't read like a standard at all.

--joel


>
> >>
> >> One thing I have noticed in benchmarks over the years is that they
> >> may want to measure say blocking time of a semaphore with a
> >> set of N threads. But they accidentally include the code associated
> >> with the first execution of each thread. So you have to be careful
> >> to not include unintended overhead.
> >>
> >> Sorry for the long answer but if you are interested in these benchmarks,
> >> you are encouraged to help improve them.
> > Sure ;)
> >
> > --
> > Andreas Kölbl
> >>
> >> --joel
> >>
> >>
> >>> --
> >>> Andreas Kölbl
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> users mailing list
> >>> users at rtems.org
> >>> http://lists.rtems.org/mailman/listinfo/users
> >
> >
> >
> > _______________________________________________
> > users mailing list
> > users at rtems.org
> > http://lists.rtems.org/mailman/listinfo/users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/users/attachments/20170530/2614aa67/attachment-0002.html>