Fwd: RTEMS pthreads performance (was RTEMS vs otherrealtimeOperating Systems ..)

[Joel Sherrill]

Till Straumann wrote:
> 
> Joel Sherrill wrote:
> >
> > Till Straumann wrote:
> >
> >>Joel Sherrill wrote:
> >>
> >>
> >>>Till Straumann wrote:
> >>>
> >>>
> >>>
> >>>>Kamen Penev wrote:
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>>Hi!
> >>>>>
> >>>>>Looking at your work comparing RTEMS, RTLinux and vxWorks, I am wondering
> >>>>>about the unusually poor RTEMS+pthreads result for context switching on a
> >>>>>loaded system. Has this been explained (better yet fixed)by the RTEMS
> >>>>>development team?
> >>>>>
> >>>>>
> >>>>
> >>>Which RTEMS version/target CPU/configuration?
> >>>
> >>
> >>I believe it was ss-20011025 (I can re-run the test later today on a
> >>later version).
> >
> >
> > That is pretty old and I am 100% positive it predates the work
> > I did to improve results on your benchmark.  The ChangeLog entry
> > for that is 2002-07-01.
> >
> >
> >>The target was a 604 series PowerPC on a Motorola MVME23xx board.
> >
> >
> > Do you know what the test is doing?
> 
> Sure - that's described in my paper. It uses a hardware timer to
> generate periodical interrupts. The ISR releases a posix semaphore
> on which a pthread blocks.
> The interrupt latency (timer rolling through zero until user ISR is
> executed) and the thread dispatching latency (ISR releases sema
> until the [high priority] thread obtains it and is scheduled)
> are recorded.

Sorry Till.  I didn't realize we were talking about your benchmark. :)

> I compared combinations of using posix semaphores vs. native RTEMS
> semaphores or RTEMS events and pthreads vs. RTEMS native threads.
> 
> The results were comparable except for the [worst case] thread
> dispatching latency which was much worse when using pthreads than when
> using native threads. (I remember a phone conversation with you and you
> suspecting that it could be caused by the posix signal delivery code [no
> signals were used in the test, though]).

There is an outstanding change request to install the POSIX signal
stuff only after an application uses it.   

> NOTE: I run this test for a while under heavy load (networking, console
> traffic etc.). The average numbers are usually pretty good. It's the
> worst case (i.e. the longest latency recorded during the test) who
> is critical.

I remember that you figured out that the biggest source of latency was
the code calling some routine to determine how much free heap was left
and that routine disabled dispatching for an O(n) scan of the heap.
I fixed this problem.  

But you have been busy and I guess I never got updated results.

> But as I said: I'll try to run the test on ss-20021007 ASAP.

Please do.  If we can track it down to some area, it would be 
possible to address.

FWIW the POSIX signal processing overhead can be eliminated very easily
for testing purposes.  Either comment out the reference to 
 _POSIX_signals_Post_switch_extension in posix/src/pthread.c or 
do a return at the top of that routine in posix/src/psignal.c.
My gut feeling is that the routine is not the problem as it has this
early exit:

 _ISR_Disable( level );
    if ( !(~api->signals_blocked &
          (api->signals_pending | _POSIX_signals_Pending)) ) {
     _ISR_Enable( level );
     return;
   }
  _ISR_Enable( level );

and if there are no signals pending that the thread is interested in, it 
would be tripped.  But I have been wrong before. :)

I could see an easy mistake where RTEMS tasks slow down when POSIX is
enabled
because they don't have the POSIX signals structure initialized
correctly but
I recently went over this part of the code and think it disables POSIX
signals 
for non-POSIX threads by default.


> -- Till
> 
> > Is it the same as yours?  Saying
> > something is slow doesn't help me much.  RTEMS usually performs
> > comparable to VxWorks on benchmarks.  But each time we hit a new
> > benchmark comparison, there is new data to optimize against.
> >
> >
> >>-- Till
> >>
> >>
> >>>
> >>>
> >>>>>Thanks.
> >>>>>
> >>>>>Kamen Penev
> >>>>>Adept Technology
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>Good question. I am not using pthreads and the honest answer is "I dont
> >>>>know".
> >>>>I remember, however having brought this to Joel Sherrill's attention
> >>>>(back in 2001
> >>>>when I did the tests) and I believe he had some ideas about the cause...
> >>>>
> >>>>-- Till
> >>>>
> >>>>
> >>>
> >>>
> >>>
> >

-- 
Joel Sherrill, Ph.D.             Director of Research & Development
joel at OARcorp.com                 On-Line Applications Research
Ask me about RTEMS: a free RTOS  Huntsville AL 35805
Support Available                (256) 722-9985