[Bug 1937] New: SMP: inter processor interrupt message problems

[bugzilla-daemon at rtems.org]

https://www.rtems.org/bugzilla/show_bug.cgi?id=1937

           Summary: SMP: inter processor interrupt message problems
           Product: RTEMS
           Version: HEAD
          Platform: All
        OS/Version: RTEMS
            Status: NEW
          Severity: normal
          Priority: P3
         Component: cpukit
        AssignedTo: joel.sherrill at oarcorp.com
        ReportedBy: daniel at gaisler.com


Created attachment 1355
  --> https://www.rtems.org/bugzilla/attachment.cgi?id=1355
SMP IPI patch

Hello,

This patch contains several fixes in the same lines of code:

1. a IPI should only be generated when the per-cpu core lock is taken, this is
because a race may arise when two CPUs generate a IPI to the same CPU at the
same time, or one CPU generates two IPIs in a row and the destination CPU is
about to handle the incoming IPI. It could lead to percpu[dest_cpu].message
becomes 0, and the lock is never returned (see IPI handler)

2. IPI should not be generated unless message==0, this is of the same reason as
above. The IRQ can have been ACKed by the receiving CPU but not handled yet,
the sending CPU would generate another IRQ, then after the receiving CPU has
handled the message another IRQ will be handled and then message=0 which will
result in same dead lock as in 1.

Note that 1. and 2. has removed the use of broadcast IPI. The problems could
probably have been solved in another way so that the broadcast could still be
used.

3. In _SMP_Broadcast_message() the lock is taken on the sending CPU (cpu)
rather than the receiving CPU (dest_cpu).

4. _SMP_Request_other_cores_to_dispatch() never sends IPIs to CPU0, I'm not
sure this is incorrect due to my limited knowledge of the RTEMS SMP scheduler
etc. But at a first glance it look incorrect to me.

Please see attached patch, or below.

Daniel


Index: cpukit/score/src/smp.c
===================================================================
RCS file: /usr1/CVS/rtems/cpukit/score/src/smp.c,v
retrieving revision 1.9
diff -u -r1.9 smp.c
--- cpukit/score/src/smp.c    22 Aug 2011 18:26:08 -0000    1.9
+++ cpukit/score/src/smp.c    17 Oct 2011 11:15:22 -0000
@@ -176,9 +176,13 @@
   #endif

   level = _SMP_lock_spinlock_simple_Obtain( &_Per_CPU_Information[cpu].lock );
-    _Per_CPU_Information[cpu].message |= message;
+    if (_Per_CPU_Information[cpu].message == 0) {
+      _Per_CPU_Information[cpu].message = message;
+      bsp_smp_interrupt_cpu( cpu );
+    } else {
+      _Per_CPU_Information[cpu].message |= message;
+    }
   _SMP_lock_spinlock_simple_Release( &_Per_CPU_Information[cpu].lock, level );
-  bsp_smp_interrupt_cpu( cpu );
 }

 /*
@@ -197,11 +201,15 @@
   for ( dest_cpu=0 ; dest_cpu <  _SMP_Processor_count; dest_cpu++ ) {
     if ( cpu == dest_cpu )
       continue;
-    level = _SMP_lock_spinlock_simple_Obtain( &_Per_CPU_Information[cpu].lock
);
-      _Per_CPU_Information[dest_cpu].message |= message;
-    _SMP_lock_spinlock_simple_Release( &_Per_CPU_Information[cpu].lock, level
);
+    level = _SMP_lock_spinlock_simple_Obtain(
&_Per_CPU_Information[dest_cpu].lock );
+      if (_Per_CPU_Information[dest_cpu].message == 0) {
+        _Per_CPU_Information[dest_cpu].message = message;
+        bsp_smp_interrupt_cpu( dest_cpu );
+      } else {
+        _Per_CPU_Information[dest_cpu].message |= message;
+      }
+    _SMP_lock_spinlock_simple_Release( &_Per_CPU_Information[dest_cpu].lock,
level );
   }
-  bsp_smp_broadcast_interrupt();
 }

 /*
@@ -230,7 +238,7 @@

   if ( !_System_state_Is_up (_System_state_Current) )
     return;
-  for (i=1 ; i < _SMP_Processor_count ; i++ ) {
+  for (i=0 ; i < _SMP_Processor_count ; i++ ) {
     if ( cpu == i )
       continue;
     if ( _Per_CPU_Information[i].state != RTEMS_BSP_SMP_CPU_UP )

-- 
Configure bugmail: https://www.rtems.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.