Mailbox RPi patch and rtems_cache_* probably broken on RPi

[Pavel Pisa]

Hello Mudit and Sebastian,

I have tested patch for reding board revision and it seems
to work as expected when called during board initialization.
They work even after bsp_memory_management_initialize()
and even in _Thread_Start_multitasking() function.

But mailbox fails after that in _Thread_Handler( void ),
this is when run in regular thread content.

I have found that when I disable MMU/cache completely
or call directly arm_cp15_data_cache_clean_and_invalidate()
to flush complete data cache before transfer then correct behavior
is restored.

This is strange because I think that cache is enabled even
in _Thread_Start_multitasking(). I have checked that problem
is not caused/restored by long delay inserted by complete cache flush.
I have tried to add "dumy" delay loop with repeating 
  RTEMS_COMPILER_MEMORY_BARRIER();
  arm_cp15_drain_write_buffer();
but this does not help.

When
  rtems_cache_flush_multiple_data_lines(buf, size);
  rtems_cache_invalidate_multiple_data_lines(buf, size);
are replaced by
  arm_cp15_data_cache_clean_and_invalidate()
all calls to mailbox exchange works.
May it be that rtems_cache_* influences only L1 cache
but then I do not see which rtems calls should be used
to synchronize access to memory with external devices/DMA
I have added some RTEMS_COMPILER_MEMORY_BARRIER(); to the code
and experienced with these in more places too but that did not
help.

This patch help me

----------------------------------------------------------------
diff --git a/c/src/lib/libbsp/arm/raspberrypi/misc/vc.c b/c/src/lib/libbsp/arm/raspberrypi/misc/vc.c
index d27668d..5e1de9e 100644
--- a/c/src/lib/libbsp/arm/raspberrypi/misc/vc.c
+++ b/c/src/lib/libbsp/arm/raspberrypi/misc/vc.c
@@ -34,6 +34,7 @@
 static inline bool
 bcm2835_mailbox_buffer_suceeded(const bcm2835_mbox_buf_hdr *hdr)
 {
+  RTEMS_COMPILER_MEMORY_BARRIER();
   return (hdr->buf_code == BCM2835_MBOX_BUF_CODE_REQUEST_SUCCEED);
 }
 
@@ -58,11 +59,20 @@ bcm2835_mailbox_buffer_flush_and_invalidate(void *buf, size_t size)
 
   sctlr_val = arm_cp15_get_control();
 
+  RTEMS_COMPILER_MEMORY_BARRIER();
   arm_cp15_drain_write_buffer();
   if (sctlr_val & (ARM_CP15_CTRL_C | ARM_CP15_CTRL_M)) {
-    arm_cp15_drain_write_buffer();
+#if 0
+    /*
+    These architecture independent RTEMS API functions should work
+    but for some reason they are not enough
+    */
     rtems_cache_flush_multiple_data_lines(buf, size);
     rtems_cache_invalidate_multiple_data_lines(buf, size);
+#else
+    /* Flush complete data cache */
+    arm_cp15_data_cache_clean_and_invalidate();
+#endif
   }
 }
 
----------------------------------------------------------------

MBOX test/HACK in start multitasking

----------------------------------------------------------------

diff --git a/cpukit/score/src/threadstartmultitasking.c b/cpukit/score/src/threadstartmultitasking.c
index 291e42c..8c5b91d 100644
--- a/cpukit/score/src/threadstartmultitasking.c
+++ b/cpukit/score/src/threadstartmultitasking.c
@@ -20,6 +20,7 @@
 
 #include <rtems/score/threadimpl.h>
 #include <rtems/score/assert.h>
+#include <bsp/vc.h>
 
 void _Thread_Start_multitasking( void )
 {
@@ -44,6 +45,23 @@ void _Thread_Start_multitasking( void )
   _CPU_SMP_Prepare_start_multitasking();
 #endif
 
+  int res;
+  bcm2835_mailbox_get_fw_rev_entries fw_rev;
+  printk("bcm2835_mailbox_get_firmware_revision ...\n");
+  res = bcm2835_mailbox_get_firmware_revision(&fw_rev);
+  printk("bcm2835_mailbox_get_firmware_revision %d %d\n", res, fw_rev.fw_rev);
+
+  bcm2835_get_board_spec_entries board_spec;
+  printk("bcm2835_mailbox_get_board_model ...\n");
+  res = bcm2835_mailbox_get_board_model(&board_spec);
+  printk("bcm2835_mailbox_get_board_model %d %d\n", res, board_spec.spec);
+
+  bcm2835_get_board_spec_entries board_spec1;
+  printk("bcm2835_mailbox_get_board_revision ...\n");
+  res = bcm2835_mailbox_get_board_revision(&board_spec1);
+  printk("bcm2835_mailbox_get_board_revision %d %d\n", res, board_spec1.spec);
+
+
 #if defined(_CPU_Start_multitasking)
   _CPU_Start_multitasking( &heir->Registers );
 #elif defined(RTEMS_SMP)
diff --git a/testsuites/samples/ticker/init.c b/testsuites/samples/ticker/init.c

----------------------------------------------------------------

Test/HACK to check functions from init thread

----------------------------------------------------------------

index 3f3cbd8..f09a6a6 100644
--- a/testsuites/samples/ticker/init.c
+++ b/testsuites/samples/ticker/init.c
@@ -13,9 +13,30 @@
 
 #define CONFIGURE_INIT
 #include "system.h"
+#include "bsp/vc.h"
 
 const char rtems_test_name[] = "CLOCK TICK";
 
+void print_rev(void)
+{
+  int res;
+
+  bcm2835_mailbox_get_fw_rev_entries fw_rev;
+  printf("bcm2835_mailbox_get_firmware_revision ...\n");
+  res = bcm2835_mailbox_get_firmware_revision(&fw_rev);
+  printf("bcm2835_mailbox_get_firmware_revision %d %d\n", res, fw_rev.fw_rev);
+
+  bcm2835_get_board_spec_entries board_spec;
+  printf("bcm2835_mailbox_get_board_model ...\n");
+  res = bcm2835_mailbox_get_board_model(&board_spec);
+  printf("bcm2835_mailbox_get_board_model %d %d\n", res, board_spec.spec);
+
+  bcm2835_get_board_spec_entries board_spec1;
+  printf("bcm2835_mailbox_get_board_revision ...\n");
+  res = bcm2835_mailbox_get_board_revision(&board_spec1);
+  printf("bcm2835_mailbox_get_board_revision %d %d\n", res, board_spec1.spec);
+}
+
 /*
  *  Keep the names and IDs in global variables so another task can use them.
  */
@@ -40,6 +61,11 @@ rtems_task Init(
   time.second = 0;
   time.ticks  = 0;
 
+  print_rev();
+  print_rev();
+  print_rev();
+  print_rev();
+
   status = rtems_clock_set( &time );
   directive_failed( status, "clock get" );
 

----------------------------------------------------------------

If there is no objection I update
  bcm2835_mailbox_buffer_flush_and_invalidate()
to call inefficient arm_cp15_data_cache_clean_and_invalidate();
to ensure that RPi development projects are not blocked.

I send Mudit changes to mainline as well if there is no
problem pointed by others.

Best wishes,

               Pavel