more cache manager issues

[Charles-Antoine Gauthier]

Till Straumann wrote:
> 
> Browsing through the cache manager implementation of
> `rtems-ss-20000929' caused me having some more questions
> / suggestions:
> 
>  - cache_aligned_malloc() (currently not called by any piece of code)
> must not
>      be used. Calling `free' on memory allocated by
> cache_aligned_malloc() results
>      in heap corruption.
> 
> All the other issues apply to the POWERPC architecture:
> 
>  - rtems_cache_flush_multiple_data_lines() etc: after repeatedly
>      flushing/invalidating single cache lines (`dcbst', `dcbi', `dcbf')
>      a `SYNC' instruction _must_ be issued to guarantee that the
> operations
>      have completed before returning from the
> rtems_cache_xxx_multiple_data_lines()
>      etc. routines. (To enhance performance, the CPU dependent single
> line operations
>      should probably be inlined).
> 
>  - At least the MPC8xx implementation (didn't look too close at the
> other powerpc cpus)
>      of _CPU_cache_enable_data() etc. is incorrect. Note that
> enabling/disabling the cache
>      does not invalidate / flush the cache (consult the MPC-860 user
> manual).
> 
>      The correct way to enable the data cache is as follows:
> 
>         1)  invalidate the complete data cache
>         2)  `sync' to make sure the operation has completed
>         3) enable the data cache
> 
>      Disabling the data cache should consist of the following steps
> 
>        1) flush (write back and invalidate) the entire data cache
>        2) `sync'
>        3) disable the data cache
> 
>       Well, there's no clue whether steps 2) are really necessary.
> Theoretically,
>       the cache could perform the posted write backs after it has been
> disabled.
>       However, I feel that it is safer doing the `sync'. Note that
> 1) are definitively
>        required. (On the other hand, I can't really see the benefits of
> the currently
>        used `isync' as step 4.)
> 
> Comments?
> 
> -- Till.

We might want to consider changing the heap manager to return blocks
that are already cache line aligned. The cache_aligned_malloc() was the
result of realizing the need for this feature. However, as Eric pointed
out, it was never used so it was never thought through carefully.

I would however point out that doing this only makes sense when the
cache lines are small. Otherwise, there can be a lot of wasted space.

For large cache lines (machines with second level caches), we should
consider allocating DMA buffers out of non-cachable memory. This now
requires a VM manager and the ability to place heaps in various regions.
Not much of a problem, I think. Ah, if only hardware designers knew
something about software...

WRT sync, if memory if marked as coherent (haven't looked at my 860
manual, so I don't know if this applies to that processor), then an
eieio might be better?

Synchronization should be required whether 1 or multiple lines are
flushed. The synchronization only needs to be done once. Consequently,
the multiple line version of a function should not call the single line
version.

I thought the single line functions were inline. If they are not, they
should be.

-- 
Charles-Antoine Gauthier
Institute for Information Technology   Institut de technologie de
l'information
National Research Council of Canada    Conseil national de recherches du
Canada