Network buffers, MPC860 and data caching

[Eric Norum]

Till Straumann wrote:
> 
> Charles-Antoine Gauthier wrote:
> 
> >
> > > Have a look at Charles Gauthier's & Darlene Steward's MBX BSP in the
> > > current snapshot. It goes a different route: All RAM is cacheable,
> > > however, once CPM buffers (not only for network) are touched, explicit
> > > cache commands for the memory covered by the buffer are used:
> > >
> > >    - on output buffers, caches are flushed after writing and before
> > >      setting the ready bit
> > >    - on read, the cache lines for the buffer are invalidated before
> > >      reading the buffer
> > >
> 
> Ok, that's what I want to do on the MPC860 powerpc FADS board.
> I thought about how to implement this because I saw that the network and
> console drivers use `malloc()ed' buffers. This complicates things because,
> obviously, receive buffers must be cache line aligned. AFAIK network
> receiving buffers are allocated by the driver, passed upstream and are
> also deallocated upstream.
> 
> Eventually, I decided to download the latest snapshot and have a look at
> your implementation. I could see that your  `rtems_cache_aligned_malloc()'
> looks pretty much like what was my first idea.
> However, (and that's the problem stumbled over), how does `free()' like
> the modified address returned by `rtems_cache_aligned_malloc'()? I'm
> afraid not very much - it seems to me that free()ing
> cache_aligned_malloc()ed memory results in heap corruption :-)
> 

Network receive buffers are allocated by MCLGET, not by malloc.  An mbuf
cluster is aligned on a cluster-sized (2 kbyte) boundary.  Network mbufs
are aligned on an mbuf-sized (128 byte) boundary.  See bsd_init in
src/libnetworking/rtems/rtems_glue.c.  Unless a cache line is larger
than an mbuf there should be no problems.

Console receive buffers are 16 bytes long and are statically allocated,
but there's no attempt to cache-line-align them.  This may be a
problem.  Charles?

I agree with your concern about passing a pointer obtained from
rtems_cache_aligned_malloc to free.
The snapshot I'm working with (rtems-ss-20000929) has *no* calls to
rtems_cache_aligned_malloc, though, so there's no possibility of bad
pointers getting passed to free().

BTW -- I see you're from SLAC.  Are you considering using EPICS with
RTEMS?
-- 
Eric Norum                                 eric at cls.usask.ca
Canadian Light Source                      Phone: (306) 966-6308
University of Saskatchewan                 FAX:   (306) 966-6058
Saskatoon, Canada.