MIPS BSPs

[Wayne Bullaughey]

I have been exchanging email with Joel about making some changes in the way
the MIPS BSPs are organized. He suggested having the discussion on the users
email list. I will try and summarize what we have been discussing. There are
many slightly different CPUs in the MIPS family from a number of different
vendors. I'm mostly familiar with the MIPS processors from IDT. Processors
from other vendors may present problems I've missed. I'm also only familiar
with MIPS ports of RTEMS so I may be missing some of the solutions to these
problems other ports have used.

One of the main issues is how should conditional compilation be specified in
files which need to have differences based on specific CPU types. The gcc
compiler has several built in defines which to some degree describe the
processor. The instruction set a MIPS processor executes is characterized by
its ISA level. The levels currently range from 1 to 4 and are mostly upward
compatible. The compiler defines a symbol __mips with the ISA value. The ISA
only characterizes the instruction set. There are other attributes of the
processor that are independent of the ISA but tend to correlate to a certain
degree. The differences include register layout (status register, config
register and TLB registers are examples), cache architecture and memory
mapping. Processors with ISA 1 tend to be similar in these areas but not
identical. ISA 3 also have similar characteristics. The second kind of
predefined symbols are more processor architecture related. They include
R3000, R3900, R6000, R4000, R4100, R4300, R4600, R4650, R5000, R8000. When
you build a cross compiler you can specify which MIPS processor to target.
Since the compiler only needs to deal with the instruction set a compiler
built for one of the ISA level 3 processors can be used for any of them so
its not very convenient to depended on these predefined symbols. Experience
with the compiler also indicates its not very consistent on how the symbols
get defined.

There are three subdirectories where MIPS specific code is:
   c\src\exec\score\cpu\mips
   c\src\lib\libcpu\mips\xxx
   c\src\lib\libbsp\mips\xxx

c\src\exec\score\cpu\mips is largely specific processor independent. It has
the lowest level interrupt handler, functions to enable and disable
interrupts, install handlers and a default idle loop. Dependencies in this
code include the layout of the status register. Early MIPS 3000 processor
used one layout and 4000 and later processors used a second. There are also
differences in which interrupt vectors exist and their locations.

I have run across two board specific issues that relate to this code. The
first is whether the operational interrupt vectors will use the boot
(normally ROM address space) vectors or the standard (normally RAM) vectors.
Some of our boards have RAM at the boot vector locations and no memory at
the standard so we install handlers at the boot vectors. The defines in
idtcpu.h are hard coded to specify the standard vectors. There is also a bit
in the status register which must select the correct vector.

The second board specific issues deals with the technique for disabling
interrupts. The normal technique of reading the status register, masking off
an enable bit and writing it back is not safe if an interrupt handler makes
changes in the status register which need to be preserved after the
interrupt handler exits. If an interrupt occurs between the read and write
of the status register then the change the interrupt handler makes is lost.
An alternate approach we use uses the syscall instruction to enter the
interrupt handler which then exits leaving interrupts disabled.

c\src\lib\libcpu\mips is where code which is quite different between
different processors should be. This is a new area for the MIPS family and
does not have very much in it yet. There is some code for handling
differences in interrupt vectors. Cache handling could go here since it
tends to be more processor specific then board specific. I see this level as
being more family organized then single CPU model.

c\src\lib\libbsp\mips has code specific to a particular board support
package. Any thing processor specific not done it the other two areas needs
to be done here.

Problem areas:

The file c\src\exec\score\cpu\mips\idtcpu.h has a lot of defines in it that
vary from processor to processor. In the latest snapshot it uses the
predefined symbol __mips to organize the differences. This organization
works for many CPUs but not all. For example the IDT 32364 is mostly an ISA
level 2 but has registers much more like other ISA level 3 processors. The
bit fields of the status and config register definitions have the most
differences. There are minor differences in interrupt vectors. The config
register defers quite a bit from one processor to another. It includes bits
for cache modes and sizes.

Of all of the common functions the CPU specific parts deal with: interrupt
handling, cache, timer, TLB none are 100% independent of the particular CPU.
The goal of the organization should be to  have as much as possible that is
common to all in the area which is shared by the most BSPs. Interrupt
handling is almost the same for all the CPUs so it makes sense to have it in
the c\src\exec\score\cpu\mips directory. The question then is how to make
the small parts that are CPU specific get the correct things needed. Its my
opinion that the compiler predefined symbols __mips and Rxxxx are not
sufficient.

One alternative I have come up with is to define a new set of macros which
would characterize the cpu and be defined by the BSP configuration and
passed in as CFLAGS. The include file idtcpu.h or renamed version of it
would use new defines for processor specific defines. Source files like
cpu_asm.S would have conditional compilation where small differences were
needed based on the defines. Functions with major differences would be in
the libcpu or libbsp directories.

The second approach is to break idtcpu.h up into 3 levels: all MIPS,
processor specific and board specific. One file would exist at the all MIPS
level (c\src\exec\score\cpu\mips). Files with the same name but in different
subdirectories would exist for the processor specific
(c\src\lib\libcpu\mips\xxx)and board specific levels
(c\src\lib\libbsp\mips\xxx). The BSP configuration would set the appropriate
include search path so the proper files would be included. I think this
approach has the advantage that as support for new CPUs and boards are added
changes do not need to be made in the common include file, only new versions
of the specific files need to be added.

I've focused on idtcpu.h since it has so many processor specific defines in
it. I'm sure there are other issues in the code its self but I felt that if
an approach for reorganizing idtcpu.h could be agreed upon that could be
applied to the files as well.

Sorry for the length of this. I hope it helps set a common understanding for
follow on discussions. Please correct or add to what I have said.

Wayne Bullaughey               Voice:  (610) 692-9526 ext: 104
Woodward McCoach, Inc.                 (877) 284-4804
1180 McDermott Drive           Fax:    (610) 436-8258
West Chester, PA 19380         Email:  wayne at wmi.com
                               web:    www.wmi.com