[6-freebsd-12 PATCH 1/2] rtemsbsd/bus: Add PCI support to the nexus bus

[Chris Johns]

On 4/3/21 5:53 pm, Sebastian Huber wrote:
> On 04/03/2021 00:18, Chris Johns wrote:
>> On 3/3/21 7:48 pm, Sebastian Huber wrote:
>>> On 03/03/2021 08:52, Chris Johns wrote:
>>>> Then I hit minimum and some validation tests and they failed because I had
>>>> removed bsp/default-initial-extension.h and I suppose it needs to come via
>>>> bsp.h
>>>> however it includes rtems.h because it needs to have access to
>>>> rtems_fatal_source etc.
>>>>
>>>> It seems there is a lot of places that subtly depend on bsp.h and what it
>>>> includes.
>>>>
>>>> I will chat to Joel tomorrow to figure out how this is to be resolved.
>>>>
>>>> Any suggestions?
>>> The minimum requirement for bsp.h is to include:
>>>
>>> rtems.h
>>>
>>> bspopts.h
>> It is news to me this is a requirement and I am unsure where it has come from.
>> Has it been documented? The information you cut from my last post shows RTEMS is
>> pretty clean and the number of issues are small so I am not sure this needs to
>> happen.
> This is what I figured out over the years.

I think the ability to have them separated is healthy.

>>
>> These powerpc BSPs predate libbsd, they have legacy networking support and are
>> important to RTEMS users.
>>
>> Your request to not including bsp.h because it includes rtems.h in a driver
>> section of a third party package that needs specific BSP bus mapping information
>> appear to conflict. The package needs low level BSP information but it cannot
>> include bsp.h by definition now.
>>
>> What is now bugging me is the layering of rule upon rule mixed with defaults.
>> The rules are complex and in places seem arbitrary. Let me write the rules out
>> as I understand them:
>>
>> 1. The BSP header bsp.h is the access to BSP interfaces
>>
>> 2. The BSP header bsp.h must include rtems.h and bspopt.h
>>
>> 3. LibBSD generic bus handling must be inline for performance reasons
>>
>> 4. LibBSD generic bus handling assumes cache coherent memory by default
>>
>> 5. BSPs must register cache coherent memory if it does not meet rule 4
>>
>> 6. LibBSD generic bus handling defaults to a flat 1:1 full memory space
>>     address map
>>
>> 7. LibBSD generic bus handling requires bsp/bus.h to provide BSP mappings
>>     if it does not meet rule 6
>>
>> 8. A BSP provided bsp/bus.h cannot include rtems.h or any header that
>>     includes rtems.h such as bsp.h. This special case is exempt from rule 1
>>
>> 9. RTEMS cannot change any header an existing BSP with bsp/bus.h
>>     includes to include rtems.h or any header that has a dependent that
>>     includes rtems.h
>>
>> <bleach> that is a lot of rules to swallow. An alternative set of rules is:
>>
>> 1. The BSP header bsp.h is the access to BSP interfaces
>>
>> 2. LibBSD generic bus handling must be inline for performance reasons
>>
>> 3. LibBSD generic bus handling requires a BSP provide suitable cache
>>     coherent memory to the cache coherent memory allocator
>>
>> 4. LibBSD generic bus handling requires a BSP provide bsp/bus.h to
>>     provide BSP IO mappings
>>
>> The second set of rules is clear, does not self reference and universally
>> applies to all BSPs. It removes the defaults from LibBSD and lets us manage them
>> in rtems.git for a BSP. Explicitly requiring a BSP to provide support lets a
>> user easily check any BSP to see what cache coherent memory is configured and
>> what the bus handlers are.
>>
>>> It shall also define BSP_INITIAL_EXTENSION (normally via #include
>>> <bsp/default-initial-extension.h>).
>> This is a recent addition and it is the only piece I found that has an issue
>> when building the motorola_powerpc. Maybe the way this is implemented is needs
>> to be reconsidered or we accept bsp.h does include rtems.h either directly or
>> indirectly.
>>
>> The ability to interchange either bsp.h or rtems.h or having code that depends
>> on one because you include the other seems wrong.
> 
> We should move away from BSP-specific interfaces. 

Could you please expand a little on this? I am wondering how "interfaces" is
being use here.

Would a bsp/bus.h interface based on the macros I listed in this thread's patch
work as a start?

I have isolated around 5 powerpc BSPs that would need to be updated as a set...

 beatnik
 psim
 motorola_powerpc
 mvme5500
 haleakala

I plan to discuss this tomorrow with Jennifer and Kinsey to figure out how to
resolve this. Your input and feedback has been valuable.

> The <bsp.h> is the only
> mandatory header file provided by a BSP and may contain all sorts of things.

Yes it is a bit of a sweeper for a lot of things, a little too much.

> From my point of view the BSP should indicate which features it
> requires/supports and then it should implement a standard interface.

That would be nice. There are defacto standards in some parts where driver
sharing required it happen but I think there is no uniform set of interfaces.

A key issue is the size of a task that needs to touch all BSPs. We tend to look
at blocks of work in the generic areas or as a specific BSP or family of BSPs.
Large refactoring of BSPs is hard to get funding for and hard to test.

> The bus API implementation in FreeBSD is architecture-specific.

It is and I am fine with how we currently have thing implemented in libbsd. The
x86 needs its own bus API and I reviewed the rtemsbsd shared bus support and it
currently fits most cases I can see. I wondered about a powerpc specific bus
version and all I would end up doing is the same thing we have in rtemsbsd plus
something like the patch in this thread. I also reviewed the FreeBSD
implementation for the powerpc, we should avoid it.

> We can do this in RTEMS as
> well and do the implementation in cpukit, for example based on the riscv
> implementation in FreeBSD. The BSP could then simply indicate if it needs a full
> featured implementation or the simple inline implementation we have currently.

Yeah this sound nice.

> The definition of BSP_INITIAL_EXTENSION can move to a separate header file.

I wonder about this but I could not see how to implement it.

> The cache coherent memory is a different topic and I think this is already
> sorted out in
> 
> http://devel.rtems.org/ticket/4243

Yes and I hopefully have an elegant solution in mind for those BSPs who
currently depend on the default heap allocation. I included it here to make the
list of rules as complete as possible to avoid the post appearing in searches
and being confused as "the" list.

Chris