[GSoC - x86_64] Interrupt manager and and port-specific glue - was Re: [GSoC - x86_64 - automake] Limit CFLAGS to specific source for librtemsbsp.a

Wed Aug 1 15:11:52 UTC 2018

On Wed, Aug 1, 2018 at 9:15 AM, Amaan Cheval <amaan.cheval at gmail.com> wrote:
> That's definitely very illuminating, thank you so much for all the details!
>
> A few more questions that have arisen for me. Feel free to skip over
> them (I'll likely figure them out given enough time, so I'm only
> asking in case any of them are obvious to anyone):
>
> - The i386 doesn't use CPU_Interrupt_frame at all. It seems like it
> stores some of the data onto the stack?
>
the interrupt frame structure was introduced during 4.11 development.
probably i386 never got updated to use a struct to encapsulate the
interrupt frame. the interrupt frame should contain the registers that
are preserved by the interrupt entry code I believe.

> - There used to be defines in cpu.h regarding hardware/software based
> interrupt stacks, and how they'd be setup, which were made
> superfluous[1] - I'm not quite sure how these are meant to work - I
> see references to "stack high" and "stack low" and I'm not quite sure
> what the code is referencing when using those.
>

a hardware interrupt stack is one that the hardware switches to during
an interrupt. i think m68k has such.

most interrupt stacks in RTEMS are software-managed, meaning that
RTEMS explicitly switches the stack region off the task stack and to
an interrupt stack region.

some stacks start high and grow down, and some stacks start low and
grow up. maybe this is what the "stack high" and "stack low" you
mention are in relation to?

> - c/src/lib/libbsp/no_cpu/no_bsp/Makefile.am doesn't include
> irq-sources.am, by the way (this is part of why I used to think a lot
> of what your email mentioned was unnecessary, until you...ahem,
> pre-empted that line of thought and helped clarify it :P). Should I
> add a ticket to update the no_bsp code to be more in line with current
> use?
>
Sure. I don't know that anyone is in particular maintaining
no_cpu/no_bsp since we can't compile it, it is basically best effort
stuff that sometimes we miss updating.

> - My understanding of _ISR_Handler is that it'll be the handler for
> _all_ interrupt vectors by default - it'll then dispatch interrupts to
> user-handlers (or internal handlers, for the timer, for eg.). Is that
> right? (I don't quite understand its interaction with the RTEMS
> interrupt manager yet, but irq-generic's "bsp_interrupt_handler_table"
> seems to be the RTEMS equivalent to the processor-specific vector
> table, and "bsp_interrupt_handler_dispatch" seems to call the actual
> handler within that table as appropriate. Accurate? (I just haven't
> found how that table actually gets its handlers setup besides during
> initialization, since rtems_interrupt_catch just calls
> _CPU_install_vector, which updates the processor vector table, not the
> RTEMS interrupt manager vector table.)
>

You have discovered a couple of different but related interrupt
processing code bases.  I can see why you get confused.

Basically, a CPU port should support two kinds of interrupts that may
be installed, "RTEMS" and "Raw" interrupts. The "Raw" interrupts are
installed directly in the processor's vector table. For processors
that use simple vectored interrupts, the "RTEMS" interrupts install a
call to the _ISR_Handler() function in the processor's vector table,
and will put the user's isr function pointer in the
_ISR_Vector_table(), which is the RTEMS Interrupt Manager's vector
table.

I'm not really familiar with the processors that use a different model
for interrupt handling besides simple vectored. Probably, you will
have to study one of them.

This irq-generic bsp_interrupt_* code is used by the "IRQ Server" that
builds from the CPU port capabilities to allow for some advanced
features like chaining multiple isrs from the same source. I don't
think you want to be focusing on those right now, but I could be
mistaken. I haven't spent much time looking at the IRQ Server
codebase.

Gedare

> - My understanding of the interaction between RTEMS' interrupt manager
> (i.e. support for nested interrupts and thread dispatch once an
> interrupt ends) and the BSP's processor-specific interrupt manager
> (code to use the APIC and IDT in my case) is that they're tied
> together through the use of irq-generic.c's "bsp_interrupt_initialize"
> - is that right? (m68k never seems to call it, though, so perhaps
> not?)
>
> Sorry about the rambling! To reiterate, I'll likely figure it out
> given enough time, so if the answers aren't at the top of your head, I
> can figure it out without wasting your time :)
>
> [1] https://devel.rtems.org/ticket/3459#comment:11
>
> On Wed, Aug 1, 2018 at 3:18 AM, Joel Sherrill <joel at rtems.org> wrote:
>>
>>
>> On Tue, Jul 31, 2018 at 3:05 PM, Amaan Cheval <amaan.cheval at gmail.com>
>> wrote:
>>>
>>> Hm, I'm not sure what to look for in the other ports specifically, really.
>>> The BSP porting documentation doesn't have a section on interrupts, so I'm
>>> doing this on more of an "as it comes up" basis.
>>>
>>> What I've got right now (the interrupt handlers in C) are what I need for
>>> calibrating the APIC timer (through the PIT) - so simply hooking IRQ0 (for
>>> the timer) and IRQ7 (spurious vector), since those are needed for the timer
>>> work to continue.
>>>
>>> What constitutes as a requirement for basic interrupt support?
>>
>>
>> There used to be a generic porting guide. I can see that this particular
>> section
>> has bit rotted some but the interrupt dispatching section. Some of this
>> will have evolved to support SMP and fine grained locking but the
>> pseudo-code
>> here will give you a push toward the right line of thinking:
>>
>> https://docs.rtems.org/releases/rtemsdocs-4.10.2/share/rtems/html/porting/porting00034.html
>>
>> The idea is that you need to ensure RTEMS knows it is inside an interrupt
>> and the current locking scheme (old was dispatching, new is ...) is honored.
>>
>> The ARM and PowerPC (plus RISCV) are good ports to look at for how SMP
>> plays into this. But the CPU supplement is thin for their interrupt
>> processing.
>>
>>
>> This is the CPU Architecture supplement section for the m68k. This is a
>> relatively simple
>> architecture to describe. There is also a section for the i386 which reads
>> similarly.
>>
>> https://docs.rtems.org/branches/master/cpu-supplement/m68xxx_and_coldfire.html#interrupt-processing
>>
>> Personally, I find the m68k a fairly easy processor to read assembly in.
>> Look at cpukit/score/cpu/m68k/cpu_asm.S and _ISR_Handler to see what
>> is done there w/o SMP. On the m68k _ISR_Handler is directly put into the
>> vector table. But this isn't the most similar example for you.
>>
>> For the i386 (better example), it is in bsps/i386/shared/irq/irq_asm.S with
>> the
>> same name. There _ISR_Handler is installed via the DISTINCT_INTERRUPT_ENTRY
>> macros at the bottom of the file where some prologue jumps to the common
>> _ISR_Handler and then the actions are similar. Usually _ISR_Handler type of
>> code ends up invoking a PIC decode method in normal C without an
>> interrupt attribute.
>>
>> Long and multi-architecture answer but maybe that makes sense. The goal
>> in ticker.exe is to take a number of tick interrupts which don't schedule
>> and
>> then take one that does -- it schedules a preemption of the idle thread.
>>
>> Hope this helps.
>>
>> --joel
>>
>>
>>
>>>
>>>
>>> On Wed, Aug 1, 2018, 1:29 AM Joel Sherrill <joel at rtems.org> wrote:
>>>>
>>>>
>>>>
>>>> On Tue, Jul 31, 2018 at 2:52 PM, Amaan Cheval <amaan.cheval at gmail.com>
>>>> wrote:
>>>>>
>>>>> Hi Chris!
>>>>>
>>>>> I currently have code like this in
>>>>> c/src/lib/libbsp/x86_64/amd64/Makefile.am:
>>>>>
>>>>>     librtemsbsp_a_SOURCES +=
>>>>> ../../../../../../bsps/x86_64/amd64/interrupts/handlers.c
>>>>>     # XXX: Needed to use GCC "interrupt" attribute directives - can we
>>>>> pass these
>>>>>     # flags only for the handlers.c source file (compile to an object
>>>>> file first and
>>>>>     # then link with the rest for librtemsbsp.a?)
>>>>>     librtemsbsp_a_CFLAGS = -mgeneral-regs-only
>>>>>
>>>>> The CFLAGS arg is required to allow us to use
>>>>> "__attribute__((interrupt))" to setup interrupt handlers in C. (See
>>>>> [1] and ctrl+f "interrupt" for more.)
>>>>>
>>>>> Is there a way to not force the CFLAGS for _all_ of librtemsbsp, but
>>>>> to limit it only to handlers.c?
>>>>>
>>>>> If not, is the above code something that would be acceptable to have
>>>>> upstream?
>>>>>
>>>>> [1]
>>>>> https://gcc.gnu.org/onlinedocs/gcc/x86-Function-Attributes.html#x86-Function-Attributes
>>>>
>>>>
>>>> Are we basically talking about the outermost layer of your interrupt
>>>> dispatching?
>>>>
>>>>
>>>> Have you looked at the basic approach taken by the other ports? They end
>>>> up switching the stack pointer to a dedicated stack on the outermost
>>>> interrupt
>>>> and, if a context switch/dispatch is needed, arrange for the interrupted
>>>> task to call _Thread_Dispatch.But tinker with its stack so some registers
>>>> are saved and it looks like it made the call itself.
>>>>
>>>> If you can do it in C, I am ok with an attribute. I just don't think you
>>>> can pull off all the stack and return to dispatch magic that way.
>>>>
>>>> --joel
>>>>
>>