[GSoC - x86_64] Interrupt manager and and port-specific glue - was Re: [GSoC - x86_64 - automake] Limit CFLAGS to specific source for librtemsbsp.a

Wed Aug 1 15:16:43 UTC 2018

On Wed, Aug 1, 2018 at 11:11 AM, Gedare Bloom <gedare at rtems.org> wrote:
> On Wed, Aug 1, 2018 at 9:15 AM, Amaan Cheval <amaan.cheval at gmail.com> wrote:
>> That's definitely very illuminating, thank you so much for all the details!
>>
>> A few more questions that have arisen for me. Feel free to skip over
>> them (I'll likely figure them out given enough time, so I'm only
>> asking in case any of them are obvious to anyone):
>>
>> - The i386 doesn't use CPU_Interrupt_frame at all. It seems like it
>> stores some of the data onto the stack?
>>
> the interrupt frame structure was introduced during 4.11 development.
> probably i386 never got updated to use a struct to encapsulate the
> interrupt frame. the interrupt frame should contain the registers that
> are preserved by the interrupt entry code I believe.
>
>> - There used to be defines in cpu.h regarding hardware/software based
>> interrupt stacks, and how they'd be setup, which were made
>> superfluous[1] - I'm not quite sure how these are meant to work - I
>> see references to "stack high" and "stack low" and I'm not quite sure
>> what the code is referencing when using those.
>>
>
> a hardware interrupt stack is one that the hardware switches to during
> an interrupt. i think m68k has such.
>
> most interrupt stacks in RTEMS are software-managed, meaning that
> RTEMS explicitly switches the stack region off the task stack and to
> an interrupt stack region.
>
> some stacks start high and grow down, and some stacks start low and
> grow up. maybe this is what the "stack high" and "stack low" you
> mention are in relation to?
>
>> - c/src/lib/libbsp/no_cpu/no_bsp/Makefile.am doesn't include
>> irq-sources.am, by the way (this is part of why I used to think a lot
>> of what your email mentioned was unnecessary, until you...ahem,
>> pre-empted that line of thought and helped clarify it :P). Should I
>> add a ticket to update the no_bsp code to be more in line with current
>> use?
>>
> Sure. I don't know that anyone is in particular maintaining
> no_cpu/no_bsp since we can't compile it, it is basically best effort
> stuff that sometimes we miss updating.
>
>> - My understanding of _ISR_Handler is that it'll be the handler for
>> _all_ interrupt vectors by default - it'll then dispatch interrupts to
>> user-handlers (or internal handlers, for the timer, for eg.). Is that
>> right? (I don't quite understand its interaction with the RTEMS
>> interrupt manager yet, but irq-generic's "bsp_interrupt_handler_table"
>> seems to be the RTEMS equivalent to the processor-specific vector
>> table, and "bsp_interrupt_handler_dispatch" seems to call the actual
>> handler within that table as appropriate. Accurate? (I just haven't
>> found how that table actually gets its handlers setup besides during
>> initialization, since rtems_interrupt_catch just calls
>> _CPU_install_vector, which updates the processor vector table, not the
>> RTEMS interrupt manager vector table.)
>>
>
> You have discovered a couple of different but related interrupt
> processing code bases.  I can see why you get confused.
>
> Basically, a CPU port should support two kinds of interrupts that may
> be installed, "RTEMS" and "Raw" interrupts. The "Raw" interrupts are
> installed directly in the processor's vector table. For processors
> that use simple vectored interrupts, the "RTEMS" interrupts install a
> call to the _ISR_Handler() function in the processor's vector table,
> and will put the user's isr function pointer in the
> _ISR_Vector_table(), which is the RTEMS Interrupt Manager's vector
> table.
>
> I'm not really familiar with the processors that use a different model
> for interrupt handling besides simple vectored. Probably, you will
> have to study one of them.
>
> This irq-generic bsp_interrupt_* code is used by the "IRQ Server" that
> builds from the CPU port capabilities to allow for some advanced
> features like chaining multiple isrs from the same source. I don't
> think you want to be focusing on those right now, but I could be
> mistaken. I haven't spent much time looking at the IRQ Server
> codebase.
>

On second thought, it appears that maybe the non-simple vectored
processors do use some of this bsp_interrupt stuff. Keep digging. :)

> Gedare
>
>> - My understanding of the interaction between RTEMS' interrupt manager
>> (i.e. support for nested interrupts and thread dispatch once an
>> interrupt ends) and the BSP's processor-specific interrupt manager
>> (code to use the APIC and IDT in my case) is that they're tied
>> together through the use of irq-generic.c's "bsp_interrupt_initialize"
>> - is that right? (m68k never seems to call it, though, so perhaps
>> not?)
>>
>> Sorry about the rambling! To reiterate, I'll likely figure it out
>> given enough time, so if the answers aren't at the top of your head, I
>> can figure it out without wasting your time :)
>>
>> [1] https://devel.rtems.org/ticket/3459#comment:11
>>
>> On Wed, Aug 1, 2018 at 3:18 AM, Joel Sherrill <joel at rtems.org> wrote:
>>>
>>>
>>> On Tue, Jul 31, 2018 at 3:05 PM, Amaan Cheval <amaan.cheval at gmail.com>
>>> wrote:
>>>>
>>>> Hm, I'm not sure what to look for in the other ports specifically, really.
>>>> The BSP porting documentation doesn't have a section on interrupts, so I'm
>>>> doing this on more of an "as it comes up" basis.
>>>>
>>>> What I've got right now (the interrupt handlers in C) are what I need for
>>>> calibrating the APIC timer (through the PIT) - so simply hooking IRQ0 (for
>>>> the timer) and IRQ7 (spurious vector), since those are needed for the timer
>>>> work to continue.
>>>>
>>>> What constitutes as a requirement for basic interrupt support?
>>>
>>>
>>> There used to be a generic porting guide. I can see that this particular
>>> section
>>> has bit rotted some but the interrupt dispatching section. Some of this
>>> will have evolved to support SMP and fine grained locking but the
>>> pseudo-code
>>> here will give you a push toward the right line of thinking:
>>>
>>> https://docs.rtems.org/releases/rtemsdocs-4.10.2/share/rtems/html/porting/porting00034.html
>>>
>>> The idea is that you need to ensure RTEMS knows it is inside an interrupt
>>> and the current locking scheme (old was dispatching, new is ...) is honored.
>>>
>>> The ARM and PowerPC (plus RISCV) are good ports to look at for how SMP
>>> plays into this. But the CPU supplement is thin for their interrupt
>>> processing.
>>>
>>>
>>> This is the CPU Architecture supplement section for the m68k. This is a
>>> relatively simple
>>> architecture to describe. There is also a section for the i386 which reads
>>> similarly.
>>>
>>> https://docs.rtems.org/branches/master/cpu-supplement/m68xxx_and_coldfire.html#interrupt-processing
>>>
>>> Personally, I find the m68k a fairly easy processor to read assembly in.
>>> Look at cpukit/score/cpu/m68k/cpu_asm.S and _ISR_Handler to see what
>>> is done there w/o SMP. On the m68k _ISR_Handler is directly put into the
>>> vector table. But this isn't the most similar example for you.
>>>
>>> For the i386 (better example), it is in bsps/i386/shared/irq/irq_asm.S with
>>> the
>>> same name. There _ISR_Handler is installed via the DISTINCT_INTERRUPT_ENTRY
>>> macros at the bottom of the file where some prologue jumps to the common
>>> _ISR_Handler and then the actions are similar. Usually _ISR_Handler type of
>>> code ends up invoking a PIC decode method in normal C without an
>>> interrupt attribute.
>>>
>>> Long and multi-architecture answer but maybe that makes sense. The goal
>>> in ticker.exe is to take a number of tick interrupts which don't schedule
>>> and
>>> then take one that does -- it schedules a preemption of the idle thread.
>>>
>>> Hope this helps.
>>>
>>> --joel
>>>
>>>
>>>
>>>>
>>>>
>>>> On Wed, Aug 1, 2018, 1:29 AM Joel Sherrill <joel at rtems.org> wrote:
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Jul 31, 2018 at 2:52 PM, Amaan Cheval <amaan.cheval at gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> Hi Chris!
>>>>>>
>>>>>> I currently have code like this in
>>>>>> c/src/lib/libbsp/x86_64/amd64/Makefile.am:
>>>>>>
>>>>>>     librtemsbsp_a_SOURCES +=
>>>>>> ../../../../../../bsps/x86_64/amd64/interrupts/handlers.c
>>>>>>     # XXX: Needed to use GCC "interrupt" attribute directives - can we
>>>>>> pass these
>>>>>>     # flags only for the handlers.c source file (compile to an object
>>>>>> file first and
>>>>>>     # then link with the rest for librtemsbsp.a?)
>>>>>>     librtemsbsp_a_CFLAGS = -mgeneral-regs-only
>>>>>>
>>>>>> The CFLAGS arg is required to allow us to use
>>>>>> "__attribute__((interrupt))" to setup interrupt handlers in C. (See
>>>>>> [1] and ctrl+f "interrupt" for more.)
>>>>>>
>>>>>> Is there a way to not force the CFLAGS for _all_ of librtemsbsp, but
>>>>>> to limit it only to handlers.c?
>>>>>>
>>>>>> If not, is the above code something that would be acceptable to have
>>>>>> upstream?
>>>>>>
>>>>>> [1]
>>>>>> https://gcc.gnu.org/onlinedocs/gcc/x86-Function-Attributes.html#x86-Function-Attributes
>>>>>
>>>>>
>>>>> Are we basically talking about the outermost layer of your interrupt
>>>>> dispatching?
>>>>>
>>>>>
>>>>> Have you looked at the basic approach taken by the other ports? They end
>>>>> up switching the stack pointer to a dedicated stack on the outermost
>>>>> interrupt
>>>>> and, if a context switch/dispatch is needed, arrange for the interrupted
>>>>> task to call _Thread_Dispatch.But tinker with its stack so some registers
>>>>> are saved and it looks like it made the call itself.
>>>>>
>>>>> If you can do it in C, I am ok with an attribute. I just don't think you
>>>>> can pull off all the stack and return to dispatch magic that way.
>>>>>
>>>>> --joel
>>>>>
>>>