Including paravirtualization headers in RTEMS

Mon May 26 00:36:00 UTC 2014

On 26/05/2014 4:39 am, Cláudio Silva wrote:
> Sorry all, miss send. Returning to where I was:
>
> Yes, Partitions are fully linked executables created "off-line". No
> linking occurs in run-time. The partition elf is simply loaded in to
> memory by a partition loader or by the bootloader.
>
> Typically, you have a thin abstraction layer defining simple functions
> that will trap into the partitioning kernel. Something like:
>
> https://github.com/t-crest/ospat/blob/master/libpok/middleware/portqueueingsend.c
>

Thanks. This lines up with my understand.

I think RTEMS needs to view POK or any virtual environment as a black 
box and we cannot assume anything about the environment other than the 
ABI presented to us. We cannot look through to the host system and do 
things as a result. Maintaining a strict black box view may not be 
possible however we should start with the black box and move to a 
position that is practical and possible.

We should not borrow (reference in the POK source tree) header files to 
pick up common definitions directly or indirectly via static libraries. 
Any interfaces defined by the partition's host need to be formally 
exported from the project and made available. This defines the ABI we 
are working against. I see POK is highly configurable so this 
complicates what is exported.

If I build POK with port queueing enabled and build a guest that uses it 
then I reconfigure POK and remove port queueing however I do not rebuild 
the guest will POK detect the problem and handle the mismatch ?

> In POK + RTEMS case, this library is being compiled by POK's compiler
> and then linked to each partitions' executable.  When this partition is
> an RTEMS one, the remaining binaries (RTEMS+Application) were compiled
> with the RTEMS' compiler. Therefore you are linking two sets of
> binaries produced by different compilers.

We cannot do this. Given the type of applications expected to use POK I 
would expect an audit to flag this as a fail. There are a number of 
reasons why this is not a sustainable path forward, the most simple one 
is having to have code in the RTEMS partition built with a compiler we 
do not control. In this example the dependency is backwards, for example 
will the POK project make sure its compiler can always build the 
interface and work with RTEMS, ie all tests pass. I suspect the POK 
maintainers will have limited or no real interest in RTEMS's development 
process and I see this as normal and expected. I do not see this path as 
being sustainable and I do not support it. As a hack to get things going 
it has been perfectly fine.

>
> I guess this is also true for the recently submitted Xtratum BSP.
>

Current Xtratum BSP is not going to happen so lets put that one to one side.

 From the RTEMS point of view I see POK as no different to any 
architecture we support. We support running on hard IP in a device, for 
example a Blackfin, a mix of hard IP and soft IP, for example a Zynq, 
soft IP, for example a NIOS2, hard IP (or mixed soft IP) and software 
intervention in a virtual environment, for example POK, and full 
software emulation via simulation, for example qemu.

For RTEMS to support an architecture we need separation so each part can 
vary in version, the version dependencies flow down to RTEMS and not the 
other direction, and we need a way to test. Testing means we have a 
stable version of POK built on our test system and we build RTEMS 
against that.

> We have several options available:
> - Let the two binaries (RTEMS + abstraction lib), compiled with
> different compilers, to be linked together (how it is currently
> happening)

No.

> - Compile the partitioning kernel abstraction layer with the RTEMS
> compiler - this will result in a complex configuration and build
> procedure.

This is possible however is depends on how this is implement and where 
the code resides.

If the code lives inside the POK source tree and references files that 
are core to POK we have an issue. This approach would place a burden on 
the POK project to have a current set of RTEMS tools for the 
architectures supported and to test the builds work. It means POK 
maintainers would need to monitor and track the RTEMS development 
process. The dependency here is from RTEMS to POK and as I mentioned 
before the dependence should be the other away around.

The code could be a separate project or sub-project of POK and both 
projects need to maintain it. The POK project would need to maintain the 
ABI and RTEMS would need to maintain the dependency with RTEMS tools and 
RTEMS. For RTEMS how we interface to this because the next issue to solve.

> - Implement the functionality of the abstraction layer completely
> inside RTEMS as part of the libbsp/libcpu - This can be a good
> solution for the CPU/TImer/Interrupt part of the abstraction layer.

If the POK interface is stable and the header dependency can be made 
thin this works for me. In the ideal world if something changes in the 
POK ABI they would publish an update and we would import it. There would 
be versioning issues, ie backwards compatibility, and this is normal and 
nothing new for us.

> However, for other elements like the queueing and sampling port APIs
> that have no match inside RTEMS...

Could you please elaborate a little more ? Are these not supported 
because RTEMS application's would not use them or this is still to be done ?

Chris

> -
>
>
> Best Regards,
> Cláudio
>
> On Sun, May 25, 2014 at 7:22 PM, Cláudio Silva <claudiodcsilva at gmail.com> wrote:
>> Hello Chris,
>>
>> Yes, Partitions are fully linked executables created "off-line". No
>> linking occurs in run-time. The partition elf is simply loaded in to
>> memory by a partition loader or the bootloader.
>>
>> Typically, you have a thin abstraction layer defining simple functions
>> that will trap into the partitioning kernel. Something like:
>>
>>
>> On Sat, May 24, 2014 at 12:27 AM, Chris Johns <chrisj at rtems.org> wrote:
>>> On 23/05/2014 6:46 pm, Philipp Eppelt wrote:
>>>>>
>>>>> On 05/21/2014 03:54 PM, Gedare Bloom wrote:> On Wed, May 21, 2014 at 4:04
>>>>> AM, Christian Mauderer
>>>>>>
>>>>>> <Christian.Mauderer at embedded-brains.de> wrote:
>>>>>>>
>>>>>>> First of all: Thanks for your comments. You will find answers below.
>>>>>>>
>>>>>>> Am 20.05.2014 16:58, schrieb Gedare Bloom:
>>>>>>>
>>>>>>> That is correct. It's part of XtratuM. Is there some preferred way of
>>>>>>> marking such headers?
>>>>>>>
>>>>>> Not that I know of. We have discussed a similar issue with the POK
>>>>>> paravirtualization project. The problem is to allow external code
>>>>>> linking to RTEMS. The design should be considered carefully and
>>>>>> probably discussed in a separate thread.
>>>>
>>>>
>>>> To pick up this discussion, I like first to picture the problem, second
>>>> to collect possible solutions and third name some known problems with
>>>> the solutions.
>>>>
>>>
>>> Thank you for posting this email. I followed the links and looked at the
>>> code and realised I need to ask more questions.
>>>
>>>
>>>>
>>>> In a paravirtualized environment we need to make function calls from our
>>>> guest system to the host system. The implementation of these functions
>>>> rely sole on the host system.
>>>
>>>
>>> Are they function calls ? To me a function call assumes some sort of linking
>>> process between the guest and the host system either static or via some form
>>> of runtime binding, ie a link editor. I also assume a guest is an RTEMS
>>> partition executable and the host is the POK microkernel providing the
>>> partitioned environment in some sort of protective environment.
>>>
>>>
>>>> Hence, at compile time the guest system is missing the implementation of
>>>> this host specific functions.
>>>
>>>
>>> My understanding is partitions are loaded into an ELF image that POK loads
>>> and executes. Is there some sort of linking happening when the partition is
>>> written into the ELF image or when it is being loaded ?
>>>
>>> For me this is the really important because I am wanting to first understand
>>> the which pieces of code are compiled into object files and by which
>>> compilers and then archived into static libraries and then linked to any
>>> RTEMS code compiled with an RTEMS compiler. I do not understand if there are
>>> function calls into the POK kernel or syscall type calls which I consider an
>>> ABI, ie a trap or VM exception.
>>>
>>> If I can understand these needs I will be in a better position to help with
>>> the remaining topics in this email.
>>>
>>> Chris
>>>
>>>
>>>>
>>>> To provide the guest system with the function implementation there are a
>>>> couple of possible solutions.
>>>>
>>>> (A) Provide a host library
>>>> The host compiles a library containing the function implementations
>>>> including all dependencies. This host library is passed to the guest
>>>> system, to resolve the `undefined reference` errors during link time.
>>>> In the case of POK+RTEMS the resulting RTEMS binary is a valid POK
>>>> partition and can run without further modifications.
>>>> The recently propose XantruM BSP seems to follow a similar approach.
>>>>
>>>> (B) Use function stubs and build a jump table in the host
>>>> The guest system makes use of function stubs, written into a special
>>>> section in the binary to build. This section is analysed by the host
>>>> system to intercept function calls  and replace them with the host
>>>> implementation of the calls.
>>>> This approach is used in L4Linux and I gave it a shot in L4RTEMS.
>>>> [1] shows the macro for the function stubs, [2] the hosts resolver part.
>>>>
>>>> (C) add your solution
>>>>
>>>>
>>>> Problems with the solutions
>>>>
>>>> P(A) Dependencies and foreign code
>>>> The library is host code, which has to be maintained, which can get out
>>>> of sync between BSP and host. Licensing is another question.
>>>> To my knowledge this approach won't get merged upstream.
>>>>
>>>> P(B) Support by the host system
>>>> The host system needs a piece of code, loading the guest binary and
>>>> providing it with knowledge about the environment (external_resolver).
>>>> We haven't discussed this approach in detail.
>>>>
>>>>
>>>> I hope, this gives a good overview and we can have a fruitful discussion.
>>>>
>>>> Cheers,
>>>> Philipp
>>>>
>>>>
>>>> [1]
>>>>
>>>> https://github.com/phipse/L4RTEMS/blob/master/c/src/lib/libbsp/l4vcpu/pc386/startup/l4lib.h
>>>> [2]
>>>>
>>>> https://github.com/phipse/L4RTEMS/blob/f5a32ed0121b3e5f0ac8ae4c53e636eab2257fb1/l4/pkg/RTEMS_wrapper/server/src/res.c
>>>> _______________________________________________
>>>> rtems-devel mailing list
>>>> rtems-devel at rtems.org
>>>> http://www.rtems.org/mailman/listinfo/rtems-devel
>>>>
>>> _______________________________________________
>>> rtems-devel mailing list
>>> rtems-devel at rtems.org
>>> http://www.rtems.org/mailman/listinfo/rtems-devel