CAmkES VM rootfs.cpio with Python3 Causes Error
Hi, I have been attempting to create a VM to run as a CAmkES component, and it is a requirement that it has Python3. I have been using the vm_minimal app as a template to get started, and was successful at getting my own buildroot filesystem image running, however when I attempted to add the Python3 package, the resulting image resulted in the following error when it booted: ... <<seL4(CPU 0) [decodeInvocation/631 T0xfec34400 "rootserver" @106f0]: Attempted to invoke a null cap> create_objects@main.c:805 [Err seL4_InvalidCapability]: Untyped retype failed with unexpected error seL4 root server abort()ed Debug halt syscall from user thread 0xfec34400 "rootserver" halting... Kernel entry via Unknown syscall, word: 1 The only change I made to the image was to add the Python3 package, and that has caused this error. Is there something the vm_minimal app is missing that is required in order to get Python3 working? Kind regards, Ben Turner ________________________________________ Roke Manor Research Limited, Romsey, Hampshire, SO51 0ZN, United Kingdom.Part of the Chemring Group. Registered in England & Wales. Registered No: 00267550 http://www.roke.co.uk _______________________________________ The information contained in this e-mail and any attachments is proprietary to Roke Manor Research Limited and must not be passed to any third party without permission. This communication is for information only and shall not create or change any contractual relationship. ________________________________________
Hi Ben, My guess is that there's something fishy going on with the bootinfo that is passed into the capDL loader application which initialises the CAmkES system. I don't have much information to go on, so I need to ask you to do a favour for me. Is it possible if you can re-run your application with the log level turned to verbose and paste the output to pastebin or similar? For convenience, you can try applying the following diff patch to the 'capdl' repository in the 'projects' folder of the project's root directory to turn the log level to verbose. diff --git capdl-loader-app/src/main.c capdl-loader-app/src/main.c index 38c3269..bdbc82e 100644 --- capdl-loader-app/src/main.c +++ capdl-loader-app/src/main.c @@ -10,6 +10,8 @@ * @TAG(DATA61_BSD) */ +#define ZF_LOG_LEVEL ZF_LOG_VERBOSE + #include <autoconf.h> #include <capdl_loader_app/gen_config.h> Also, can you please turn the CMake variable 'CapDLLoaderPrintCapDLObjects' on in the build directory using ccmake or similar. I'm not sure if this may help or not but, if you are using QEMU, perhaps try increasing the amount of memory available to QEMU. Regards, Damon ________________________________ From: Devel <devel-bounces@sel4.systems> on behalf of Turner, Ben <ben.turner@roke.co.uk> Sent: Wednesday, 20 May 2020 5:18 PM To: devel@sel4.systems <devel@sel4.systems> Subject: [seL4] CAmkES VM rootfs.cpio with Python3 Causes Error Hi, I have been attempting to create a VM to run as a CAmkES component, and it is a requirement that it has Python3. I have been using the vm_minimal app as a template to get started, and was successful at getting my own buildroot filesystem image running, however when I attempted to add the Python3 package, the resulting image resulted in the following error when it booted: ... <<seL4(CPU 0) [decodeInvocation/631 T0xfec34400 "rootserver" @106f0]: Attempted to invoke a null cap> create_objects@main.c:805 [Err seL4_InvalidCapability]: Untyped retype failed with unexpected error seL4 root server abort()ed Debug halt syscall from user thread 0xfec34400 "rootserver" halting... Kernel entry via Unknown syscall, word: 1 The only change I made to the image was to add the Python3 package, and that has caused this error. Is there something the vm_minimal app is missing that is required in order to get Python3 working? Kind regards, Ben Turner ________________________________________ Roke Manor Research Limited, Romsey, Hampshire, SO51 0ZN, United Kingdom.Part of the Chemring Group. Registered in England & Wales. Registered No: 00267550 http://www.roke.co.uk _______________________________________ The information contained in this e-mail and any attachments is proprietary to Roke Manor Research Limited and must not be passed to any third party without permission. This communication is for information only and shall not create or change any contractual relationship. ________________________________________ _______________________________________________ Devel mailing list Devel@sel4.systems https://sel4.systems/lists/listinfo/devel
Hi Damon, Thanks for getting back to me. For clarity, I am attempting to get this working on an Odroid XU4. This has been working generally, however I noticed that the overlay dts file doesn't seem to map the full memory region that the linux kernel and u-boot assume to be there. In the DTS file provided<https://github.com/hardkernel/linux/blob/odroidn2-4.9.y/arch/arm/boot/dts/exynos5422-odroidxu3-common.dtsi> by hardkernel linux the following memory region is defined: memory@40000000 { device_type = "memory"; reg = <0x40000000 0x7EA00000>; }; And from the U-Boot DTS file<https://github.com/hardkernel/u-boot/blob/odroidxu4-v2017.05/arch/arm/dts/exynos5422-odroidxu4.dts>, essentially the same region is defined as: memory { device-type = "memory"; reg = <0x40000000 0x10000000 0x50000000 0x10000000 0x60000000 0x10000000 0x70000000 0x10000000 0x80000000 0x10000000 0x90000000 0x10000000 0xa0000000 0x10000000 0xb0000000 0xea00000>; } However the overlay dts file<https://github.com/seL4/seL4/blob/master/src/plat/exynos5/overlay-exynos5422.dts> provided for the XU4 in the seL4 kernel only reflects a small amount of this memory region: /* HACK: 0xe0000000..0xff000000 is the largest contiguous region * in the kernel window; we clamp to that and discard memory * after the ASID PD hole (0xff200000..0xfff00000). This is a * workaround for userspace tools (hardware_gen, elfloader, etc) * which are not yet aware of the memory hole. */ memory@40000000 { reg = <0x60000000 0x1f000000>; }; vm-memory@40000000 { reg = <0x40000000 0x20000000>; }; Q: Why are there two separate memory regions defined here? Q: Why is the bulk of the memory provided by the device missing from this file? Q: The comment that states that this file contains a hack to get around an ASID memory hole, however I don't understand how that hole can exist on this platform, as the memory is supposed to be contiguous from 0x40000000 to 0xbea00000? Q: I also don't understand how the addresses stated in the comment have anything to do with the XU4's memory region, as the addresses referenced are far higher than the 0xbea00000 address at the end of the XU4's memory region? Q: How is the kernel load address determined? From my investigations it seems this is simply the first memory region defined in the overlay file (which may explain why there are multiple) however when I attempt to change that region to anything other than 0x60000000 it doesn't seem to work. I have added the missing memory regions manually, and have had varying success in using different memory regions. For example, I have been able to declare memory from 0x80000000 and up, and specify that as the VM_RAM_BASE which works (wihout python3 installed). Q: Out of the available memory region (0x40000000 - 0xbea00000), what can I use and what constrains the kernel load address within this available memory? I have done as you asked and the resulting boot sequences can be found at: * No-Python<https://pastebin.com/Gnp4bEmG> * With-Python<https://pastebin.com/ZGvsxzFF> The only thing that I can see from this extra logging that may hint at something not being quite right is on line 756 where it attempts to create an object from untyped 0... I appreciate your help, and hopefully you can shed some light on the other questions I have raised. Kind regards, Ben Turner ________________________________________ Roke Manor Research Limited, Romsey, Hampshire, SO51 0ZN, United Kingdom.Part of the Chemring Group. Registered in England & Wales. Registered No: 00267550 http://www.roke.co.uk _______________________________________ The information contained in this e-mail and any attachments is proprietary to Roke Manor Research Limited and must not be passed to any third party without permission. This communication is for information only and shall not create or change any contractual relationship. ________________________________________ From: Lee, Damon (Data61, Kensington NSW) <Damon.Lee@data61.csiro.au> Sent: 26 May 2020 06:29 To: Turner, Ben <ben.turner@roke.co.uk>; devel@sel4.systems Subject: Re: CAmkES VM rootfs.cpio with Python3 Causes Error Hi Ben, My guess is that there's something fishy going on with the bootinfo that is passed into the capDL loader application which initialises the CAmkES system. I don't have much information to go on, so I need to ask you to do a favour for me. Is it possible if you can re-run your application with the log level turned to verbose and paste the output to pastebin or similar? For convenience, you can try applying the following diff patch to the 'capdl' repository in the 'projects' folder of the project's root directory to turn the log level to verbose. diff --git capdl-loader-app/src/main.c capdl-loader-app/src/main.c index 38c3269..bdbc82e 100644 --- capdl-loader-app/src/main.c +++ capdl-loader-app/src/main.c @@ -10,6 +10,8 @@ * @TAG(DATA61_BSD) */ +#define ZF_LOG_LEVEL ZF_LOG_VERBOSE + #include <autoconf.h> #include <capdl_loader_app/gen_config.h> Also, can you please turn the CMake variable 'CapDLLoaderPrintCapDLObjects' on in the build directory using ccmake or similar. I'm not sure if this may help or not but, if you are using QEMU, perhaps try increasing the amount of memory available to QEMU. Regards, Damon ________________________________ From: Devel <devel-bounces@sel4.systems<mailto:devel-bounces@sel4.systems>> on behalf of Turner, Ben <ben.turner@roke.co.uk<mailto:ben.turner@roke.co.uk>> Sent: Wednesday, 20 May 2020 5:18 PM To: devel@sel4.systems<mailto:devel@sel4.systems> <devel@sel4.systems<mailto:devel@sel4.systems>> Subject: [seL4] CAmkES VM rootfs.cpio with Python3 Causes Error Hi, I have been attempting to create a VM to run as a CAmkES component, and it is a requirement that it has Python3. I have been using the vm_minimal app as a template to get started, and was successful at getting my own buildroot filesystem image running, however when I attempted to add the Python3 package, the resulting image resulted in the following error when it booted: ... <<seL4(CPU 0) [decodeInvocation/631 T0xfec34400 "rootserver" @106f0]: Attempted to invoke a null cap> create_objects@main.c:805<mailto:create_objects@main.c:805> [Err seL4_InvalidCapability]: Untyped retype failed with unexpected error seL4 root server abort()ed Debug halt syscall from user thread 0xfec34400 "rootserver" halting... Kernel entry via Unknown syscall, word: 1 The only change I made to the image was to add the Python3 package, and that has caused this error. Is there something the vm_minimal app is missing that is required in order to get Python3 working? Kind regards, Ben Turner ________________________________________ Roke Manor Research Limited, Romsey, Hampshire, SO51 0ZN, United Kingdom.Part of the Chemring Group. Registered in England & Wales. Registered No: 00267550 http://www.roke.co.uk<http://www.roke.co.uk> _______________________________________ The information contained in this e-mail and any attachments is proprietary to Roke Manor Research Limited and must not be passed to any third party without permission. This communication is for information only and shall not create or change any contractual relationship. ________________________________________ _______________________________________________ Devel mailing list Devel@sel4.systems<mailto:Devel@sel4.systems> https://sel4.systems/lists/listinfo/devel<https://sel4.systems/lists/listinfo/devel>
Hi Ben, Sorry, it took so long to get back to you, I was quite busy and I had to do a little research for your questions. I'm not confident I'm correct in some of these answers as I mostly work above the kernel. Q: Why are there two separate memory regions defined here? *and* Q: Out of the available memory region (0x40000000 - 0xbea00000), what can I use and what constrains the kernel load address within this available memory? A: The 'memory' block is used for normal applications and the 'vm-memory' block is memory used exclusively for the VMs. As I recall, the VM memory region is as it is due to some limitations of Linux or perhaps the memory controller of the platform in regards to memory interactions with the platform peripherals. I'm not too sure on this, and perhaps someone else more familiar with the VMs can chime in. Q: The comment that states that this file contains a hack to get around an ASID memory hole, however I don't understand how that hole can exist on this platform, as the memory is supposed to be contiguous from 0x40000000 to 0xbea00000? A: The hole is mostly a kernel implementation issue. The kernel window for the kernel ELF and addressable memory is 0xe0000000 to 0xff000000. Addresses after this is reserved for other things, hardware ASIDs, devices, vectors, etc. Also it just so happens that the overlay contains a typo, the ASID memory hole is actually starts at 0xff000000. Q: Why is the bulk of the memory provided by the device missing from this file? *and* Q: I also don't understand how the addresses stated in the comment have anything to do with the XU4's memory region, as the addresses referenced are far higher than the 0xbea00000 address at the end of the XU4's memory region? A: Due to memory limitations of the kernel. The whole memory region is given to the kernel and is mapped at a specific address. The kernel is mapped at 0xe0000000 on Exynos5422 (odroid-xu4) platforms. There's a difference of 0x1f000000 bytes between that and the top of the largest contiguous region in the kernel window. The kernel only creates normal untyped memory from physical memory that it can access virtually hence the need to clamp the memory region. I'm not too sure if this is exactly the case, and hopefully someone with more knowledge about the kernel can expand on this more. Q: How is the kernel load address determined? From my investigations it seems this is simply the first memory region defined in the overlay file (which may explain why there are multiple) however when I attempt to change that region to anything other than 0x60000000 it doesn't seem to work. I have added the missing memory regions manually, and have had varying success in using different memory regions. For example, I have been able to declare memory from 0x80000000 and up, and specify that as the VM_RAM_BASE which works Regards, Damon
Hi Damon, Just to update something I have discovered since the last email I sent - I have managed to boot with the kernel memory region set to <0xa0000000 0x1ea00000> so it seems the kernel can be put elsewhere, not sure why it failed the last time I attempted to move it. Otherwise my questions still stand, and python3 still breaks things. Thanks, Ben Turner ________________________________________ Roke Manor Research Limited, Romsey, Hampshire, SO51 0ZN, United Kingdom.Part of the Chemring Group. Registered in England & Wales. Registered No: 00267550 http://www.roke.co.uk _______________________________________ The information contained in this e-mail and any attachments is proprietary to Roke Manor Research Limited and must not be passed to any third party without permission. This communication is for information only and shall not create or change any contractual relationship. ________________________________________
UPDATE: I have managed to get past the first hurdle of the error that I originally posted about here. This was caused by me increasing the wrong memory region. It was actually the kernel memory region that needed additional space, not the VM memory region. I am now able to get a rootfs.cpio image with Python3 installed past the untyped-retype error my previous posts allude to, however I now have a kernel panic occurring during the linux boot process of the virtual machine. The logs for this can be found at https://pastebin.com/BPhKpyD9 Any clue what the issue could be here? Thanks, Ben ________________________________________ Roke Manor Research Limited, Romsey, Hampshire, SO51 0ZN, United Kingdom.Part of the Chemring Group. Registered in England & Wales. Registered No: 00267550 http://www.roke.co.uk _______________________________________ The information contained in this e-mail and any attachments is proprietary to Roke Manor Research Limited and must not be passed to any third party without permission. This communication is for information only and shall not create or change any contractual relationship. ________________________________________
participants (2)
-
Lee, Damon (Data61, Kensington NSW) -
Turner, Ben