I'm using the docker image to build the CAmkES ARM VMM project using roughly the instructions here: https://docs.sel4.systems/VM/CAmkESARMVM.html (they are slightly out of date). I do: ../init-build.sh -DAARCH32=TRUE -DCAMKES_VM_APP=vm_minimal -DPLATFORM=tk1 ninja Then I copy the resulting image from the images directory to an SD card and put that in my TK1. When U-Boot starts I use the following commands to try to boot fatload mmc 1 0x10000000 capdl-loader-image-arm-tk1 bootelf 0x10000000 To which I get an error: No elf image at address 0x10000000 I tried with an older version of U-Boot (that worked following this same procedure about a year ago). It is U-Boot SPL 2014.10-rc2-g3127911 (Jun 07 2016 - 21:00:01) I also tried updating U-Boot to U-Boot SPL 2020.01-00442-gc00bd81ae0 (Jan 10 2020 - 11:10:18 -0500). Same error. Any thoughts or suggestions on how I get this to boot?
Mike, Can you do a readelf of your outputted image to make sure its not compiling as a binary? If it is, you’ll have to use the “go” command in u-boot instead. Thanks, Chris Guikema DornerWorks From: Devel <devel-bounces@sel4.systems> On Behalf Of Mike Clark Sent: Friday, January 10, 2020 3:22 PM To: devel@sel4.systems Subject: [seL4] Problem booting camkes arm vmm on TK1 from SD Card CAUTION: This email originated from outside the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. I'm using the docker image to build the CAmkES ARM VMM project using roughly the instructions here: https://docs.sel4.systems/VM/CAmkESARMVM.html (they are slightly out of date). I do: ../init-build.sh -DAARCH32=TRUE -DCAMKES_VM_APP=vm_minimal -DPLATFORM=tk1 ninja Then I copy the resulting image from the images directory to an SD card and put that in my TK1. When U-Boot starts I use the following commands to try to boot fatload mmc 1 0x10000000 capdl-loader-image-arm-tk1 bootelf 0x10000000 To which I get an error: No elf image at address 0x10000000 I tried with an older version of U-Boot (that worked following this same procedure about a year ago). It is U-Boot SPL 2014.10-rc2-g3127911 (Jun 07 2016 - 21:00:01) I also tried updating U-Boot to U-Boot SPL 2020.01-00442-gc00bd81ae0 (Jan 10 2020 - 11:10:18 -0500). Same error. Any thoughts or suggestions on how I get this to boot?
Chris, Looks like I might have spoken too soon. When I use go instead of bootelf, the board actually resets. Here are the log messages: Tegra124 (Jetson TK1) # fatload mmc 1 0x82000000 capdl-loader-image-arm-tk1 20435184 bytes read in 964 ms (20.2 MiB/s) Tegra124 (Jetson TK1) # go 0x82000000 ## Starting application at 0x82000000 ... data abort pc : [<82000110>] lr : [<fff51d84>] reloc pc : [<021c3110>] lr : [<80114d84>] sp : fd7f8610 ip : 00000002 fp : fff6be40 r10: 00000002 r9 : fd7ffed0 r8 : fffcbe9c r7 : fda60420 r6 : 82000000 r5 : 00000001 r4 : 00000000 r3 : 82000000 r2 : fda60424 r1 : fda60424 r0 : 00006090 Flags: nzCv IRQs off FIQs off Mode SVC_32 Code: 00000000 00000000 00000000 42100040 (7865742e) Resetting CPU ... resetting ... U-Boot SPL 2020.01-00442-gc00bd81ae0 (Jan 10 2020 - 11:10:18 -0500) Trying to boot from RAM U-Boot 2020.01-00442-gc00bd81ae0 (Jan 10 2020 - 11:10:18 -0500) SoC: tegra124 Reset cause: SW_MAIN Model: NVIDIA Jetson TK1 Board: NVIDIA Jetson TK1 DRAM: 2 GiB MMC: sdhci@700b0400: 1, sdhci@700b0600: 0 Loading Environment from MMC... OK In: serial Out: serial Err: serial Net: No ethernet found. Hit any key to stop autoboot: 0 Tegra124 (Jetson TK1) # Doing readelf on the image that is being built doesn't work either $ readelf -h capdl-loader-image-arm-tk1 readelf: Error: Not an ELF file - it has the wrong magic bytes at the start My old images from a year ago still boot just fine with bootelf. Any thoughts? Mike On Fri, Jan 10, 2020 at 3:46 PM Mike Clark <undefinedspace@gmail.com> wrote:
Hi Mike, Could you run "file images/capdl-loader-image-arm-tk1"? If you see "images/capdl-loader-image-arm-tk1: MS-DOS executable", the image is an EFI image, so bootefi command should help. Also, the link, https://docs.sel4.systems/Hardware/jetsontk1.html, may be helpful. Regards, Yanyan On Fri, 2020-01-24 at 15:24 -0500, Mike Clark wrote:
Thanks. I checked the output of file and it does list it as MS-DOS executable. Now I tried # fatload mmc 1 ${loadaddr} capdl-loader-image-arm-tk1 # bootefi ${loadaddr} After that I get nothing. The system just sits there and I get no further output. Anything else I can try? Is there any way to change the type of image that is getting created? I never had any issues with bootelf before and my old elf images still work. Mike On Sat, Jan 25, 2020 at 10:03 AM Shen, Yanyan (Data61, Kensington NSW) < Yanyan.Shen@data61.csiro.au> wrote:
Mike, Do you happen to have the “ApplyData61ElfloaderSettings()” command in your project files? function(ApplyData61ElfLoaderSettings kernel_platform kernel_sel4_arch) set( binary_list "tx1;hikey;am335x;odroidc2;imx8mq-evk;rockpro64;imx8mm-evk;hifive" ) set(efi_list "tk1") set(uimage_list "tx2") That sets any compiled images for the tk1 to be efi. You can remove the “tk1” from the “efi_list” and running “ccmake .” should allow you to choose the output setting as an elf. I’ve seen some issues where efi images don’t boot properly, but not with u-boot, so I can’t be sure your exact problem. Thanks, Chris Guikema DornerWorks From: Mike Clark <undefinedspace@gmail.com> Sent: Monday, January 27, 2020 8:53 AM To: Shen, Yanyan (Data61, Kensington NSW) <Yanyan.Shen@data61.csiro.au> Cc: Chris Guikema <Chris.Guikema@dornerworks.com>; devel@sel4.systems Subject: Re: [seL4] Problem booting camkes arm vmm on TK1 from SD Card CAUTION: This email originated from outside the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. Thanks. I checked the output of file and it does list it as MS-DOS executable. Now I tried # fatload mmc 1 ${loadaddr} capdl-loader-image-arm-tk1 # bootefi ${loadaddr} After that I get nothing. The system just sits there and I get no further output. Anything else I can try? Is there any way to change the type of image that is getting created? I never had any issues with bootelf before and my old elf images still work. Mike On Sat, Jan 25, 2020 at 10:03 AM Shen, Yanyan (Data61, Kensington NSW) <Yanyan.Shen@data61.csiro.au<mailto:Yanyan.Shen@data61.csiro.au>> wrote: Hi Mike, Could you run "file images/capdl-loader-image-arm-tk1"? If you see "images/capdl-loader-image-arm-tk1: MS-DOS executable", the image is an EFI image, so bootefi command should help. Also, the link, https://docs.sel4.systems/Hardware/jetsontk1.html, may be helpful. Regards, Yanyan On Fri, 2020-01-24 at 15:24 -0500, Mike Clark wrote:
Hi Chris, Thank you for the suggestion. I removed tk1 from the efi_list (so now it is the empty string). I rebuilt everything. Now when I run file on the resulting image I get: capdl-loader-image-arm-tk1: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, with debug_info, not stripped Running readelf -h on the file returns: ELF Header: Magic: 7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00 Class: ELF32 Data: 2's complement, little endian Version: 1 (current) OS/ABI: UNIX - System V ABI Version: 0 Type: EXEC (Executable file) Machine: ARM Version: 0x1 Entry point address: 0x817f0000 Start of program headers: 52 (bytes into file) Start of section headers: 20647556 (bytes into file) Flags: 0x5000200, Version5 EABI, soft-float ABI Size of this header: 52 (bytes) Size of program headers: 32 (bytes) Number of program headers: 2 Size of section headers: 40 (bytes) Number of section headers: 20 Section header string table index: 19 I copied that image to the SD card and tried to boot with fatload followed by bootelf. It sits there doing nothing for a little while (maybe a minute) then resets the CPU: Tegra124 (Jetson TK1) # bootelf ${loadaddr} ## Starting application at 0x817f0000 ... data abort pc : [<817f7590>] lr : [<817f7650>] reloc pc : [<019ba590>] lr : [<019ba650>] sp : 81809f98 ip : 00000002 fp : 81809fb4 r10: 00000000 r9 : fd7ffed0 r8 : 00000014 r7 : fda60610 r6 : 00000000 r5 : 823b117c r4 : 81000000 r3 : ffffffff r2 : ffffefff r1 : fda60610 r0 : 82b7aa40 Flags: Nzcv IRQs off FIQs off Mode SVC_32 Code: e50b300c e51b3018 e5932000 e51b300c (e5933000) Resetting CPU ... resetting ... U-Boot SPL 2020.01-00442-gc00bd81ae0 (Jan 10 2020 - 11:10:18 -0500) Trying to boot from RAM This is all with basically a fresh clone of the camkes-arm-vm-manifest git project then doing "../init-build.sh -DAARCH32=TRUE -DTk1Insecure=TRUE -DCAMKES_VM_APP=vm_minimal -DPLATFORM=tk1" then "ninja" in the build folder. Mike On Mon, Jan 27, 2020 at 9:58 AM Chris Guikema <Chris.Guikema@dornerworks.com> wrote:
One update on this. I pulled the sel4test-manifest project and built that for the TK1. When it built the efi image, I wasn't able to get it to work on my TK1 device. When I made the changes to the settings and got it to build the elf file instead, the tests ran just fine. I did double check to make sure I had done this. I am still not able to boot the ARM VMM. setenv bootm_boot_mode nonsec saveenv On Mon, Jan 27, 2020 at 10:41 AM Mike Clark <undefinedspace@gmail.com> wrote:
Hi all, One additional update. I noticed updates to the camkes-arm-vm-manifest project today, so I pulled those in and tried a few things. I still have EFI turned off in the build. When I do init-build, if I have -DTk1Insecure=TRUE set, then when I try to boot the resulting image, I noticed that the cpu resets almost immediately. On the other hand, if I remove that flag, then when I try to boot the resulting image, I get nothing on the screen at all after issuing the bootelf command. Not sure if that helps trigger any thoughts. I'm at a loss for how to get the latest versions to boot. Mike On Mon, Jan 27, 2020 at 11:27 AM Mike Clark <undefinedspace@gmail.com> wrote:
Hi Mike, Here is a copy of an EFI binary for the camkes-arm-vm tk1 project that runs on our tk1 development board internally: https://cloudstor.aarnet.edu.au/plus/s/FHVMZi0ei1Q6IYA. A reason why ELF works for seL4test and not for camkes-arm-vm is potentially because the Elfloader doesn't get started in monitor mode and is unable to start the kernel in Hyp mode. Here is some output from our build logs: U-Boot 2019.04-00592-g6c5f8dd540 (Apr 24 2019 - 12:42:58 +1000) TEGRA124 Model: NVIDIA Jetson TK1 Board: NVIDIA Jetson TK1 DRAM: 2 GiB MMC: sdhci@700b0400: 1, sdhci@700b0600: 0 Loading Environment from MMC... OK In: serial Out: serial Err: serial Net: No ethernet found. Hit any key to stop autoboot: 2 0 Tegra124 (Jetson TK1) # pci enum && setenv autoload no && dhcp && tftpboot 0x81000000 jetson1/sel4-image pci enum && setenv autoload no && dhcp && tftpboot 0x81000000 jetson1/sel4-image Warning: eth_rtl8169 using MAC address from ROM BOOTP broadcast 1 BOOTP broadcast 2 BOOTP broadcast 3 DHCP client bound to address 10.13.1.186 (1262 ms) Using eth_rtl8169 device TFTP from server 10.13.0.28; our IP address is 10.13.1.186 Filename 'jetson1/sel4-image'. Load address: 0x81000000 Loading: *################################################################# ################################################################# ################################################################# ################################################################# ################################################################# ################################################################# ################################################################# ################################################################# ################################################################# ################################################################# ################################################################# ################################################################# ################################################################# ################################################################# ################################################################# ################################################################# ################################################################# ################################################################# ################################################################# ################################################################# ################################################################# ################################################## 4 MiB/s done Bytes transferred = 20762864 (13cd0f0 hex) Tegra124 (Jetson TK1) # bootelf 0x81000000 || bootefi 0x81000000 bootelf 0x81000000 || bootefi 0x81000000 ## No elf image at address 0x81000000 78MMC: no card present Scanning disk sdhci@700b0400.blk... Disk sdhci@700b0400.blk not ready Scanning disk sdhci@700b0600.blk... Found 3 disks WARNING: booting without device tree ## Starting EFI application at 81000000 ... ELF loader: monitor mode init done Copy monitor mode vector from fb6785bc to a7f00000 size 50 Number of IRQs: 192 Load seL4 in nonsecure HYP mode 600001da ELF-loader started on CPU: ARM Ltd. Cortex-A15 r3p3 paddr=[fb671000..fca3efff] No DTB passed in from boot loader. Looking for DTB in CPIO archive...found at fb7423a4. Loaded DTB from fb7423a4. paddr=[80059000..80069fff] ELF-loading image 'kernel' paddr=[80000000..80058fff] vaddr=[e0000000..e0058fff] virt_entry=e0000000 ELF-loading image 'capdl-loader' paddr=[8006a000..81442fff] vaddr=[10000..13e8fff] virt_entry=17b1c relocating from fb671000-fca3f000 to dec30000-dfffe000... size=0x13ce000 (padded size = 0x13d0000) ELF loader relocated, continuing boot... Enabling hypervisor MMU and paging Jumping to kernel-image entry point... Bootstrapping kernel Total 28 IOASID set up Region [c to 28) for SMMU caps Booting all finished, dropped to user space
Thanks Kent for looking at this. I downloaded the image you sent and it gets stuck after I issue the bootefi command. I let it sit for a few minutes and it never goes anywhere (console output is pasted below) You mentioned that "A reason why ELF works for seL4test and not for camkes-arm-vm is potentially because the Elfloader doesn't get started in monitor mode and is unable to start the kernel in Hyp mode. " What I don't understand about that is that I have a couple of images that I built back in August 2018 that still work just fine on this device when I use bootelf to boot them. seL4 starts up and drops to the VM and I can log into the VM. U-Boot SPL 2020.01-00442-gc00bd81ae0 (Jan 10 2020 - 11:10:18 -0500) Trying to boot from RAM U-Boot 2020.01-00442-gc00bd81ae0 (Jan 10 2020 - 11:10:18 -0500) SoC: tegra124 Reset cause: POR Model: NVIDIA Jetson TK1 Board: NVIDIA Jetson TK1 DRAM: 2 GiB MMC: sdhci@700b0400: 1, sdhci@700b0600: 0 Loading Environment from MMC... OK In: serial Out: serial Err: serial Net: No ethernet found. Hit any key to stop autoboot: 0 Tegra124 (Jetson TK1) # fatload mmc 1 ${loadaddr} capdl-loader-image-arm-tk1 20762864 bytes read in 974 ms (20.3 MiB/s) Tegra124 (Jetson TK1) # bootefi ${loadaddr} On Tue, Jan 28, 2020 at 8:05 PM Mcleod, Kent (Data61, Kensington NSW) < Kent.Mcleod@data61.csiro.au> wrote:
Hi Mike,
You mentioned that "A reason why ELF works for seL4test and not for camkes-arm-vm is potentially because the Elfloader doesn't get started in monitor mode and is unable to start the kernel in Hyp mode. " What I don't understand about that is that I have a couple of images that I built back in August 2018 that still work just fine on this device when I use bootelf to boot them. seL4 starts up and drops to the VM and I can log into the VM. Ok.
Tegra124 (Jetson TK1) # fatload mmc 1 ${loadaddr} capdl-loader-image-arm-tk1 What is loadaddr set to? Does it fail with loadaddr set to 0x81000000?
Which Jetson-tk1 board are you using?
Hi Kent, Thanks for the suggestions. My loadaddr was set to 0x81000000. I tried using 0x84000000 and it booted just fine in secure mode. This is on the Nvidia TK1. If I build with -DTk1Insecure=TRUE, I do have some issues. I get: Undelivered IRQ: 85 Loading Linux: 'linux' dtb: 'linux-dtb' install_linux_devices@main.c:613 module name: map_frame_hack install_linux_devices@main.c:613 module name: plat 32 bit ARM insts not decoded -------- Pagefault from [Linux]: write fault @ PC: 0x80226ab0 IPA: 0x40000000, FSR: 0x2000046 Context: r0: 0xfe400000 r1: 0xc0020420 r2: 0x120 r3: 0xf10e01d3 r4: 0xe3006000 r5: 0xe3476000 r6: 0xe5966804 r7: 0xe2066cff r8: 0xe1a06426 r9: 0xc0993e70 r10: 0x80000200 r11: 0x80008000 r12: 0xe3560020 pc: 0xc0226ab0 r14: 0x82050000 sp: 0x82000000 cpsr: 0x200000d3 m-------- Assertion failed: rt >= 0 (/host/latest-camkes-arm-vm/projects/seL4_projects_libs/libsel4vm/src/arch/arm/fault.c: fault_get_width: 623) On Mon, Feb 3, 2020 at 1:27 AM Mcleod, Kent (Data61, Kensington NSW) < Kent.Mcleod@data61.csiro.au> wrote:
Hi Mike, I tried the insecure setting and saw the exact same issue as you. The issue is that the insecure setting would disable access to a region of RAM at 0x40000000 that should still be accessible. Re-enabling this device is a 1 line change which I'll put up a PR for soon (see the patch below). Once I've made this change the insecure version boots to login. project projects/vm/ diff --git a/components/VM/src/modules/plat/tk1/init.c b/components/VM/src/modules/plat/tk1/init.c index dce7809..4bae973 100644 --- a/components/VM/src/modules/plat/tk1/init.c +++ b/components/VM/src/modules/plat/tk1/init.c @@ -38,7 +38,6 @@ static const struct device *linux_pt_devices[] = { static const struct device *linux_ram_devices[] = { #ifndef CONFIG_TK1_INSECURE &dev_rtc_kbc_pmc, - &dev_data_memory, &dev_exception_vectors, &dev_system_registers, &dev_ictlr, @@ -46,6 +45,7 @@ static const struct device *linux_ram_devices[] = { &dev_fuse, &dev_gpios, #endif /* CONFIG_TK1_INSECURE */ + &dev_data_memory, }; extern reboot_hooks_list_t reboot_hooks_list;
participants (4)
-
Chris Guikema -
Mcleod, Kent (Data61, Kensington NSW) -
Mike Clark -
Shen, Yanyan (Data61, Kensington NSW)