Camkes SerialServer and util_libs/libplatsupport with ODROID-C4
paddr=0xff803000 dev->size=0x1000 uart_init@serial.c:56 chardev_map mapped page_vaddr 0
Hi everyone
Summary - trying to get util_libs/libplatsupport going for a HardKernel
ODROID-C4, using the camkes serialserver_polling app - but not having any
luck with mapping the UART. I'd welcome any explanations of what you think
is happening here. The background and my progress follows.
I thought I'd dive into the seL4 world and do some experimenting. I
thought a good place to start would be the UNSW AOS project (17 years after
I first did it) - but with no more ODROID-C2, got a hold of an ODROID-C4
and figured that would be a good project in itself (and well timed with a
Sydney lockdown).
So - good news: The peripheral devices we need are all very similar to a
C2, so by getting a new DTS file and updating a few addresses it starts off
fairly happily. sel4-test builds and runs (All is well...); so does the
camkes adder app; and with some mods to provide enough slots (maybe because
it has 4GB rather than 2GB) I have got the AOS milestone 0 stub to run
(including the uart initialisation). (More work to actually finish m0 and
keep going, but a job for later)
But then: I moved onto the util_libs / libplatsupport project to try and
port over as much of the device settings as possible; and using the camkes
app serialserver_polling, under the hood using SerialServer, I can't get
the uart to initialiise (or more strictly can't get the uart address to
map). chardev_map / ps_io_map fails, returning a vaddr of 0.
I've added a bunch of debug prints to the process; and for comparison also
run it on QEMU emulating a Sabre board.
On odroidc4 hardware:
## Starting application at 0x01080000 ...
ELF-loader started on CPU: ARM Ltd. Cortex-A55 r1p0
paddr=[95e000..d5710f]
No DTB passed in from boot loader.
Looking for DTB in CPIO archive...found at a86de8.
Loaded DTB from a86de8.
paddr=[23f000..24afff]
ELF-loading image 'kernel' to 0
paddr=[0..23efff]
vaddr=[ffffff8000000000..ffffff800023efff]
virt_entry=ffffff8000000000
ELF-loading image 'capdl-loader' to 24b000
paddr=[24b000..563fff]
vaddr=[400000..718fff]
virt_entry=408e48
Enabling MMU and paging
Jumping to kernel-image entry point...
Bootstrapping kernel
Booting all finished, dropped to user space
pre_init@serial.c:501 About to init port, io_ops=0x487cf0
ps_cdev_init@chardev.c:46 About to init_fn 3
ps_cdev_init@chardev.c:47 dev_defn[3].paddr: 0xff803000, size 1000
uart_init@serial.c:51 about to chardev_map: dev=0x473948, defn->id=3,
defn->padd
r=0xff803000
chardev_map@chardev.h:26 ops->io_mapper=0x487be0 &(ops->io_mapper)=0x442728
dev-
plat_pre_init@plat.c:64 Failed to initialise character device
plat_pre_init@plat.c:65 PS_SERIAL_DEFAULT: 3
On Sabre QEMU:
./simulate: qemu-system-arm -machine sabrelite -nographic -serial null
-serial mon:stdio -m size=1024M -kernel images/capdl-loader-image-
arm-imx6
ELF-loader started on CPU: ARM Ltd. Cortex-A9 r0p0
paddr=[106fc000..10aa98e3]
No DTB passed in from boot loader.
Looking for DTB in CPIO archive...found at 107fe33c.
Loaded DTB from 107fe33c.
paddr=[1003d000..10047fff]
ELF-loading image 'kernel' to 10000000
paddr=[10000000..1003cfff]
vaddr=[e0000000..e003cfff]
virt_entry=e0000000
ELF-loading image 'capdl-loader' to 10048000
paddr=[10048000..10300fff]
vaddr=[10000..2c8fff]
virt_entry=187f8
Enabling MMU and paging
Jumping to kernel-image entry point...
Bootstrapping kernel
Warning: Could not infer GIC interrupt target ID, assuming 0.
Booting all finished, dropped to user space
pre_init@serial.c:501 About to init port, io_ops=0x69cf0
ps_cdev_init@chardev.c:45 About to init_fn 1
ps_cdev_init@chardev.c:46 dev_defn[1].paddr: 0x21e8000, size 1000
uart_init@serial.c:224 about to chardev_map: dev=0x7f8c8, defn->id=1,
defn->paddr=0x21e8000
chardev_map@chardev.h:26 ops->io_mapper=0x69cf0 &(ops->io_mapper)=0x1f328
dev->paddr=0x1f654 dev->size=0x4ebc4
uart_init@serial.c:226 chardev_map mapped vaddr 0xbb000
pollcli: Hello Serial Server!
pollcli: abc
This has me confused - it's good it works, but I can't follow in the flow
how the paddr gets changed from the value supplied to chardev_map (ie
0x21e80000 becomes 0x1f654), yet in that case it works. Or is it a
complication of using QEMU?
A similar kind of test with AOS on odroidc4 - particularly the success
of "*uart_init@uart.c:35
Mapped uart paddr:ff803000 size:1000 vaddr:b0000000*"
## Starting application at 0x01080000 ...
ELF-loader started on CPU: ARM Ltd. Cortex-A55 r1p0
paddr=[88a000..c2b10f]
No DTB passed in from boot loader.
Looking for DTB in CPIO archive...found at 9f3068.
Loaded DTB from 9f3068.
paddr=[247000..252fff]
ELF-loading image 'kernel' to 0
paddr=[0..246fff]
vaddr=[ffffff8000000000..ffffff8000246fff]
virt_entry=ffffff8000000000
ELF-loading image 'sos' to 253000
paddr=[253000..48cfff]
vaddr=[400000..639fff]
virt_entry=4172e0
Enabling MMU and paging
Jumping to kernel-image entry point...
Bootstrapping kernel
Booting all finished, dropped to user space
SOS Starting...
<
On Wed, Jun 30, 2021 at 8:12 PM Robert Reid
Hi everyone
Summary - trying to get util_libs/libplatsupport going for a HardKernel ODROID-C4, using the camkes serialserver_polling app - but not having any luck with mapping the UART. I'd welcome any explanations of what you think is happening here. The background and my progress follows.
I thought I'd dive into the seL4 world and do some experimenting. I thought a good place to start would be the UNSW AOS project (17 years after I first did it) - but with no more ODROID-C2, got a hold of an ODROID-C4 and figured that would be a good project in itself (and well timed with a Sydney lockdown).
So - good news: The peripheral devices we need are all very similar to a C2, so by getting a new DTS file and updating a few addresses it starts off fairly happily. sel4-test builds and runs (All is well...); so does the camkes adder app; and with some mods to provide enough slots (maybe because it has 4GB rather than 2GB) I have got the AOS milestone 0 stub to run (including the uart initialisation). (More work to actually finish m0 and keep going, but a job for later)
But then: I moved onto the util_libs / libplatsupport project to try and port over as much of the device settings as possible; and using the camkes app serialserver_polling, under the hood using SerialServer, I can't get the uart to initialiise (or more strictly can't get the uart address to map). chardev_map / ps_io_map fails, returning a vaddr of 0.
I've added a bunch of debug prints to the process; and for comparison also run it on QEMU emulating a Sabre board.
On odroidc4 hardware: ## Starting application at 0x01080000 ...
ELF-loader started on CPU: ARM Ltd. Cortex-A55 r1p0 paddr=[95e000..d5710f] No DTB passed in from boot loader. Looking for DTB in CPIO archive...found at a86de8. Loaded DTB from a86de8. paddr=[23f000..24afff] ELF-loading image 'kernel' to 0 paddr=[0..23efff] vaddr=[ffffff8000000000..ffffff800023efff] virt_entry=ffffff8000000000 ELF-loading image 'capdl-loader' to 24b000 paddr=[24b000..563fff] vaddr=[400000..718fff] virt_entry=408e48 Enabling MMU and paging Jumping to kernel-image entry point...
paddr=0xff803000 dev->size=0x1000 uart_init@serial.c:56 chardev_map mapped page_vaddr 0
Bootstrapping kernel Booting all finished, dropped to user space pre_init@serial.c:501 About to init port, io_ops=0x487cf0 ps_cdev_init@chardev.c:46 About to init_fn 3 ps_cdev_init@chardev.c:47 dev_defn[3].paddr: 0xff803000, size 1000 uart_init@serial.c:51 about to chardev_map: dev=0x473948, defn->id=3, defn->padd r=0xff803000 chardev_map@chardev.h:26 ops->io_mapper=0x487be0 &(ops->io_mapper)=0x442728 dev- plat_pre_init@plat.c:64 Failed to initialise character device plat_pre_init@plat.c:65 PS_SERIAL_DEFAULT: 3
It could be that the capability to the mmio frame isn't being given to the serialserver camkes component. The serialserver component's CAmkES definition picks the serial device referred to by the stdout-path property of the chosen node in the platform device tree. In the build directory there will be a generated file "serialserver_polling.cdl". This file contains all of the capabilities that will be created during system initialization. If you search this file for the physical address of the MMIO you are trying to have mapped you can check whether it is being given to the SerialServer component. If it has been declared correctly in this generated file, then the chardev_init in libplatsupport should be able to correctly request and map it in. libplatsupport currently uses a platform specific header file to pick which serial device it initializes by default, so if the definition of PS_SERIAL_DEFAULT doesn't refer to the same device as the stdout-path property in the device tree the driver could be trying to initialize a different serial device than the one camkes allocated resources for.
On Sabre QEMU: ./simulate: qemu-system-arm -machine sabrelite -nographic -serial null -serial mon:stdio -m size=1024M -kernel images/capdl-loader-image- arm-imx6 ELF-loader started on CPU: ARM Ltd. Cortex-A9 r0p0 paddr=[106fc000..10aa98e3] No DTB passed in from boot loader. Looking for DTB in CPIO archive...found at 107fe33c. Loaded DTB from 107fe33c. paddr=[1003d000..10047fff] ELF-loading image 'kernel' to 10000000 paddr=[10000000..1003cfff] vaddr=[e0000000..e003cfff] virt_entry=e0000000 ELF-loading image 'capdl-loader' to 10048000 paddr=[10048000..10300fff] vaddr=[10000..2c8fff] virt_entry=187f8 Enabling MMU and paging Jumping to kernel-image entry point...
Bootstrapping kernel Warning: Could not infer GIC interrupt target ID, assuming 0. Booting all finished, dropped to user space pre_init@serial.c:501 About to init port, io_ops=0x69cf0 ps_cdev_init@chardev.c:45 About to init_fn 1 ps_cdev_init@chardev.c:46 dev_defn[1].paddr: 0x21e8000, size 1000 uart_init@serial.c:224 about to chardev_map: dev=0x7f8c8, defn->id=1, defn->paddr=0x21e8000 chardev_map@chardev.h:26 ops->io_mapper=0x69cf0 &(ops->io_mapper)=0x1f328 dev->paddr=0x1f654 dev->size=0x4ebc4 uart_init@serial.c:226 chardev_map mapped vaddr 0xbb000 pollcli: Hello Serial Server! pollcli: abc
This has me confused - it's good it works, but I can't follow in the flow how the paddr gets changed from the value supplied to chardev_map (ie 0x21e80000 becomes 0x1f654), yet in that case it works. Or is it a complication of using QEMU?
A similar kind of test with AOS on odroidc4 - particularly the success of "*uart_init@uart.c:35 Mapped uart paddr:ff803000 size:1000 vaddr:b0000000*" ## Starting application at 0x01080000 ...
ELF-loader started on CPU: ARM Ltd. Cortex-A55 r1p0 paddr=[88a000..c2b10f] No DTB passed in from boot loader. Looking for DTB in CPIO archive...found at 9f3068. Loaded DTB from 9f3068. paddr=[247000..252fff] ELF-loading image 'kernel' to 0 paddr=[0..246fff] vaddr=[ffffff8000000000..ffffff8000246fff] virt_entry=ffffff8000000000 ELF-loading image 'sos' to 253000 paddr=[253000..48cfff] vaddr=[400000..639fff] virt_entry=4172e0 Enabling MMU and paging Jumping to kernel-image entry point...
Bootstrapping kernel Booting all finished, dropped to user space
SOS Starting... <
> < > < > < > < > < > < > uart_init@uart.c:35 Mapped uart paddr:ff803000 size:1000 vaddr:b0000000 SOS Started! test_cspace@tests.c:59 Test cspace run_tests@tests.c:174 Root CSpace test passed! test_cspace@tests.c:59 Test cspace run_tests@tests.c:182 Single level cspace test passed! test_cspace@tests.c:59 Test cspace cspace_alloc_slot@cspace.c:335 Cspace is full! run_tests@tests.c:195 Double level cspace test passed! run_tests@tests.c:199 DMA test passed! run_tests@tests.c:203 Frame table test passed! Network init dwmac.b0002000 Waiting for PHY auto negotiation to complete........ done Speed: 1000, full duplex Protocol ethernet registered (layer: 2). Protocol ipv4 registered (layer: 3). Protocol ipv6 registered (layer: 3). Protocol icmp4 registered (layer: 4). Protocol icmp6 registered (layer: 4). Protocol igmp registered (layer: 4). Protocol udp registered (layer: 4). Protocol tcp registered (layer: 4). Assigned ipv4 0.0.0.0 to device sos picotcp DHCP client: update link Assigned ipv4 192.168.200.125 to device sos picotcp DHCP client: renewal time (T1) 43200 DHCP client: rebinding time (T2) 75600 DHCP client: lease time 86400 DHCP client: netmask 255.255.255.0 DHCP client: gateway 192.168.200.4 Timer init Start first process Starting ttytest at 0x4001c8
SOS entering syscall loop aos: Error attempting syscall 29 task: Hello world, I'm tty_test! handle_syscall@main.c:128 Unknown syscall 1
<
> In this case it has worked, sos_map_device works as expected (although the spurious interrupts are another task to figure out). But trying to understand what's really going on underneath ps_io_map and ultimately why libplatsupport won't work for me has got me confused. Any ideas would be appreciated.
Regards Rob _______________________________________________ Devel mailing list -- devel@sel4.systems To unsubscribe send an email to devel-leave@sel4.systems
On Thu, 1 Jul 2021 at 17:16, Kent Mcleod
It could be that the capability to the mmio frame isn't being given to the serialserver camkes component. The serialserver component's CAmkES definition picks the serial device referred to by the stdout-path property of the chosen node in the platform device tree. In the build directory there will be a generated file "serialserver_polling.cdl". This file contains all of the capabilities that will be created during system initialization. If you search this file for the physical address of the MMIO you are trying to have mapped you can check whether it is being given to the SerialServer component. If it has been declared correctly in this generated file, then the chardev_init in libplatsupport should be able to correctly request and map it in. libplatsupport currently uses a platform specific header file to pick which serial device it initializes by default, so if the definition of PS_SERIAL_DEFAULT doesn't refer to the same device as the stdout-path property in the device tree the driver could be trying to initialize a different serial device than the one camkes allocated resources for.
On the right track with the right capability not being provided, but still can't get it working from here. I am still using PS_SERIAL_DEFAULT to be the same as stdout, UART0_AO @ 0xff803000, so safe there. stdout-path was chosen via an alias: stdout-path = "serial0:115200n8"; serial0 = "/soc/bus@ff800000/serial@3000" I end up with the following in the build directory: *capdl_spec.c:* [0] = { #ifdef CONFIG_DEBUG_BUILD .name = "serial.serial_dev_0_data_0_obj", #endif .type = CDL_Frame, .size_bits = 12, .frame_extra = { .paddr = 0x3000,.fill = { } }, }, *serialserver_polling.cdl:* serial.serial_dev_0_data_0_obj = frame (4k, paddr: 0x3000, fill: []) I thought there might be some issue with the relative address in the dts; compared to sabre.dts where a full physical address is specified. However changing the dts to use a full physical address: defined the alias: serial0 = "/soc/bus@ff800000/serial@ff803000"; chosen the serial port as stdout-path: stdout-path = "serial0:115200n8"; and updated the actual device definition later in the DTS: serial@ff803000 { ... } still gives me paddr: 0x3000 in the cdl. Going further, I've removed the alias so it was directly defined; and eventually changed hard coding the global-components/components/SerialServer/include/plat/arm_common/plat/serial.h file: #define HARDWARE_SERIAL_CONFIG \ serial_dev.dtb = dtb({"path" : "/soc/bus@ff800000/serial@ff803000"}); \ serial_dev.generate_interrupts = 1; - but all for the same result, no good. I notice building for -DPLATFORM=odroidc2 results in similar (in its case paddr: 0x0000). I wonder whether there's a difference between aarch32 and aarch64 going on here. Time to go looking at what the dtb() tool is doing?
On Thu, Jul 1, 2021 at 8:28 PM Robert Reid
On Thu, 1 Jul 2021 at 17:16, Kent Mcleod
wrote: It could be that the capability to the mmio frame isn't being given to the serialserver camkes component. The serialserver component's CAmkES definition picks the serial device referred to by the stdout-path property of the chosen node in the platform device tree. In the build directory there will be a generated file "serialserver_polling.cdl". This file contains all of the capabilities that will be created during system initialization. If you search this file for the physical address of the MMIO you are trying to have mapped you can check whether it is being given to the SerialServer component. If it has been declared correctly in this generated file, then the chardev_init in libplatsupport should be able to correctly request and map it in. libplatsupport currently uses a platform specific header file to pick which serial device it initializes by default, so if the definition of PS_SERIAL_DEFAULT doesn't refer to the same device as the stdout-path property in the device tree the driver could be trying to initialize a different serial device than the one camkes allocated resources for.
On the right track with the right capability not being provided, but still can't get it working from here.
I am still using PS_SERIAL_DEFAULT to be the same as stdout, UART0_AO @ 0xff803000, so safe there. stdout-path was chosen via an alias: stdout-path = "serial0:115200n8"; serial0 = "/soc/bus@ff800000/serial@3000"
I end up with the following in the build directory: capdl_spec.c: [0] = { #ifdef CONFIG_DEBUG_BUILD .name = "serial.serial_dev_0_data_0_obj", #endif .type = CDL_Frame, .size_bits = 12, .frame_extra = { .paddr = 0x3000,.fill = { } }, },
serialserver_polling.cdl: serial.serial_dev_0_data_0_obj = frame (4k, paddr: 0x3000, fill: [])
I thought there might be some issue with the relative address in the dts; compared to sabre.dts where a full physical address is specified.
I think this would be the issue. Looking at the implementation of "dtb-query-common.template.c" that is imported by seL4DTBHardware-to.template.c (the instantiation of which causes the allocation of caps based on the original dtb() property) it assumes that the reg property contains an absolute address. It isn't handling the case where the reg value inherits the parent's reg property.
However changing the dts to use a full physical address: defined the alias: serial0 = "/soc/bus@ff800000/serial@ff803000"; chosen the serial port as stdout-path: stdout-path = "serial0:115200n8"; and updated the actual device definition later in the DTS: serial@ff803000 { ... }
still gives me paddr: 0x3000 in the cdl. Going further, I've removed the alias so it was directly defined; and eventually changed hard coding the global-components/components/SerialServer/include/plat/arm_common/plat/serial.h file: #define HARDWARE_SERIAL_CONFIG \ serial_dev.dtb = dtb({"path" : "/soc/bus@ff800000/serial@ff803000"}); \ serial_dev.generate_interrupts = 1;
- but all for the same result, no good.
I notice building for -DPLATFORM=odroidc2 results in similar (in its case paddr: 0x0000). I wonder whether there's a difference between aarch32 and aarch64 going on here.
Time to go looking at what the dtb() tool is doing?
On Thu, 1 Jul 2021 at 21:54, Kent Mcleod
On Thu, Jul 1, 2021 at 8:28 PM Robert Reid
wrote: I am still using PS_SERIAL_DEFAULT to be the same as stdout, UART0_AO @ 0xff803000, so safe there. stdout-path was chosen via an alias: stdout-path = "serial0:115200n8"; serial0 = "/soc/bus@ff800000/serial@3000"
I end up with the following in the build directory: capdl_spec.c: [0] = { #ifdef CONFIG_DEBUG_BUILD .name = "serial.serial_dev_0_data_0_obj", #endif .type = CDL_Frame, .size_bits = 12, .frame_extra = { .paddr = 0x3000,.fill = { } }, },
serialserver_polling.cdl: serial.serial_dev_0_data_0_obj = frame (4k, paddr: 0x3000, fill: [])
I thought there might be some issue with the relative address in the dts; compared to sabre.dts where a full physical address is specified.
I think this would be the issue. Looking at the implementation of "dtb-query-common.template.c" that is imported by seL4DTBHardware-to.template.c (the instantiation of which causes the allocation of caps based on the original dtb() property) it assumes that the reg property contains an absolute address. It isn't handling the case where the reg value inherits the parent's reg property.
Thanks Kent - your comment made me look harder at the DTS and found the reg = < ... > line for that UART, updating 0x3000 to 0xff803000 there as well and things are starting to look better: Booting all finished, dropped to user space pre_init@serial.c:501 About to init port, io_ops=0x487cf0 ps_cdev_init@chardev.c:46 About to init_fn 3 ps_cdev_init@chardev.c:47 dev_defn[3].paddr: 0xff803000, size 1000 uart_init@serial.c:51 about to chardev_map: dev=0x473948, defn->id=3, defn->paddr=0 xff803000 chardev_map@chardev.h:26 ops->io_mapper=0x487be0 &(ops->io_mapper)=0x442728 dev->pa ddr=0xff803000 dev->size=0x1000 uart_init@serial.c:56 chardev_map mapped page_vaddr 0x4b1000 pollcli: Hello Serial Server! It prints fine, but not reading / filling the buffer. So on to the next problem, but at least it's got this far. Thanks for the help, greatly appreciated! Regards Rob
participants (2)
-
Kent Mcleod
-
Robert Reid