Hello,
I have a seL4 project that creates a number of processes with multiple threads. An application works fine. Every process periodically calculates the checksum of the executable segment and the system (Monitor process) keeps track that these values should be the same over the time. Some time when I change (add) some code in amu process I have an error that Monitor process prints
CHECKSUM ERROR -------- Someone modified executable memory process=Startup 0x400000 8868997 last->10daf436f961b809 this->95ebd3119aee7736!!!!
It always complains about the root (Startup) task. Checksum value changes every time the Startup process calculates it.
The code that calculate executable segment are:
typedef struct { unsigned char e_ident[EI_NIDENT]; /* Magic number and other info */ Elf64_Half e_type; /* Object file type */ Elf64_Half e_machine; /* Architecture */ Elf64_Word e_version; /* Object file version */ Elf64_Addr e_entry; /* Entry point virtual address */ Elf64_Off e_phoff; /* Program header table file offset */ Elf64_Off e_shoff; /* Section header table file offset */ Elf64_Word e_flags; /* Processor-specific flags */ Elf64_Half e_ehsize; /* ELF header size in bytes */ Elf64_Half e_phentsize; /* Program header table entry size */ Elf64_Half e_phnum; /* Program header table entry count */ Elf64_Half e_shentsize; /* Section header table entry size */ Elf64_Half e_shnum; /* Section header table entry count */ Elf64_Half e_shstrndx; /* Section header string table index */ } Elf64_Ehdr;
extern char __executable_start[];
// function returns executable segment start address and size int initStartupElfSegsParams(Elf64_Ehdr *pELFHeader, uintptr_t *startAddr, uintptr_t *size) { seL4_Word phoff = pELFHeader->e_phoff; Elf_Phdr *currentPH = (Elf_Phdr *)((void *)pELFHeader + phoff); for (int i=0; i<pELFHeader->e_phnum; i++) { ZF_LOGD("ELF: p_type=%x p_flags=%x p_vaddr=%p p_memsz=%lu", currentPH[i].p_type, currentPH[i].p_flags, (void *)currentPH[i].p_vaddr, currentPH[i].p_memsz); if ( ((currentPH[i].p_type & 1) == 1) && ((currentPH[i].p_flags & 1) == 1)) // p_flags & 1 == 1 - executable { *startAddr = (uintptr_t)currentPH[i].p_vaddr; *size = (uintptr_t)currentPH[i].p_memsz / sizeof(uint64_t); return 0; } }
return -1; };
initStartupElfSegsParams((Elf64_Ehdr *)__executable_start, &proc_validate_params[STARTUP_INDEX].p_addr, &proc_validate_params[STARTUP_INDEX].p_size);
Any help will be appreciated. Thanks
Hello Leonid,
On 2024-02-06 19:48, Leonid Meyerovich wrote:
Some time when I change (add) some code in amu process I have an error that Monitor process prints
CHECKSUM ERROR -------- Someone modified executable memory process=Startup 0x400000 8868997 last->10daf436f961b809 this->95ebd3119aee7736!!!!
It always complains about the root (Startup) task. Checksum value changes every time the Startup process calculates it.
Sounds like it may depend on whether the file size is odd or even. Are you accidentally checksumming past the end of the section?
The code that calculate executable segment are:
You can check if you're checking the right segment by comparing what your code finds with `readelf -l` output on your binary (not on the final image, but on the root task's binary).
Greetings,
Indan
Thank you Indan,
Using readelf helps, but I think there is some difference between running readelf on the file and parsing elf header in the memory (because my root process is already in the memory) As you can see below size of executable segment is different: readelf =*3923cb0, *mycode=*3923EB0* Also Entry point is different
Second segment starting address and size are the same (?) I use __executable_start to parse ELF header in the memory
Thanks, Leonid
readelf -l Startup
Elf file type is EXEC (Executable file) Entry point 0x40ef88 There are 4 program headers, starting at offset 64
Program Headers: Type Offset VirtAddr PhysAddr FileSiz MemSiz Flags Align LOAD 0x0000000000000000 0x0000000000400000 0x0000000000400000 0x0000000003923cb0 *0x0000000003923cb0 *RWE 0x1000 LOAD 0x0000000000000000 0x0000000003d30000 0x0000000003d30000 0x0000000000000000 0x0000000000aca428 RW 0x1000 TLS 0x00000000000cb490 0x00000000004cb490 0x00000000004cb490 0x0000000000000000 0x000000000000000c R 0x8 GNU_STACK 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 RW 0x10
Section to Segment mapping: Segment Sections... 00 .init .text .fini .rodata .eh_frame .init_array .fini_array .got .got.plt .data ._archive_cpio __vsyscall _ps_irqchips 01 .bss 02 .tbss 03 ------------------------------------ my code ---------------------------- ELF Header: Type: 2 Machine: 183 Version: 1 Entry point address: 0x40f218 Start of program headers: 64 bytes into file Number of program headers: 4 Size of program header: 56 bytes Segment 0: Type: 1 Flags: 7 Virtual address: 0x400000 Physical address: 0x400000 Size in file: 59915952 bytes (*3923EB0*) Size in memory: 59915952 bytes Alignment: 4096 Segment 1: Type: 1 Flags: 6 Virtual address: 0x3d30000 Physical address: 0x3d30000 Size in file: 0 bytes Size in memory: 11314216 bytes ( *aca428*) Alignment: 4096 Segment 2: Type: 7 Flags: 4 Virtual address: 0x4cb990 Physical address: 0x4cb990 Size in file: 0 bytes Size in memory: 12 bytes Alignment: 8 Segment 3: Type: 1685382481 Flags: 6 Virtual address: 0x0 Physical address: 0x0 Size in file: 0 bytes Size in memory: 0 bytes Alignment: 16
On Wed, Feb 7, 2024 at 9:24 AM Indan Zupancic indan@nul.nu wrote:
Hello Leonid,
On 2024-02-06 19:48, Leonid Meyerovich wrote:
Some time when I change (add) some code in amu process I have an error that Monitor process prints
CHECKSUM ERROR -------- Someone modified executable memory process=Startup 0x400000 8868997 last->10daf436f961b809 this->95ebd3119aee7736!!!!
It always complains about the root (Startup) task. Checksum value changes every time the Startup process calculates it.
Sounds like it may depend on whether the file size is odd or even. Are you accidentally checksumming past the end of the section?
The code that calculate executable segment are:
You can check if you're checking the right segment by comparing what your code finds with `readelf -l` output on your binary (not on the final image, but on the root task's binary).
Greetings,
Indan
Hello Leonid,
On 2024-02-07 18:15, Leonid Meyerovich wrote:
Using readelf helps, but I think there is some difference between running readelf on the file and parsing elf header in the memory (because my root process is already in the memory) As you can see below size of executable segment is different: readelf =3923cb0, mycode=3923EB0 Also Entry point is different
This is unexpected, are you sure you are comparing the same binary?
Second segment starting address and size are the same (?) I use __executable_start to parse ELF header in the memory
I don't think you can count on the ELF header being there though.
readelf -l Startup
Elf file type is EXEC (Executable file) Entry point 0x40ef88 There are 4 program headers, starting at offset 64
Program Headers: Type Offset VirtAddr PhysAddr FileSiz MemSiz Flags Align LOAD 0x0000000000000000 0x0000000000400000 0x0000000000400000 0x0000000003923cb0 0x0000000003923cb0 RWE 0x1000 LOAD 0x0000000000000000 0x0000000003d30000 0x0000000003d30000 0x0000000000000000 0x0000000000aca428 RW 0x1000 TLS 0x00000000000cb490 0x00000000004cb490 0x00000000004cb490 0x0000000000000000 0x000000000000000c R 0x8 GNU_STACK 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 RW 0x10
Section to Segment mapping: Segment Sections... 00 .init .text .fini .rodata .eh_frame .init_array .fini_array .got .got.plt .data ._archive_cpio __vsyscall _ps_irqchips
Anyway, writeable sections are put into segment 0 too, like .data, so you can't just checksum the whole segment.
If you want to do these kind of things it's better to do with your own linker script, then you have full control and can also let the linker define section start and end symbols so you don't need to parse ELF headers.
That said, the seL4 kernel has limitations, which is the reason everything is put in the same segment for the startup task.
In your case just checksum between __executable_start and __etext, which marks the end of the executable section.
(If you worry that your program is being modified, you can just load your own program with proper memory protection where you do everything and do as little as possible in the startup task.)
Greetings,
Indan
Thanks Indan,
Yes I see all my other tasks have different flags for segment 0 - RE, Startup flag is RWI Is there any document that describes this limitation?
Thanks, Leonid
readelf -l Logger
Elf file type is EXEC (Executable file) Entry point 0x40018c There are 5 program headers, starting at offset 64
Program Headers: Type Offset VirtAddr PhysAddr FileSiz MemSiz Flags Align LOAD 0x0000000000000000 0x0000000000400000 0x0000000000400000 0x00000000000a5e94 0x00000000000a5e94 * R E* 0x1000 LOAD 0x00000000000a5f90 0x00000000004a6f90 0x00000000004a6f90 0x00000000000013e8 0x0000000000219310 RW 0x1000 TLS 0x00000000000a5f90 0x00000000004a6f90 0x00000000004a6f90 0x0000000000000000 0x000000000000000c R 0x8 GNU_STACK 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 RW 0x10 GNU_RELRO 0x00000000000a5f90 0x00000000004a6f90 0x00000000004a6f90 0x0000000000000070 0x0000000000000070 R 0x1
Section to Segment mapping: Segment Sections... 00 .init .text .fini .rodata .eh_frame 01 .init_array .fini_array .got .got.plt .data __vsyscall _ps_irqchips .bss 02 .tbss 03 04 .init_array .fini_array .got .got.plt
On Wed, Feb 7, 2024 at 2:57 PM Indan Zupancic indan@nul.nu wrote:
Hello Leonid,
On 2024-02-07 18:15, Leonid Meyerovich wrote:
Using readelf helps, but I think there is some difference between running readelf on the file and parsing elf header in the memory (because my root process is already in the memory) As you can see below size of executable segment is different: readelf =3923cb0, mycode=3923EB0 Also Entry point is different
This is unexpected, are you sure you are comparing the same binary?
Second segment starting address and size are the same (?) I use __executable_start to parse ELF header in the memory
I don't think you can count on the ELF header being there though.
readelf -l Startup
Elf file type is EXEC (Executable file) Entry point 0x40ef88 There are 4 program headers, starting at offset 64
Program Headers: Type Offset VirtAddr PhysAddr FileSiz MemSiz Flags Align LOAD 0x0000000000000000 0x0000000000400000 0x0000000000400000 0x0000000003923cb0 0x0000000003923cb0 RWE 0x1000 LOAD 0x0000000000000000 0x0000000003d30000 0x0000000003d30000 0x0000000000000000 0x0000000000aca428 RW 0x1000 TLS 0x00000000000cb490 0x00000000004cb490 0x00000000004cb490 0x0000000000000000 0x000000000000000c R 0x8 GNU_STACK 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 RW 0x10
Section to Segment mapping: Segment Sections... 00 .init .text .fini .rodata .eh_frame .init_array .fini_array .got .got.plt .data ._archive_cpio __vsyscall _ps_irqchips
Anyway, writeable sections are put into segment 0 too, like .data, so you can't just checksum the whole segment.
If you want to do these kind of things it's better to do with your own linker script, then you have full control and can also let the linker define section start and end symbols so you don't need to parse ELF headers.
That said, the seL4 kernel has limitations, which is the reason everything is put in the same segment for the startup task.
In your case just checksum between __executable_start and __etext, which marks the end of the executable section.
(If you worry that your program is being modified, you can just load your own program with proper memory protection where you do everything and do as little as possible in the startup task.)
Greetings,
Indan
Hello Leonid,
On 2024-02-07 20:19, Leonid Meyerovich wrote:
Yes I see all my other tasks have different flags for segment 0 - RE, Startup flag is RWI Is there any document that describes this limitation?
I don't think the ABI between the kernel and Elfloader/root task is documented anywhere. Only bootinfo is documented, but that's only one part of the boot ABI. That said, because it's all part of the same project, I don't think this is fixed or guaranteed to stay the same, so users shouldn't assume more than what is provided by bootinfo. Anyway, e.g. for aarch64 it's somewhat described here:
https://github.com/seL4/seL4/blob/master/src/arch/arm/64/head.S#L127
Greetings,
Indan
Thank you Indan.
On Thu, Feb 8, 2024 at 7:47 AM Indan Zupancic indan@nul.nu wrote:
Hello Leonid,
On 2024-02-07 20:19, Leonid Meyerovich wrote:
Yes I see all my other tasks have different flags for segment 0 - RE, Startup flag is RWI Is there any document that describes this limitation?
I don't think the ABI between the kernel and Elfloader/root task is documented anywhere. Only bootinfo is documented, but that's only one part of the boot ABI. That said, because it's all part of the same project, I don't think this is fixed or guaranteed to stay the same, so users shouldn't assume more than what is provided by bootinfo. Anyway, e.g. for aarch64 it's somewhat described here:
https://github.com/seL4/seL4/blob/master/src/arch/arm/64/head.S#L127
Greetings,
Indan
-
Indan Zupancic -
Leonid Meyerovich