I have a system configured to use dynamic heap through the vspace. I'm working with 5.2.0 and using pc99 simulator through qemu.

Everything is occurring in the root task, as I haven't even completed my bootstrapping code.

In this configuration, whenever my code calls malloc() and there is a low memory condition (where there is not enough space on the heap to fulfill my request), malloc() end up calling __expand_heap().

Here it first attempts to expand the heap through brk syscall, then if fails uses mmap() to attempt to get more pages from the virtual memory.

In both cases (brk and mmap) end up calling a function in the vspace that attempts to make available new pages.

Unfortunately the code in the libself4utils/src/vspace/vspace.c, end up calling malloc() for the sel4utils_res structure.

As you expect, we are already in a no-memory condition, so a subsequent call to malloc() might  end up calling __expand_heap() and so on...  (depending on the bytes  left on the heap, this nested malloc might succeed if the initial block was larger than the size of sel4utils_res structure).

So, everything loops until the whole system crash because of stack overflow.

Why vspace uses malloc() to dynamically allocate those structures, causing the whole system to fail?

To test, I have changed the implementation in vspace.c to use a static buffer pool, and it seems it solve my issue, but am I doing something wrong in the bootstrap code ?

Any help is appreciated,

Regards,

Fabrizio Bertocci
Real-Time Innovations, Inc.

Sunnyvale, CA