Answering my own question here--maybe this will help someone else trying to get this kind of setup running.
The problem is that there are additional libraries required. The CMakeLists.txt for the hello project looks like this:
cmake_minimum_required(VERSION 3.7.2)
project(hello C) # create a new C project called 'Hello'
add_executable(hello src/main.c) # add files to our project. Paths are relative to this file.
target_link_libraries(hello sel4platsupport sel4muslcsys ) # we need to link against the standard C lib for printf
# Set this image as the rootserver DeclareRootserver(hello)
----------------------------------
This works for the debug build. For the release build there is a dependency on some debugging for early print functions. I think if those are removed it might work ok.