This could be due to an IOMMU configuration issue. Have you tried turning on the CONFIG_SMMU_INTERRUPT_ENABLE as well as CONFIG_PRINTING options in the seL4 kernel configuration? This will print out IOMMU faults that indicate the IOMMU isn't configured correctly.
Kent.