Hi Mike, In order to be acting as a hypervisor in the default configuration that you say booted fine seL4 will already be setup to run in nonsecure hypervisor mode. Given this can you elaborate on what 'the unsecure mode' is and how you are switching to it? Adrian On Sun 10-Jun-2018 10:08 PM, Mike Clark wrote:

I just checked out a fresh copy of https://github.com/ SEL4PROJ/camkes-arm-vm-manifest and built an image for the TK1 using the docker build environment. I copied the resulting image to an SD card and it boots fine.

If I switch to the unsecure mode, I get an error on boot.

SMMU Address translation error: ID: 98 address: 0xae5be000 type: 6 direction: 0x0 IOPT permission: read 0x0 write 0x0 nonsecure 0x0 [ 10.204777] mmc1: ADMA error

I have not made any changes, so my vm.smmu settings are all default.

Any thoughts or help?

Mike

_______________________________________________ Devel mailing list Devel@sel4.systems https://sel4.systems/lists/listinfo/devel

Anyone else running into this issue with the latest clone? I had this all working just fine previously (maybe 6 months ago), but something must have been updated that breaks this. Not sure why. Mike On Tue, Jun 12, 2018 at 8:45 AM, Mike Clark wrote:

Adrian,

I am doing `make menuconfig`, then going to "Applications" and checking the box that says "Insecure: Give all devices to VM component". That is the only change I do, then recompile, copy the new image to my SD card, and boot.

Mike

On Mon, Jun 11, 2018 at 9:20 PM, wrote:

...

Hi Mike,

In order to be acting as a hypervisor in the default configuration that you say booted fine seL4 will already be setup to run in nonsecure hypervisor mode. Given this can you elaborate on what 'the unsecure mode' is and how you are switching to it?

Adrian

...

I just checked out a fresh copy of https://github.com/ SEL4PROJ/camkes-arm-vm-manifest and built an image for the TK1 using

On Sun 10-Jun-2018 10:08 PM, Mike Clark wrote: the

...

docker build environment. I copied the resulting image to an SD card and it boots fine.

If I switch to the unsecure mode, I get an error on boot.

SMMU Address translation error: ID: 98 address: 0xae5be000 type: 6 direction: 0x0 IOPT permission: read 0x0 write 0x0 nonsecure 0x0 [ 10.204777] mmc1: ADMA error

I have not made any changes, so my vm.smmu settings are all default.

Any thoughts or help?

Mike

_______________________________________________ Devel mailing list Devel@sel4.systems https://sel4.systems/lists/listinfo/devel

Kent, Thanks. Glad you were able to fix that bug. I noticed that the build system changed yesterday and tried with it, but was running into some different issues. Looks like that is all fixed now today as I was able to boot just fine. One thing I noticed when I booted today is that I don't appear to have access to ethernet any more. `ip addr` use to give me an eth0 device when I was last using this. With this new image, I was expecting to see an `enp0s1` or something like that, but I don't get one. Has that device been removed from the insecure mode? One thing I was doing before that I'd like to get back to is booting off a separate partition on my SD card that has it's own linux image on it. The partition I'm using comes from an image that you had provided to me. Is there an easy way to get back to that booting configuration? Finally, just to let you know. Once I get this all back up and running I'll test the issue that I was having previously. But, just so you know, I had figured out a work around for that issue, so it didn't stop me from doing the testing I wanted to do. Mike On Thu, Jun 21, 2018 at 9:12 AM, wrote:

Hi Mike,

I'm not sure what changes in the last 6 months would have changed your build from a working to broken state. The below commits[1] are the only changes that I am aware would change the behavior of smmu permissions.

ID: 98​ corresponds to sdmmc3 (there is a table in the tk1 reference manual, around page 719 [2]), and this corresponds to a vm.smmu id of 22, which is absent from the TK1INSECURE configuration.

Adding that ID made my VM crash---I tracked the cause down to a iommu page table initialisation bug here: https://github.com/ SEL4PROJ/seL4_projects_libs/commit/396542fc8b7d0ab41bf92d8850b308 2da80c69e7

Fixing that allowed my vm to boot with the extra vm.smmu id of 22. (Note: I boot over tftpboot and not using an sdcard, so it also boots without id 22 for me, however the ID: 98 error you are getting very likely corresponds to id 22 being missing).

The latest checkout that I tested is this one:

https://github.com/SEL4PROJ/camkes-arm-vm-manifest/commit/ 8f83660cdd98c51bdaa388f9bb77ce2ea5116203 Also note that yesterday we merged some refactors that migrated the project to use a new CMake based build system.

running the following:

mkdir build && cd build && ../init-build.sh -DAARCH32=TRUE -DTk1Insecure=True -DCAMKES_VM_APP=tk1_vm && ninja

should build you an equivalent image to what I tested.

Kent.

PS: That bug I mentioned was causing corrupted vspace shadow state and could have potentially been the cause of that hypercall issue you were experiencing about a year ago. (Sorry I dropped the email thread about trying to solve it back then :-( )

[1]

https://github.com/seL4/seL4/commit/80d32b160989c55f9fdc33fce1bf60 5ac1b42f1e

https://github.com/SEL4PROJ/camkes-arm-vm/commit/ f3948cf41978bee33179ac7a814eefc4fe30b5c2

[2] https://developer.nvidia.com/embedded/tegra-k1-reference ------------------------------ *From:* Devel on behalf of Mike Clark < undefinedspace@gmail.com> *Sent:* Wednesday, June 20, 2018 10:35 PM *To:* Danis, Adrian (Data61, Kensington NSW) *Cc:* devel@sel4.systems *Subject:* Re: [seL4] Problems with tk1 nonsecure

Anyone else running into this issue with the latest clone? I had this all working just fine previously (maybe 6 months ago), but something must have been updated that breaks this. Not sure why.

Mike

On Tue, Jun 12, 2018 at 8:45 AM, Mike Clark wrote:

...

Adrian,

I am doing `make menuconfig`, then going to "Applications" and checking the box that says "Insecure: Give all devices to VM component". That is the only change I do, then recompile, copy the new image to my SD card, and boot.

Mike

On Mon, Jun 11, 2018 at 9:20 PM, wrote:

...

Hi Mike,

In order to be acting as a hypervisor in the default configuration that you say booted fine seL4 will already be setup to run in nonsecure hypervisor mode. Given this can you elaborate on what 'the unsecure mode' is and how you are switching to it?

Adrian

...

I just checked out a fresh copy of https://github.com/ SEL4PROJ/camkes-arm-vm-manifest and built an image for the TK1 using

On Sun 10-Jun-2018 10:08 PM, Mike Clark wrote: the

...

docker build environment. I copied the resulting image to an SD card and it boots fine.

If I switch to the unsecure mode, I get an error on boot.

SMMU Address translation error: ID: 98 address: 0xae5be000 type: 6 direction: 0x0 IOPT permission: read 0x0 write 0x0 nonsecure 0x0 [ 10.204777] mmc1: ADMA error

I have not made any changes, so my vm.smmu settings are all default.

Any thoughts or help?

Mike

_______________________________________________ Devel mailing list Devel@sel4.systems https://sel4.systems/lists/listinfo/devel

Kent, After syncing the repos to the latest and following your suggestion, I'm now back to where I started with this error: *SMMU Address translation error:* *ID: 98 address: 0xae5bd000 type: 6 direction: 0x0* *IOPT permission: read 0x0 write 0x0 nonsecure 0x0* *[ 10.468422] mmc1: ADMA error* As I mentioned before, I am booting off MMC, so maybe that is what the issue is. I'll try to get TFTP setup to make sure the problem doesn't persist. Thought you'd like to know though that there is still an issue. Mike On Mon, Jun 25, 2018 at 3:54 AM, wrote:

Hi Mike,

For some reason, the Tk1Insecure setting gets reset to false during the initial configuration which is why you can't see the ethernet card. Running the below command, or opening the cmake cache using (ccmake .) you can correct the setting.

cmake -DTk1Insecure=True . && ninja

Kent. ------------------------------ *From:* Mike Clark *Sent:* Friday, June 22, 2018 12:01 AM *To:* Mcleod, Kent (Data61, Kensington NSW) *Cc:* Danis, Adrian (Data61, Kensington NSW); devel@sel4.systems

*Subject:* Re: [seL4] Problems with tk1 nonsecure

Kent,

Thanks. Glad you were able to fix that bug. I noticed that the build system changed yesterday and tried with it, but was running into some different issues. Looks like that is all fixed now today as I was able to boot just fine.

One thing I noticed when I booted today is that I don't appear to have access to ethernet any more. `ip addr` use to give me an eth0 device when I was last using this. With this new image, I was expecting to see an `enp0s1` or something like that, but I don't get one. Has that device been removed from the insecure mode?

One thing I was doing before that I'd like to get back to is booting off a separate partition on my SD card that has it's own linux image on it. The partition I'm using comes from an image that you had provided to me. Is there an easy way to get back to that booting configuration?

Finally, just to let you know. Once I get this all back up and running I'll test the issue that I was having previously. But, just so you know, I had figured out a work around for that issue, so it didn't stop me from doing the testing I wanted to do.

Mike

On Thu, Jun 21, 2018 at 9:12 AM, wrote:

...

Hi Mike,

I'm not sure what changes in the last 6 months would have changed your build from a working to broken state. The below commits[1] are the only changes that I am aware would change the behavior of smmu permissions.

ID: 98​ corresponds to sdmmc3 (there is a table in the tk1 reference manual, around page 719 [2]), and this corresponds to a vm.smmu id of 22, which is absent from the TK1INSECURE configuration.

Adding that ID made my VM crash---I tracked the cause down to a iommu page table initialisation bug here: https://github.com/SEL4P ROJ/seL4_projects_libs/commit/396542fc8b7d0ab41bf92d8850b3082da80c69e7

Fixing that allowed my vm to boot with the extra vm.smmu id of 22. (Note: I boot over tftpboot and not using an sdcard, so it also boots without id 22 for me, however the ID: 98 error you are getting very likely corresponds to id 22 being missing).

The latest checkout that I tested is this one:

https://github.com/SEL4PROJ/camkes-arm-vm-manifest/commit/8f 83660cdd98c51bdaa388f9bb77ce2ea5116203 Also note that yesterday we merged some refactors that migrated the project to use a new CMake based build system.

running the following:

mkdir build && cd build && ../init-build.sh -DAARCH32=TRUE -DTk1Insecure=True -DCAMKES_VM_APP=tk1_vm && ninja

should build you an equivalent image to what I tested.

Kent.

PS: That bug I mentioned was causing corrupted vspace shadow state and could have potentially been the cause of that hypercall issue you were experiencing about a year ago. (Sorry I dropped the email thread about trying to solve it back then :-( )

[1]

https://github.com/seL4/seL4/commit/80d32b160989c55f9fdc33fc e1bf605ac1b42f1e

https://github.com/SEL4PROJ/camkes-arm-vm/commit/f3948cf4197 8bee33179ac7a814eefc4fe30b5c2

[2] https://developer.nvidia.com/embedded/tegra-k1-reference ------------------------------ *From:* Devel on behalf of Mike Clark < undefinedspace@gmail.com> *Sent:* Wednesday, June 20, 2018 10:35 PM *To:* Danis, Adrian (Data61, Kensington NSW) *Cc:* devel@sel4.systems *Subject:* Re: [seL4] Problems with tk1 nonsecure

Anyone else running into this issue with the latest clone? I had this all working just fine previously (maybe 6 months ago), but something must have been updated that breaks this. Not sure why.

Mike

On Tue, Jun 12, 2018 at 8:45 AM, Mike Clark wrote:

...

Adrian,

I am doing `make menuconfig`, then going to "Applications" and checking the box that says "Insecure: Give all devices to VM component". That is the only change I do, then recompile, copy the new image to my SD card, and boot.

Mike

On Mon, Jun 11, 2018 at 9:20 PM, wrote:

...

Hi Mike,

In order to be acting as a hypervisor in the default configuration that you say booted fine seL4 will already be setup to run in nonsecure hypervisor mode. Given this can you elaborate on what 'the unsecure mode' is and how you are switching to it?

Adrian

...

I just checked out a fresh copy of https://github.com/ SEL4PROJ/camkes-arm-vm-manifest and built an image for the TK1 using

On Sun 10-Jun-2018 10:08 PM, Mike Clark wrote: the

...

docker build environment. I copied the resulting image to an SD card and it boots fine.

If I switch to the unsecure mode, I get an error on boot.

SMMU Address translation error: ID: 98 address: 0xae5be000 type: 6 direction: 0x0 IOPT permission: read 0x0 write 0x0 nonsecure 0x0 [ 10.204777] mmc1: ADMA error

I have not made any changes, so my vm.smmu settings are all default.

Any thoughts or help?

Mike

_______________________________________________ Devel mailing list Devel@sel4.systems https://sel4.systems/lists/listinfo/devel

Kent, I had forgotten about that step. Thanks! It works now. Mike On Mon, Jun 25, 2018 at 6:05 PM, wrote:

Hi Mike,

Do still get the error when you add ID 22 to this list yourself?: https://github.com/SEL4PROJ/camkes-arm-vm/blob/master/components/VM/vm_ common.camkes#L278​

Kent.

------------------------------ *From:* Mike Clark *Sent:* Tuesday, June 26, 2018 6:00 AM

*To:* Mcleod, Kent (Data61, Kensington NSW) *Cc:* Danis, Adrian (Data61, Kensington NSW); devel@sel4.systems *Subject:* Re: [seL4] Problems with tk1 nonsecure

Kent,

After syncing the repos to the latest and following your suggestion, I'm now back to where I started with this error:

*SMMU Address translation error:* *ID: 98 address: 0xae5bd000 type: 6 direction: 0x0* *IOPT permission: read 0x0 write 0x0 nonsecure 0x0* *[ 10.468422] mmc1: ADMA error*

As I mentioned before, I am booting off MMC, so maybe that is what the issue is. I'll try to get TFTP setup to make sure the problem doesn't persist. Thought you'd like to know though that there is still an issue.

Mike

On Mon, Jun 25, 2018 at 3:54 AM, wrote:

...

Hi Mike,

For some reason, the Tk1Insecure setting gets reset to false during the initial configuration which is why you can't see the ethernet card. Running the below command, or opening the cmake cache using (ccmake .) you can correct the setting.

cmake -DTk1Insecure=True . && ninja

Kent. ------------------------------ *From:* Mike Clark *Sent:* Friday, June 22, 2018 12:01 AM *To:* Mcleod, Kent (Data61, Kensington NSW) *Cc:* Danis, Adrian (Data61, Kensington NSW); devel@sel4.systems

*Subject:* Re: [seL4] Problems with tk1 nonsecure

Kent,

Thanks. Glad you were able to fix that bug. I noticed that the build system changed yesterday and tried with it, but was running into some different issues. Looks like that is all fixed now today as I was able to boot just fine.

One thing I noticed when I booted today is that I don't appear to have access to ethernet any more. `ip addr` use to give me an eth0 device when I was last using this. With this new image, I was expecting to see an `enp0s1` or something like that, but I don't get one. Has that device been removed from the insecure mode?

One thing I was doing before that I'd like to get back to is booting off a separate partition on my SD card that has it's own linux image on it. The partition I'm using comes from an image that you had provided to me. Is there an easy way to get back to that booting configuration?

Finally, just to let you know. Once I get this all back up and running I'll test the issue that I was having previously. But, just so you know, I had figured out a work around for that issue, so it didn't stop me from doing the testing I wanted to do.

Mike

On Thu, Jun 21, 2018 at 9:12 AM, wrote:

...

Hi Mike,

I'm not sure what changes in the last 6 months would have changed your build from a working to broken state. The below commits[1] are the only changes that I am aware would change the behavior of smmu permissions.

ID: 98​ corresponds to sdmmc3 (there is a table in the tk1 reference manual, around page 719 [2]), and this corresponds to a vm.smmu id of 22, which is absent from the TK1INSECURE configuration.

Adding that ID made my VM crash---I tracked the cause down to a iommu page table initialisation bug here: https://github.com/SEL4P ROJ/seL4_projects_libs/commit/396542fc8b7d0ab41bf92d8850b3082da80c69e7

Fixing that allowed my vm to boot with the extra vm.smmu id of 22. (Note: I boot over tftpboot and not using an sdcard, so it also boots without id 22 for me, however the ID: 98 error you are getting very likely corresponds to id 22 being missing).

The latest checkout that I tested is this one:

https://github.com/SEL4PROJ/camkes-arm-vm-manifest/commit/8f 83660cdd98c51bdaa388f9bb77ce2ea5116203 Also note that yesterday we merged some refactors that migrated the project to use a new CMake based build system.

running the following:

mkdir build && cd build && ../init-build.sh -DAARCH32=TRUE -DTk1Insecure=True -DCAMKES_VM_APP=tk1_vm && ninja

should build you an equivalent image to what I tested.

Kent.

PS: That bug I mentioned was causing corrupted vspace shadow state and could have potentially been the cause of that hypercall issue you were experiencing about a year ago. (Sorry I dropped the email thread about trying to solve it back then :-( )

[1]

https://github.com/seL4/seL4/commit/80d32b160989c55f9fdc33fc e1bf605ac1b42f1e

https://github.com/SEL4PROJ/camkes-arm-vm/commit/f3948cf4197 8bee33179ac7a814eefc4fe30b5c2

[2] https://developer.nvidia.com/embedded/tegra-k1-reference ------------------------------ *From:* Devel on behalf of Mike Clark < undefinedspace@gmail.com> *Sent:* Wednesday, June 20, 2018 10:35 PM *To:* Danis, Adrian (Data61, Kensington NSW) *Cc:* devel@sel4.systems *Subject:* Re: [seL4] Problems with tk1 nonsecure

Anyone else running into this issue with the latest clone? I had this all working just fine previously (maybe 6 months ago), but something must have been updated that breaks this. Not sure why.

Mike

On Tue, Jun 12, 2018 at 8:45 AM, Mike Clark wrote:

...

Adrian,

I am doing `make menuconfig`, then going to "Applications" and checking the box that says "Insecure: Give all devices to VM component". That is the only change I do, then recompile, copy the new image to my SD card, and boot.

Mike

On Mon, Jun 11, 2018 at 9:20 PM, wrote:

...

Hi Mike,

In order to be acting as a hypervisor in the default configuration that you say booted fine seL4 will already be setup to run in nonsecure hypervisor mode. Given this can you elaborate on what 'the unsecure mode' is and how you are switching to it?

Adrian

On Sun 10-Jun-2018 10:08 PM, Mike Clark wrote:

...

I just checked out a fresh copy of https://github.com/ SEL4PROJ/camkes-arm-vm-manifest and built an image for the TK1 using the docker build environment. I copied the resulting image to an SD card and it boots fine.

If I switch to the unsecure mode, I get an error on boot.

SMMU Address translation error: ID: 98 address: 0xae5be000 type: 6 direction: 0x0 IOPT permission: read 0x0 write 0x0 nonsecure 0x0 [ 10.204777] mmc1: ADMA error

I have not made any changes, so my vm.smmu settings are all default.

Any thoughts or help?

Mike

_______________________________________________ Devel mailing list Devel@sel4.systems https://sel4.systems/lists/listinfo/devel