答复: Devel Digest, Vol 130, Issue 2
Hi Matthew: Firstly, thank you for your information. form your answer, we understand the design intention of group, we Iook at compilation proces and may be find the probelm of compiler, we the assume the following scenario: The client A and client B are in the same group, they all have the same global variable like "global_buf", when use objcopy with "--keep-global-symbol" param, it will change the "global_buf" symbol form "gloabl" to "local", this change make the same name global variable of different client in same space can work well, they can access their own global variable as local variable. But In out test, we find if the "global_buf" not be initialized, in the group, there is only one "global_buf", if client A change the value of "global_buf", client B also can see the change, this is may be wrong, we find the gcc objcopy with "--keep-global-symbol" can only affect ".data" and ".bss" section, but "global_buf" which not be initialized, will place in ".common" section, "--keep-global-symbol" will not change the symbol of ".common" form global to local, this may be solved by add "-fno-common" when linker in camkes project related to "group" scenario ________________________________ 发件人: devel-request@sel4.systems <devel-request@sel4.systems> 发送时间: 2021年12月28日 9:00:05 收件人: devel@sel4.systems 主题: Devel Digest, Vol 130, Issue 2 Send Devel mailing list submissions to devel@sel4.systems To subscribe or unsubscribe via email, send a message with subject or body 'help' to devel-request@sel4.systems You can reach the person managing the list at devel-owner@sel4.systems When replying, please edit your Subject line so it is more specific than "Re: Contents of Devel digest..." Today's Topics: 1. Re: Some questions about group of camkes (adjust the format) (Matthew Fernandez) ---------------------------------------------------------------------- Message: 1 Date: Mon, 27 Dec 2021 16:26:26 -0800 From: Matthew Fernandez <matthew.fernandez@gmail.com> Subject: [seL4] Re: Some questions about group of camkes (adjust the format) To: "yadong.li" <yadong.li@horizon.ai> Cc: "devel@sel4.systems" <devel@sel4.systems>, "xiaonan.wang" <xiaonan.wang@horizon.ai>, "lei.mao" <lei.mao@horizon.ai> Message-ID: <DA97FC78-B670-4CF7-88AC-9810B31D1015@gmail.com> Content-Type: text/plain; charset=utf-8
On Dec 23, 2021, at 08:50, yadong.li <yadong.li@horizon.ai> wrote:
Hi
* We learn the group of camkes from the https://github.com/seL4/camkes-tool/blob/master/docs/index.md * group can colocate two component instances in a single address space, we want to use the share var by this way, like directcall transfer string virtual address, But we feel confused about the global symbols who have the same symbol name include function and variable in a single address space, Their behavior seems undefined 。
My information on how this works is out of date and I can’t answer all your questions, but I can tell you how this feature was originally implemented. Single address space components were originally implemented using post-compilation symbol mangling. As you’ve noticed, naive linking of two component instances together presents two problems: 1. Unrelated homonyms between the instances now conflict. A symbol called `foo` in component instance A and a symbol called `foo` in component instance B now refer to the same thing when linked together. 2. References to glue code have the opposite problem: their names may differ in component instance A and component instance B, but need to refer to the same functions when the two are linked together. Both problems were solved with the same mechanism: GNU objcopy. The invocations to objcopy were template-generated so they could take advantage of knowledge about the system begin compiled. A generated objcopy invocation would do the following: 1. Adjust all non-generated symbols to have internal visibility. Component instance A and component instance B have already been (partially) linked, so at this point the only unresolved symbols that need to remain externally visible are those related to the connection(s) between them. This solves problem 1 above. 2. Name-mangle all remaining symbols to something prefixed with the relevant connection instance’s name. IIRC the name mangling scheme was something like ‘<connection instance name><space><original symbol name>’. This guaranteed uniqueness because <space> is not a character that existed in C or assembly symbols. Whether using a space in a symbol name is legal or not, I don’t know, but all the binutils seemed fine with this. 3. Rename the second component in the name mangling above to a common name. I don’t recall exactly how this name was chosen, but this solves problem 2 above. This sounds pretty unorthodox and brittle, but it actually worked surprisingly well. All combinations of single address space component systems seemed to Just Work, with a few notable sharp edges: * Anything involving MMIO was tricky. These symbols frequently needed to remain externally visible and the two component instances would often have differing names but the same addresses for them. * GNU ld was more or less required. The multiple steps involving partial linking was only supported by GNU ld and Gold at the time. LLVM’s lld may have caught up in the interim years. * The objcopy name mangling broke cross-section references used by GCC’s implementation of Link-Time Optimization. As a result, any LTO compilation degraded to LTO being disabled. This wouldn’t have been a big deal except that one of the primary reasons to put two components in a single address space is to enable cross-component inlining, usually facilitated by LTO. AFAICT this (playing objcopy tricks and expecting LTO to still work) was simply not a supported use case. We explored how to work around this and got some one-off efforts working for benchmarking, but proper support would have involved altering the way binutils work. ------------------------------ Subject: Digest Footer _______________________________________________ Devel mailing list -- devel@sel4.systems To unsubscribe send an email to devel-leave@sel4.systems ------------------------------ End of Devel Digest, Vol 130, Issue 2 *************************************
[Altered the subject back to something more specific] What you describe certainly sounds like a problem, yes. But how do you end up with a symbol like that in “.common”? I would have thought “global_buf” in your scenario is either an initialized top-level variable (in which case it will go in “.data” or “.rodata”) or an uninitialized top-level variable (in which case it will go in “.bss”). To get “global_buf” in “.common” I thought you would have to be (incorrectly) defining the same global in multiple translation units within the same component. It’s possible I’m over-simplifying the compiler’s actions here or CAmkES itself is somehow incorrectly defining a generated global in multiple translation units. FYI -fno-common becomes the default in GCC ≥ 10, https://gcc.gnu.org/bugzilla/show_bug.cgi?id=85678
On Jan 2, 2022, at 07:30, yadong.li <yadong.li@horizon.ai> wrote:
Hi Matthew:
Firstly, thank you for your information.
form your answer, we understand the design intention of group, we Iook at compilation proces and may be find the probelm of compiler, we the assume the following scenario:
The client A and client B are in the same group, they all have the same global variable like "global_buf", when use objcopy with "--keep-global-symbol" param, it will change the "global_buf" symbol form "gloabl" to "local", this change make the same name global variable of different client in same space can work well, they can access their own global variable as local variable.
But In out test, we find if the "global_buf" not be initialized, in the group, there is only one
"global_buf", if client A change the value of "global_buf", client B also can see the change, this is may be wrong, we find the gcc objcopy with "--keep-global-symbol" can only affect ".data" and ".bss" section, but "global_buf" which not be initialized, will place in ".common" section, "--keep-global-symbol" will not change the symbol of ".common" form global to local, this may be solved by add "-fno-common" when linker in camkes project related to "group" scenario
________________________________ 发件人: devel-request@sel4.systems <devel-request@sel4.systems> 发送时间: 2021年12月28日 9:00:05 收件人: devel@sel4.systems 主题: Devel Digest, Vol 130, Issue 2
Send Devel mailing list submissions to devel@sel4.systems
To subscribe or unsubscribe via email, send a message with subject or body 'help' to devel-request@sel4.systems
You can reach the person managing the list at devel-owner@sel4.systems
When replying, please edit your Subject line so it is more specific than "Re: Contents of Devel digest..."
Today's Topics:
1. Re: Some questions about group of camkes (adjust the format) (Matthew Fernandez)
----------------------------------------------------------------------
Message: 1 Date: Mon, 27 Dec 2021 16:26:26 -0800 From: Matthew Fernandez <matthew.fernandez@gmail.com> Subject: [seL4] Re: Some questions about group of camkes (adjust the format) To: "yadong.li" <yadong.li@horizon.ai> Cc: "devel@sel4.systems" <devel@sel4.systems>, "xiaonan.wang" <xiaonan.wang@horizon.ai>, "lei.mao" <lei.mao@horizon.ai> Message-ID: <DA97FC78-B670-4CF7-88AC-9810B31D1015@gmail.com> Content-Type: text/plain; charset=utf-8
On Dec 23, 2021, at 08:50, yadong.li <yadong.li@horizon.ai> wrote:
Hi
* We learn the group of camkes from the https://github.com/seL4/camkes-tool/blob/master/docs/index.md * group can colocate two component instances in a single address space, we want to use the share var by this way, like directcall transfer string virtual address, But we feel confused about the global symbols who have the same symbol name include function and variable in a single address space, Their behavior seems undefined 。
My information on how this works is out of date and I can’t answer all your questions, but I can tell you how this feature was originally implemented.
Single address space components were originally implemented using post-compilation symbol mangling. As you’ve noticed, naive linking of two component instances together presents two problems: 1. Unrelated homonyms between the instances now conflict. A symbol called `foo` in component instance A and a symbol called `foo` in component instance B now refer to the same thing when linked together. 2. References to glue code have the opposite problem: their names may differ in component instance A and component instance B, but need to refer to the same functions when the two are linked together.
Both problems were solved with the same mechanism: GNU objcopy. The invocations to objcopy were template-generated so they could take advantage of knowledge about the system begin compiled. A generated objcopy invocation would do the following: 1. Adjust all non-generated symbols to have internal visibility. Component instance A and component instance B have already been (partially) linked, so at this point the only unresolved symbols that need to remain externally visible are those related to the connection(s) between them. This solves problem 1 above. 2. Name-mangle all remaining symbols to something prefixed with the relevant connection instance’s name. IIRC the name mangling scheme was something like ‘<connection instance name><space><original symbol name>’. This guaranteed uniqueness because <space> is not a character that existed in C or assembly symbols. Whether using a space in a symbol name is legal or not, I don’t know, but all the binutils seemed fine with this. 3. Rename the second component in the name mangling above to a common name. I don’t recall exactly how this name was chosen, but this solves problem 2 above.
This sounds pretty unorthodox and brittle, but it actually worked surprisingly well. All combinations of single address space component systems seemed to Just Work, with a few notable sharp edges: * Anything involving MMIO was tricky. These symbols frequently needed to remain externally visible and the two component instances would often have differing names but the same addresses for them. * GNU ld was more or less required. The multiple steps involving partial linking was only supported by GNU ld and Gold at the time. LLVM’s lld may have caught up in the interim years. * The objcopy name mangling broke cross-section references used by GCC’s implementation of Link-Time Optimization. As a result, any LTO compilation degraded to LTO being disabled. This wouldn’t have been a big deal except that one of the primary reasons to put two components in a single address space is to enable cross-component inlining, usually facilitated by LTO. AFAICT this (playing objcopy tricks and expecting LTO to still work) was simply not a supported use case. We explored how to work around this and got some one-off efforts working for benchmarking, but proper support would have involved altering the way binutils work.
------------------------------
Subject: Digest Footer
_______________________________________________ Devel mailing list -- devel@sel4.systems To unsubscribe send an email to devel-leave@sel4.systems
------------------------------
End of Devel Digest, Vol 130, Issue 2 ************************************* _______________________________________________ Devel mailing list -- devel@sel4.systems To unsubscribe send an email to devel-leave@sel4.systems
participants (2)
-
Matthew Fernandez -
yadong.li