All, Any thoughts on conducting panels at the seL4 Summit? I was thinking about writing an abstract for a presentation on DornerWorks' experience with mainlining. Focusing on the good pieces, the challenges (which come from our customers, our limited funds, our process, from github, and from the foundation processes), and some thoughts on how to improve. My colleagues thought it could be better as a panel so that we could get perspectives from others in the community, as well as the foundation. I wholly agree with them. So I was wondering if others thought a panel or two would be a good idea. If so, should it still be submitted as an abstract? Thanks, Robbie VanVossen
Robbie,
I was thinking about writing an abstract for a presentation on DornerWorks’ experience with mainlining. Focusing on the good pieces, the challenges (which come from our customers, our limited funds, our process, from github, and from the foundation processes), and some thoughts on how to improve. My colleagues thought it could be better as a panel so that we could get perspectives from others in the community, as well as the foundation. I wholly agree with them. So I was wondering if others thought a panel or two would be a good idea. If so, should it still be submitted as an abstract?
My opinion is, that a presentation would be a good input to kick off an open discussion on the topic. But I have some doubts that a panel discussion for a general audience with a few selected speakers is really a way forward. It would take a few more presentations how others do it and then see what could be adapted. Seems more a topic for a break-out session or a working group with dedicated resources that can improve things. Axel
I agree that the specific topic may be too narrow for a panel, but would be a great presentation (structured to encourage discussion). Another option would be to broaden the scope and make it a panel - e.g. to have a panel about contribution in general. Ihor.
On 26 Apr 2022, at 03:18, Heider, Axel <axel.heider@hensoldt.net> wrote:
Robbie,
I was thinking about writing an abstract for a presentation on DornerWorks’ experience with mainlining. Focusing on the good pieces, the challenges (which come from our customers, our limited funds, our process, from github, and from the foundation processes), and some thoughts on how to improve. My colleagues thought it could be better as a panel so that we could get perspectives from others in the community, as well as the foundation. I wholly agree with them. So I was wondering if others thought a panel or two would be a good idea. If so, should it still be submitted as an abstract?
My opinion is, that a presentation would be a good input to kick off an open discussion on the topic. But I have some doubts that a panel discussion for a general audience with a few selected speakers is really a way forward. It would take a few more presentations how others do it and then see what could be adapted. Seems more a topic for a break-out session or a working group with dedicated resources that can improve things.
Axel _______________________________________________ Summit-pc mailing list -- summit-pc@sel4.systems To unsubscribe send an email to summit-pc-leave@sel4.systems
I'd actually be interested in hearing about problems with GitHub and processes long before the summit. Maybe they can even be fixed by then. Is there anything specific that should be improved? Cheers, Gerwin
On 27 Apr 2022, at 10:53, Ihor Kuz <ihor@kry10.com> wrote:
I agree that the specific topic may be too narrow for a panel, but would be a great presentation (structured to encourage discussion).
Another option would be to broaden the scope and make it a panel - e.g. to have a panel about contribution in general.
Ihor.
On 26 Apr 2022, at 03:18, Heider, Axel <axel.heider@hensoldt.net> wrote:
Robbie,
I was thinking about writing an abstract for a presentation on DornerWorks’ experience with mainlining. Focusing on the good pieces, the challenges (which come from our customers, our limited funds, our process, from github, and from the foundation processes), and some thoughts on how to improve. My colleagues thought it could be better as a panel so that we could get perspectives from others in the community, as well as the foundation. I wholly agree with them. So I was wondering if others thought a panel or two would be a good idea. If so, should it still be submitted as an abstract?
My opinion is, that a presentation would be a good input to kick off an open discussion on the topic. But I have some doubts that a panel discussion for a general audience with a few selected speakers is really a way forward. It would take a few more presentations how others do it and then see what could be adapted. Seems more a topic for a break-out session or a working group with dedicated resources that can improve things.
Axel _______________________________________________ Summit-pc mailing list -- summit-pc@sel4.systems To unsubscribe send an email to summit-pc-leave@sel4.systems
_______________________________________________ Summit-pc mailing list -- summit-pc@sel4.systems To unsubscribe send an email to summit-pc-leave@sel4.systems
I'd actually be interested in hearing about problems with GitHub and processes long before the summit. Maybe they can even be fixed by then.
Is there anything specific that should be improved?
I suppose I don't have any major gripes with github. It is actually quite impressive with CI. I haven’t actually gathered all my thoughts yet on the subject. I think gerrit is a better review tool for open source, because it is much easier to compare various rebases, but the convenience of having reviews, issue tracking, and the code all in a single location really makes up for less than perfect reviewing. Which is the same reason we use gitlab internally instead of gerrit. Process issues: - Reviews tend to go stale. My guess is that this is mostly due to lack of resources. Hopefully the dedicated reviewer role can help with this. - Some of the reviews that are stale are in our corner as well. We are also resource constrained. - Hard to reach a conclusion on RFCs. Maybe this is on us for not continuing the discussion, but the conclusion of https://sel4.atlassian.net/browse/RFC-9 was that we(the community, the foundation?) should think through more implementation options. Changes like this, which are needed to support basic customer features, sitting around for a while results in us needing to push our patches along as the code updates over time. This is also made harder by the multiple repositories. I am not sure there is a great solution for this. Some thoughts are accepting changes like these with the knowledge that they will get incremental improvements. Alternatively, maybe there could be an experimental or staging branch that periodically gets updated. Thanks, Robbie -----Original Message----- From: Gerwin Klein <kleing@unsw.edu.au> Sent: Tuesday, April 26, 2022 9:13 PM To: ihor@kry10.com Cc: Heider, Axel <axel.heider@hensoldt.net>; summit-pc@sel4.systems; Robert VanVossen <Robert.VanVossen@dornerworks.com> Subject: Re: [Summit-pc] seL4 Summit Panels CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. I'd actually be interested in hearing about problems with GitHub and processes long before the summit. Maybe they can even be fixed by then. Is there anything specific that should be improved? Cheers, Gerwin
On 27 Apr 2022, at 10:53, Ihor Kuz <ihor@kry10.com> wrote:
I agree that the specific topic may be too narrow for a panel, but would be a great presentation (structured to encourage discussion).
Another option would be to broaden the scope and make it a panel - e.g. to have a panel about contribution in general.
Ihor.
On 26 Apr 2022, at 03:18, Heider, Axel <axel.heider@hensoldt.net> wrote:
Robbie,
I was thinking about writing an abstract for a presentation on DornerWorks’ experience with mainlining. Focusing on the good pieces, the challenges (which come from our customers, our limited funds, our process, from github, and from the foundation processes), and some thoughts on how to improve. My colleagues thought it could be better as a panel so that we could get perspectives from others in the community, as well as the foundation. I wholly agree with them. So I was wondering if others thought a panel or two would be a good idea. If so, should it still be submitted as an abstract?
My opinion is, that a presentation would be a good input to kick off an open discussion on the topic. But I have some doubts that a panel discussion for a general audience with a few selected speakers is really a way forward. It would take a few more presentations how others do it and then see what could be adapted. Seems more a topic for a break-out session or a working group with dedicated resources that can improve things.
Axel _______________________________________________ Summit-pc mailing list -- summit-pc@sel4.systems To unsubscribe send an email to summit-pc-leave@sel4.systems
_______________________________________________ Summit-pc mailing list -- summit-pc@sel4.systems To unsubscribe send an email to summit-pc-leave@sel4.systems
These are some good points. You are right that expert reviews are severely resource constrained and prioritisation tends to follow activity on the PR, i.e. if the contributor can respond quickly to questions/comments, the likelihood that things can get merged quickly is significantly higher than for other PRs. That is mostly because the context switching cost for some of these PRs is high. This is pretty common in open source projects (pretty much the default situation), but it is of course not exactly optimal. More reviewers will help, which is starting to happen but will take time. What can also help is to just ping on a stale PR. We could potentially automate some of this part, but we'll have to think about doing this in a way that actually improves things. Suggestions on that are welcome. RFCs that are hard to reach conclusions on: I should probably take a more active role on pushing these forward as the chair. The expectation is that the proposer advances the discussion when it goes stale, because presumably they want the RFC to be accepted. This has worked on the majority of RFCs so far, but it is true that it is not a fast process. I don't think RFCs should be a particularly fast process, but there is of course no point in unproductive delays, and some obvious RFCs are done in 2 weeks total (minimum of 2 weeks to give people time to raise issues). As you said, the last comment on the SMC RFC in particular is that a more detailed argument is needed. It may be a basic customer feature, but the way it currently looks, accepting it would kill the WCET guarantees seL4 provides so far. So its implementation implications are deep and subtle and require a principled argument. This work needs to be done by someone. The only people available for that are the proposers or volunteers. As far as I can see, in general RFCs get blocked when more work is requested or necessary, but nobody is available to do that work. The expectation is on the proposer for that, but some features are of course of broader interest and the community would help. I'd think that RFC-9 is one of these, but in the end all of this comes down to time availability from experts being in short supply. We're have hopefully passed the low point of that now, at least the trajectory of how many people are around with seL4 expertise is looking good. Another option is to ask the foundation board to allocate money to a particular feature and either hire or contract for someone to work on it. I would be against accepting such changes with "we will fix problems later", because a) if they have no path to being fixed now they will never be fixed, and b) seL4 is then immediately no longer a high-assurance kernel. The situation is different if it is clear what needs to be done and it's just work that needs to be scheduled on an optional feature, but in this case, the decision might be that a completely different design is necessary. In general for high assurance, changes that are not fully understood should not be accepted. You can see the same in some pull requests where people argue "changing this line makes my problem go away". This is a red flag. If the contributor cannot argue what the problem is and why/how the change fixes it, it is better to have a known problem than a change with unknown cause or consequences. (Of course such a proposed change might still be right, but it is just the very first step in fixing a problem, not the last step). Cheers, Gerwin
On 27 Apr 2022, at 22:37, Robert VanVossen <Robert.VanVossen@dornerworks.com> wrote:
I'd actually be interested in hearing about problems with GitHub and processes long before the summit. Maybe they can even be fixed by then.
Is there anything specific that should be improved?
I suppose I don't have any major gripes with github. It is actually quite impressive with CI. I haven’t actually gathered all my thoughts yet on the subject. I think gerrit is a better review tool for open source, because it is much easier to compare various rebases, but the convenience of having reviews, issue tracking, and the code all in a single location really makes up for less than perfect reviewing. Which is the same reason we use gitlab internally instead of gerrit.
Process issues: - Reviews tend to go stale. My guess is that this is mostly due to lack of resources. Hopefully the dedicated reviewer role can help with this. - Some of the reviews that are stale are in our corner as well. We are also resource constrained. - Hard to reach a conclusion on RFCs. Maybe this is on us for not continuing the discussion, but the conclusion of https://sel4.atlassian.net/browse/RFC-9 was that we(the community, the foundation?) should think through more implementation options. Changes like this, which are needed to support basic customer features, sitting around for a while results in us needing to push our patches along as the code updates over time. This is also made harder by the multiple repositories. I am not sure there is a great solution for this. Some thoughts are accepting changes like these with the knowledge that they will get incremental improvements. Alternatively, maybe there could be an experimental or staging branch that periodically gets updated.
Thanks, Robbie
-----Original Message----- From: Gerwin Klein <kleing@unsw.edu.au> Sent: Tuesday, April 26, 2022 9:13 PM To: ihor@kry10.com Cc: Heider, Axel <axel.heider@hensoldt.net>; summit-pc@sel4.systems; Robert VanVossen <Robert.VanVossen@dornerworks.com> Subject: Re: [Summit-pc] seL4 Summit Panels
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
I'd actually be interested in hearing about problems with GitHub and processes long before the summit. Maybe they can even be fixed by then.
Is there anything specific that should be improved?
Cheers, Gerwin
On 27 Apr 2022, at 10:53, Ihor Kuz <ihor@kry10.com> wrote:
I agree that the specific topic may be too narrow for a panel, but would be a great presentation (structured to encourage discussion).
Another option would be to broaden the scope and make it a panel - e.g. to have a panel about contribution in general.
Ihor.
On 26 Apr 2022, at 03:18, Heider, Axel <axel.heider@hensoldt.net> wrote:
Robbie,
I was thinking about writing an abstract for a presentation on DornerWorks’ experience with mainlining. Focusing on the good pieces, the challenges (which come from our customers, our limited funds, our process, from github, and from the foundation processes), and some thoughts on how to improve. My colleagues thought it could be better as a panel so that we could get perspectives from others in the community, as well as the foundation. I wholly agree with them. So I was wondering if others thought a panel or two would be a good idea. If so, should it still be submitted as an abstract?
My opinion is, that a presentation would be a good input to kick off an open discussion on the topic. But I have some doubts that a panel discussion for a general audience with a few selected speakers is really a way forward. It would take a few more presentations how others do it and then see what could be adapted. Seems more a topic for a break-out session or a working group with dedicated resources that can improve things.
Axel _______________________________________________ Summit-pc mailing list -- summit-pc@sel4.systems To unsubscribe send an email to summit-pc-leave@sel4.systems
_______________________________________________ Summit-pc mailing list -- summit-pc@sel4.systems To unsubscribe send an email to summit-pc-leave@sel4.systems
Hi all, sorry for my late reply and thanks for the discussions. Regarding the initial question about talk submission vs panel, I would suggest to submit the topic anyway, and we can discuss during the PC meeting what would be the best way to address it. Panels are an option. But a Birds of a feather / smaller group discussion is also an option for this topic. Regarding panels, I would generally like to consider AMA (Ask Me Anything) sessions, potentially as an alternative to panels, as my experience is that this framing tends to create more discussion, and more questions from the audience. All this can be discussed in the PC meeting. But having all topics of interest is important for this discussion, so I’d suggest to send a submission :) Cheers June
On 4 May 2022, at 9:09 am, Gerwin Klein <kleing@unsw.edu.au> wrote:
These are some good points.
You are right that expert reviews are severely resource constrained and prioritisation tends to follow activity on the PR, i.e. if the contributor can respond quickly to questions/comments, the likelihood that things can get merged quickly is significantly higher than for other PRs. That is mostly because the context switching cost for some of these PRs is high.
This is pretty common in open source projects (pretty much the default situation), but it is of course not exactly optimal. More reviewers will help, which is starting to happen but will take time. What can also help is to just ping on a stale PR. We could potentially automate some of this part, but we'll have to think about doing this in a way that actually improves things. Suggestions on that are welcome.
RFCs that are hard to reach conclusions on: I should probably take a more active role on pushing these forward as the chair. The expectation is that the proposer advances the discussion when it goes stale, because presumably they want the RFC to be accepted. This has worked on the majority of RFCs so far, but it is true that it is not a fast process. I don't think RFCs should be a particularly fast process, but there is of course no point in unproductive delays, and some obvious RFCs are done in 2 weeks total (minimum of 2 weeks to give people time to raise issues).
As you said, the last comment on the SMC RFC in particular is that a more detailed argument is needed. It may be a basic customer feature, but the way it currently looks, accepting it would kill the WCET guarantees seL4 provides so far. So its implementation implications are deep and subtle and require a principled argument. This work needs to be done by someone.
The only people available for that are the proposers or volunteers. As far as I can see, in general RFCs get blocked when more work is requested or necessary, but nobody is available to do that work. The expectation is on the proposer for that, but some features are of course of broader interest and the community would help. I'd think that RFC-9 is one of these, but in the end all of this comes down to time availability from experts being in short supply. We're have hopefully passed the low point of that now, at least the trajectory of how many people are around with seL4 expertise is looking good. Another option is to ask the foundation board to allocate money to a particular feature and either hire or contract for someone to work on it.
I would be against accepting such changes with "we will fix problems later", because a) if they have no path to being fixed now they will never be fixed, and b) seL4 is then immediately no longer a high-assurance kernel. The situation is different if it is clear what needs to be done and it's just work that needs to be scheduled on an optional feature, but in this case, the decision might be that a completely different design is necessary.
In general for high assurance, changes that are not fully understood should not be accepted. You can see the same in some pull requests where people argue "changing this line makes my problem go away". This is a red flag. If the contributor cannot argue what the problem is and why/how the change fixes it, it is better to have a known problem than a change with unknown cause or consequences. (Of course such a proposed change might still be right, but it is just the very first step in fixing a problem, not the last step).
Cheers, Gerwin
On 27 Apr 2022, at 22:37, Robert VanVossen <Robert.VanVossen@dornerworks.com> wrote:
I'd actually be interested in hearing about problems with GitHub and processes long before the summit. Maybe they can even be fixed by then.
Is there anything specific that should be improved?
I suppose I don't have any major gripes with github. It is actually quite impressive with CI. I haven’t actually gathered all my thoughts yet on the subject. I think gerrit is a better review tool for open source, because it is much easier to compare various rebases, but the convenience of having reviews, issue tracking, and the code all in a single location really makes up for less than perfect reviewing. Which is the same reason we use gitlab internally instead of gerrit.
Process issues: - Reviews tend to go stale. My guess is that this is mostly due to lack of resources. Hopefully the dedicated reviewer role can help with this. - Some of the reviews that are stale are in our corner as well. We are also resource constrained. - Hard to reach a conclusion on RFCs. Maybe this is on us for not continuing the discussion, but the conclusion of https://sel4.atlassian.net/browse/RFC-9 was that we(the community, the foundation?) should think through more implementation options. Changes like this, which are needed to support basic customer features, sitting around for a while results in us needing to push our patches along as the code updates over time. This is also made harder by the multiple repositories. I am not sure there is a great solution for this. Some thoughts are accepting changes like these with the knowledge that they will get incremental improvements. Alternatively, maybe there could be an experimental or staging branch that periodically gets updated.
Thanks, Robbie
-----Original Message----- From: Gerwin Klein <kleing@unsw.edu.au> Sent: Tuesday, April 26, 2022 9:13 PM To: ihor@kry10.com Cc: Heider, Axel <axel.heider@hensoldt.net>; summit-pc@sel4.systems; Robert VanVossen <Robert.VanVossen@dornerworks.com> Subject: Re: [Summit-pc] seL4 Summit Panels
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
I'd actually be interested in hearing about problems with GitHub and processes long before the summit. Maybe they can even be fixed by then.
Is there anything specific that should be improved?
Cheers, Gerwin
On 27 Apr 2022, at 10:53, Ihor Kuz <ihor@kry10.com> wrote:
I agree that the specific topic may be too narrow for a panel, but would be a great presentation (structured to encourage discussion).
Another option would be to broaden the scope and make it a panel - e.g. to have a panel about contribution in general.
Ihor.
On 26 Apr 2022, at 03:18, Heider, Axel <axel.heider@hensoldt.net> wrote:
Robbie,
I was thinking about writing an abstract for a presentation on DornerWorks’ experience with mainlining. Focusing on the good pieces, the challenges (which come from our customers, our limited funds, our process, from github, and from the foundation processes), and some thoughts on how to improve. My colleagues thought it could be better as a panel so that we could get perspectives from others in the community, as well as the foundation. I wholly agree with them. So I was wondering if others thought a panel or two would be a good idea. If so, should it still be submitted as an abstract?
My opinion is, that a presentation would be a good input to kick off an open discussion on the topic. But I have some doubts that a panel discussion for a general audience with a few selected speakers is really a way forward. It would take a few more presentations how others do it and then see what could be adapted. Seems more a topic for a break-out session or a working group with dedicated resources that can improve things.
Axel _______________________________________________ Summit-pc mailing list -- summit-pc@sel4.systems To unsubscribe send an email to summit-pc-leave@sel4.systems
_______________________________________________ Summit-pc mailing list -- summit-pc@sel4.systems To unsubscribe send an email to summit-pc-leave@sel4.systems
_______________________________________________ Summit-pc mailing list -- summit-pc@sel4.systems To unsubscribe send an email to summit-pc-leave@sel4.systems
participants (5)
-
Gerwin Klein -
Heider, Axel -
Ihor Kuz -
June Andronick (Gmail) -
Robert VanVossen