Periodic Technical report - Part B - RP3: Sept 2018-August 2019

[nthiery]

Like for the previous reporting period, we need to write a technical report for RP3. Its content
will look like:

1. Explanation of the work carried out by the beneficiaries and Overview of the progress
   1.1. Explanation of work carried out per Objective
   1.2. Explanation of the work carried per Work Package
   1.3. Impact
   1.4. Infrastructures
2. Update of the plan for exploitation and dissemination of result (if applicable)
3. Update of the Data Management Plan
4. Follow-up of recommendations and Quality Management
   4.1. Follow-up of recommendations
   4.2. Risk management
   4.3. Quality assurance plan
5. Deviations from Annex 1
   5.1. Tasks
   5.2. Use of resources
6. Key Performance Indicators (KPIs) broken up by AIM

I initiated a stub report by copy pasting that of last year. Here are the sources.

Work Package leaders: I will need you for:

• 1.2. Explanation of the work carried per Work Package
  A few pages per WP, with a couple paragraphs for the overview, for each relevant task and for each relevant milestone.
• 1.1 Explanation of work carried out per Objective
  1 to 2 pages essentially explaining which task contributed to which objective, and how
• 6 Key Performance Indicator
  For each aim, we have a couple KPI's. For each KPI we should briefly explain how the indicator evolved over the project and reflect about it. This will be mostly about updating the text from last year; we had made progress on KPI's by the time of the review, so it's worth having a look at the presentations we made.
• 3 Data Management Plan: @kohlhase: you may want to do some updates there.

[kohlhase]

I have written the bulk of text for WP6 in memory-dump mode. @florian-rabe should have a careful read of this.
We still need text on memoization from @mtorpey and some more on the Data.MathHub.info from @kohlhase

[kohlhase]

@nthiery I am not sure what to do about the data managment plan. If I understand correctly, then there will not be a data management plan V3 correct?

[kohlhase]

Ah, I can answer this to myself: #23
I will add some things there and describe the changes here.

[florian-rabe]

I have written the bulk of text for WP6 in memory-dump mode. @florian-rabe should have a careful read of this.

Done

[jdemeyer]

I updated the section about T4.4 (Sphinx) with a mention of PEP 590.

[kohlhase]

Ah, I can answer this to myself: #23
I will add some things there and describe the changes here.

@nthiery I have to correct myself, that is the "innovation management plan", it does not address the data management. Where should I deal with the data management issue?

[nthiery]

@nthiery I have to correct myself, that is the "innovation management plan", it does not address the data management. Where should I deal with the data management issue?

The two potential locations I see are: - As you expected, in your dataset section of the innovation management plan - Or in Section 3 of the periodic technical report #290 Whichever makes more sense to you. In anycase, a crossref from the other side is presumably in order. Thanks!

[minrk]

I've updated the WP4 report

• @jdemeyer can you update the pari task in the WP4 report? "No work to report in this period" is fine, if true
• @kohlhase can you update the mathhub and structdocs tasks with your work? Same as above, if there is no work to report since the deliverables.
• @fangohr same for the oommf task in WP4

[fangohr]

Hi Min,

there were no scheduled tasks for WP4 for us in the last year. Things we did included:

• Further integration of Ubermag functionalities into Jupyter notebook.
• 3D plotting of vector and scalar field in Jupyter notebooks using k3d
• Improvements of plotting 2D slices of discretised fields using matplotlib
• Setting up and maintenance of Micromagnetic VRE via Binder

This is reported under WP2 and deliverable D2.13.

[nthiery]

Hi @dimpase
Thanks for your offer to help! I could use it for the Technical Report :-) Indeed, except for setting up the stage in the repository, I did not yet get to go through all the sections of the current text, explicitly mark outdated ones, and add TODO's named after the usual suspects. Would you have a go at it? Possibly ping the usual suspects too as needed (keeping me in the loop to avoid double-pinging).

[dimpase]

@nthiery - could you point me to the correct document(s) to proofread?

[nthiery]

See the "sources" link in the issue description above. Feel free to edit that description to make it prominent. If you missed it, then it was too hidden and others will :-)

[dimpase]

I was confused by the naming: OpenDreamKit/ReportingPeriod3/TechnicalReport/
wrongly named period?

[nthiery]

I was confused by the naming: `OpenDreamKit/ReportingPeriod3/TechnicalReport/` wrongly named period?

No, that's right. RP1 was Sept 2015--Feb 2017, RP2 was March 2017-Aug 2018, RP3 is Sept 2018 -- Aug 2019. Cheers,

[dimpase]

isn't it more than just proofreading? e.g. the WP1 report has to be changed a lot.

[dimpase]

regarding WP1:

• what are the correct amendment numbers to mention?

• what are sites created/removed? (IMHO Leeds was gone, at least de facto, I am not sure whether this was official).

• UPS has hired a female RSE, after all.

[nthiery]

Yes, it is definitely more than proofreading. See
#290 (comment) . Checking what needs updating and pinging people.

[defeo]

I edited the section on WP3. I need input from Logilab on Simulagora, @ocayrol ?

And there's also an item about Sage and CLANG on which I don't know much. Do you know, @dimpase, maybe?

[defeo]

Regarding KPIs, any news from webservices.opendreamkit.org, @nthiery ?

[dimpase]

And there's also an item about Sage and CLANG on which I don't know much. Do you know, @dimpase, maybe?

I presume it's work around https://trac.sagemath.org/ticket/12426 that has been done, allowing Sage to build with clang (in particular on OSX).

Could you point out at that item more precisely? (As usual, I am a bit lost :-))

[defeo]

From the proposal (WP3):

Other urgent tasks include porting \Sage to \Python 3,
and porting \Sage to the primary OSX C/C++ compiler \clang.
The latter is important for porting of \Sage to Conda, see
\taskref{component-architecture}{mod-packaging}.

[...]

\item Porting \Sage to \clang; as the MacOS vendor (Apple) adds more and more
Objective C-style code to the system headers, more and more \Sage components need to be
built with \clang rather than with \gcc. A full port of \Sage to \clang is thus very
desirable, and a considerable amount of work in this direction has been done.
\item Porting \Sage to more \Fortran compilers gets urgent, especially as new
high-performance \Fortran compilers become available.

Those lines were written in September 2017, according to git. #12426 was completed a couple of months later.

[dimpase]

unless it has been already reported, that clang-related thing out to be mentioned. (there are more tickets related to it, certainly)

…

On Tue, 17 Sep 2019 13:53 Luca De Feo, ***@***.***> wrote: From the proposal (WP3): Other urgent tasks include porting \Sage to \Python 3, and porting \Sage to the primary OSX C/C++ compiler \clang. The latter is important for porting of \Sage to Conda, see \taskref{component-architecture}{mod-packaging}. [...] \item Porting \Sage to \clang; as the MacOS vendor (Apple) adds more and more Objective C-style code to the system headers, more and more \Sage components need to be built with \clang rather than with \gcc. A full port of \Sage to \clang is thus very desirable, and a considerable amount of work in this direction has been done. \item Porting \Sage to more \Fortran compilers gets urgent, especially as new high-performance \Fortran compilers become available. Those lines were written in September 2017, according to git. #12426 was completed a couple of months later. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#290?email_source=notifications&email_token=AAJXYHA2IYINZU7HOQL464DQKDHMLA5CNFSM4IQCHWB2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD64NL2I#issuecomment-532207081>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAJXYHBRFQJXD7EW2UPIB2DQKDHMLANCNFSM4IQCHWBQ> .

[nthiery]

Regarding KPIs, any news from webservices.opendreamkit.org, @nthiery ?

@defeo The technical support checked the VM, and apparently the disk is corrupted, preventing the kernel to boot. They can attempt to rescue the data. Do you know from the top of your head where in the file system the relevant data is? Just to save them a bit of time.

[nthiery]

I updated the "follow up to recommendation sections", and added the following ednotes:

• \ednote{ @kohlhase : MitM use case and comic
• \ednote{ @kohlhase: disseminating adoption of Logipedia}
• \ednote{ @defeo , @VivianePons , ...: develop our use case section and report here}
• \ednote{ @ClementPernet : hardware architecture recommendations}

[kohlhase]

Dear Nicolas, I am currently working my way through the ednotes. In particular the one about the comic, where I tried to integrate it into the use case. But I cannot figure out how to set the image on the post. Doing it by analogy does not seem to work, giving the full path neither. Can you help? See OpenDreamKit/OpenDreamKit.github.io@78528b4...a64f57e Michael

[kohlhase]

Dear Nicolas thanks for putting up all these ednotes, that is very helpful. I have acted on all (but two, which concern LogiPedia) the ednotes addressed to me. These I have punted to @frabe. FLORIAN, could you take care of these? Michael

[florian-rabe]

I have acted on all (but two, which concern LogiPedia) the ednotes addressed to me. These I have punted to @frabe. FLORIAN, could you take care of these?

I took care of the ednotes.

[nthiery]

Thanks everyone for the progress! Been teaching all day. I'll be back to this tomorrow afternoon.

[nthiery]

Thanks @kohlhase and @florian-rabe for the work on the use case!

... how to set the image on the post.

Fixed!

Currently the use case title is about the integration of LMFDB/Pari/Sage, when the comic is about integrating Singular/Sage/GAP. What about generalizing the use case to something like "Composing mathematical computational software and databases: MitM to the rescue", with a brief general introduction, the requirements for both examples, and a description of the solution using ODK. And a brief description of ODK contribution.
Ideally, this would be actionable, with demos / instructions / recommendations (or links to them) for the user to play with.

[kohlhase]

good point, I have done that --> much better post. Michael

On 11.10.19 08:34, Nicolas M. Thiéry wrote: Thanks @kohlhase <https://github.com/kohlhase> and @florian-rabe <https://github.com/florian-rabe> for the work on the use case! ... how to set the image on the post. Fixed! Currently the use case title is about the integration of LMFDB/Pari/Sage, when the comic is about integrating Singular/Sage/GAP. What about generalizing the use case to something like "Composing mathematical computational software and databases: MitM to the rescue", with a brief general introduction, the requirements for both examples, and a description of the solution using ODK. And a brief description of ODK contribution. Ideally, this would be actionable, with demos / instructions / recommendations (or links to them) for the user to play with. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#290?email_source=notifications&email_token=AA6ZXIU6JILONBIM64IJOSLQOAM6TA5CNFSM4IQCHWB2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEA67L7I#issuecomment-540931581>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA6ZXIRPZXJUS7KYGB3HFM3QOAM6TANCNFSM4IQCHWBQ>.

-- ---------------------------------------------------------------------- Prof. Dr. Michael Kohlhase, http://kwarc.info/kohlhase, skype: mibein42 Professur für Wissensrepräsentation & -verarbeitung Informatik, FAU Erlangen Nürnberg, Martensstr. 3, D-91058 Erlangen, Room 11.139, tel/fax: (49) 9131-85-64052/55, michael.kohlhase@fau.de ----------------------------------------------------------------------

[nthiery]

good point, I have done that --> much better post.

Thanks! Btw: is the last paragraph up to date or was progress made in the mean time?

[ocayrol]

Hi all, During this last period, we developed a JupyterLab extension for our Python trainings. This extension provides an environment where learners can autonomously do their exercises. We are currently testing it in our trainings and are quite satisfied with this product. See https://gitlab.com/logilab/jupyterlab-training for the extension and https://gitlab.com/logilab/jupyterlab-training-workspace for the exercises. I feel this work can be mentioned somewhere inside the WP4 report as it is a new tool in the Jupyter ecosystem and it is dedicated to the training/learning process. I am not sure where to insert it in the document... inside the struct docs task? Olivier.

[nthiery]

During this last period, we developed a JupyterLab extension for our Python trainings. This extension provides an environment where learners can autonomously do their exercises. We are currently testing it in our trainings and are quite satisfied with this product. See https://gitlab.com/logilab/jupyterlab-training for the extension and https://gitlab.com/logilab/jupyterlab-training-workspace for the exercises.

Excellent!

I feel this work can be mentioned somewhere inside the WP4 report as it is a new tool in the Jupyter ecosystem and it is dedicated to the training/learning process. I am not sure where to insert it in the document... inside the struct docs task?

Yeah, there is no ideal location; struct docs is a reasonable one.

[defeo]

FYI, I restored the analytics server for opendreamkit.org here: https://keats.prism.uvsq.fr/odk-websrvc/. Login and password are unchanged.

Last data point is from July 8, presumably the server went down after that. I did not restore live analytics on opendreamkit.org, I don't think it would be relevant to re-start collecting them from now.

[nthiery]

@defeo, @minrk, @ClementPernet, @alex-konovalov, @embray, @VivianePons

You still have ednotes to resolve (me and @IzabelaFaguet too), and we need to submit the Technical Report very soon. Like a couple days ago. Pleaaaaasee!

[stevelinton]

Alex is away, getting back tomorrow. Is it anything I can deal with? Steve

…

On 15 Oct 2019, at 17:35, Nicolas M. Thiéry ***@***.***> wrote: @defeo, @minrk, @ClementPernet, @alex-konovalov, @embray, @VivianePons You still have ednotes to resolve (me and @IzabelaFaguet too), and we need to submit the Technical Report very soon. Like a couple days ago. Pleaaaaasee! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe. [ { ***@***.***": "http://schema.org", ***@***.***": "EmailMessage", "potentialAction": { ***@***.***": "ViewAction", "target": "#290?email_source=notifications\u0026email_token=ABQQIRT4QQYPEEFLQ5YT2STQOXWNXA5CNFSM4IQCHWB2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBJNQ4Q#issuecomment-542300274", "url": "#290?email_source=notifications\u0026email_token=ABQQIRT4QQYPEEFLQ5YT2STQOXWNXA5CNFSM4IQCHWB2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBJNQ4Q#issuecomment-542300274", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { ***@***.***": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

[nthiery]

@stevelinton : yes, presumably. See the two ednotes mentionning Alex above about KPI's and WP5.

Thanks!

[VivianePons]

I'm on it today. Le mar. 15 oct. 2019 à 18:35, Nicolas M. Thiéry <notifications@github.com> a écrit :

…

@defeo <https://github.com/defeo>, @minrk <https://github.com/minrk>, @ClementPernet <https://github.com/ClementPernet>, @alex-konovalov <https://github.com/alex-konovalov>, @embray <https://github.com/embray>, @VivianePons <https://github.com/VivianePons> You still have ednotes to resolve (me and @IzabelaFaguet <https://github.com/IzabelaFaguet> too), and we need to submit the Technical Report very soon. Like a couple days ago. Pleaaaaasee! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#290?email_source=notifications&email_token=AA3QNYEIGUVJJT3XQHJK6YDQOXWNVA5CNFSM4IQCHWB2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBJNQ4Q#issuecomment-542300274>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA3QNYCEAH6FUDTZRAJ46ZLQOXWNVANCNFSM4IQCHWBQ> .

[ClementPernet]

I've addressed my ed notes except the last one (recommendation 10): I should write a blog post about it, but won't have time for neither today nor in the next coming days. Shall I create draft blog post and point to it and fill it up later on?

[nthiery]

I've addressed my ed notes

Thank you Clément!

except the last one (recommendation 10): I should write a blog post about it, but won't have time for neither today nor in the next coming days. Shall I create draft blog post and point to it and fill it up later on?

Yes, I think it's fair enough to state that a guidelines document is *in preparation*, and provide the link. This could actually would make a good new text for our "use case" section; something like "use case: exploiting modern architectures", starting with a few scenarios, providing guidelines on how to achieve them, and concluding with a brief description of ODK's contribution, with links to the related reports.

[nthiery]

@florian-rabe, @kohlhase, @fangohr, @VivianePons, @defeo, @minrk, @ClementPernet: the Publishable Summary is taking shape. I left TODO's for you. Generally speaking, the press release will be rebased on it, but you may want to double check the latter for elements of language.

[florian-rabe]

I've made some minor changes. @kohlhase should also have a look.
If you want more expansion, how many words should we roughly use?

[nthiery]

I've made some minor changes. @kohlhase should also have a look.

Thank you Florian!

If you want more expansion, how many words should we roughly use?

Generally speaking, the current text should be about the right size (a few pages). That being said, feel free to expand here and there the description of the achievements with a few sentences if this brings much additional value. Current status: except for a missing conclusion and paragraphs about EOSC that need work, we have a complete draft. Please proofread, comment, polish!

[nthiery]

Alright, time to go to bed. Tomorrow will be fully packed with teaching.

There are 34 ednotes left. @minrk, @defeo, @embray, @IzabelaFaguet, @VivianePons, @fangohr, @mikecroucher, @videlec, @jdemeyer: I count on you. It would be really good to submit on Friday.

Also, I would really appreciate too if one or two persons could browse through the document, and check that it stands as a whole and no piece was accidentally left alone. @stevelinton ? @dimpase ?

[stevelinton]

I will look over it tomorrow. Steve

…

On 16 Oct 2019, at 22:14, Nicolas M. Thiéry ***@***.***> wrote: Alright, time to go to bed. Tomorrow will be fully packed with teaching. There are 34 ednotes left. @minrk, @defeo, @embray, @IzabelaFaguet, @VivianePons, @fangohr, @mikecroucher, @videlec, @jdemeyer: I count on you. It would be really good to submit on Friday. Also, I would really appreciate too if one or two persons could browse through the document, and check that it stands as a whole and no piece was accidentally left alone. @stevelinton ? @dimpase ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe. [ { ***@***.***": "http://schema.org", ***@***.***": "EmailMessage", "potentialAction": { ***@***.***": "ViewAction", "target": "#290?email_source=notifications\u0026email_token=ABQQIRR3GQPDTWNTY4YZUETQO574BA5CNFSM4IQCHWB2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBN6TXI#issuecomment-542894557", "url": "#290?email_source=notifications\u0026email_token=ABQQIRR3GQPDTWNTY4YZUETQO574BA5CNFSM4IQCHWB2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBN6TXI#issuecomment-542894557", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { ***@***.***": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

[defeo]

* [ ]   \ednote{@defeo, @embray: update Windows installer download stats}

@embray, I got these page hit stats for http://www.sagemath.org/download.html from the Google analytics dashboard

Do you have anything more precise?

[nthiery]

Still about 32 ednotes on the Technical Report ...

I worked on the Publishable Summary tonight. It's now tentatively complete, including the conclusion. It still has ednotes for @defeo, @ClementPernet, @minrk, @fangohr, and would need general proofread and polishing by any volunteer.

Have a good night.

[stevelinton]

I read through the technical report. No major problems. There are quite a lot of typos and bits of slightly awkward English, but very little that affects readability. I noticed some sections that still need updating, but there seem to be ednotes covering them. Everything else seems to tell a consistent and comprehensible story. I’ll commit changes to a few of the more obvious and uncotroversial English things. One thing that’s probably work fixing because it’s on the front cover is capitalisation in the section titles. Some have all important words capitalised, some have none, some are even more random. Steve

…

On 17 Oct 2019, at 22:59, Nicolas M. Thiéry ***@***.***> wrote: Still about 32 ednotes on the Technical Report ... I worked on the Publishable Summary tonight. It's now tentatively complete, including the conclusion. It still has ednotes for @defeo, @ClementPernet, @minrk, @fangohr, and would need general proofread and polishing by any volunteer. Have a good night. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe. [ { ***@***.***": "http://schema.org", ***@***.***": "EmailMessage", "potentialAction": { ***@***.***": "ViewAction", "target": "#290?email_source=notifications\u0026email_token=ABQQIRULGJGSRCBT7KXEBILQPDN5NA5CNFSM4IQCHWB2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBRU7SQ#issuecomment-543379402", "url": "#290?email_source=notifications\u0026email_token=ABQQIRULGJGSRCBT7KXEBILQPDN5NA5CNFSM4IQCHWB2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBRU7SQ#issuecomment-543379402", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { ***@***.***": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

[defeo]

I updated the "follow up to recommendation sections", and added the following ednotes:
[...]
* [ ] \ednote{ @defeo , @VivianePons , ...: develop our use case section and report here}

I'm not sure what I'm supposed to do here. I was not involved much in the "Use Case" page.

All other ednotes done from my side, except for download stats on the Windows installer, on which I think @embray has better numbers.

[nthiery]

Really appreciate this @stevelinton! It's much comforting to know someone else checked the big picture.

[nthiery]

@defeo: thanks; I took care of the use-case ednote (rather lamely though). That's kind of the issue: few are involved in our use-case page. I'll run a sprint on this topic next week.

There remains your ednote in the Publishable Summary.

[embray]

@nthiery @defeo I updated the Windows download statistics. I can provide (slightly) more detailed stats if need-be but I don't think it's necessary to into more details since it is very rough anyways.