Identify "finished" notebooks and pin them to versions of HARK

[shaunagm]

Per discussions on issue #43 and PR #50, the only way to have multiple notebooks in a repository that use different versions of HARK is to add a cell to the top of the notebooks installing the desired version, which would look something like:

%%capture
! pip install econ-ark==0.10.whatever

(The %%capture is there to prevent the installation's progress from displaying within the notebook -- it suppresses all output.)

Once we do this, we can set the default in the repository to be the latest stable release. Any notebooks that are not "finished" and pinned to an older version will automatically be updated to the new release, and we can test them when that happens.

[llorracc]

That solution would be fine for notebooks that are being run on transient machines in the cloud. But it occurs to me that it is kind of rude for the notebook to install something, maybe something very large, on the user's machine without even asking permission.

Maybe a more robust solution: For "pinned" notebooks, there should be a test for the availability of the particular version needed; if it does not exist, then the user should be asked whether it is OK to install it. (If it does exist, then the pip install is not necessary, but does no harm (=takes little time). I take it that the reason to do it regardless is that the process of installing puts the correct version of python in the path?

[shaunagm]

I see what you're saying, but the user will have installed econ-ark already, this is just a different version.

The process of installing puts the correct version of Hark on the path.

[llorracc]

Well, they might not have installed it at all, right? They might just have downloaded a notebook without realizing they would need to install HARK to run it. When they learn they will need to install a software package on their computer they might decide that it's not worth it.

[shaunagm]

If they want to run our notebooks, they'll need to install HARK, there's no way around that. Currently this is done by installing things in the requirements.txt.

[llorracc]

My point is that if they download a notebook without having installed HARK (and therefore obviously haven't understood how the whole thing works), they might change their mind and decide they don't want to run the notebook after all because they don't want to install another package on their computer.

[shaunagm]

But they can't run the notebook anyway without installing HARK. So if they don't want to install another package there's nothing we can do that will end up with them running the notebook locally - we can only point them to hosted solutions.

Is your worry that you think it's harder to get consent for installing HARK when doing so from inside the notebook, whereas users are explicitly consenting when they run pip install requirements.txt from the command line? If not, can you explain again from the beginning, I feel like we're talking past each other here and I'm quite confused.

[llorracc]

I think our difficulty communicating on this is that you are assuming that people who come across our notebooks have much more knowledge than they do (or at least some of them do) about how the whole python/Jupyter/open source world works. Everything you say is right from the perspective of someone who knows the rudiments. But a lot of first year PhD students in economics, and even considerable numbers of faculty who are new to the area: 1. Have never heard of Jupyter notebooks 1. Have no idea what a requirements.txt file is 1. Would not know how to pip install requirements.txt 1. Have very little space left on the default storage of their computer * This last is extremely common among grad students Software that installs 500mb (or however much is required to satisfy our requirements.txt file for a machine with none of it -- like Windows machines) might be perceived more as malware and bloat than as useful additions to their computer's capabilities. Having, myself, regretted installation of a lot of stuff that I never used again, I have an aversion to setting things up in such a way that large blocks of code are installed on a person's computer without them really knowing what is going on. This is another reason to prefer could-based hosting ...

…

On Wed, Aug 7, 2019 at 1:17 PM Shauna ***@***.***> wrote: But they can't run the notebook anyway without installing HARK. So if they don't want to install another package there's nothing we can do that will end up with them running the notebook locally - we can only point them to hosted solutions. Is your worry that you think it's harder to get consent for installing HARK when doing so from inside the notebook, whereas users are explicitly consenting when they run pip install requirements.txt from the command line? If not, can you explain again from the beginning, I feel like we're talking past each other here and I'm quite confused. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#51?email_source=notifications&email_token=AAKCK7ZO37R5QCOOXQ6FXGDQDL7ULA5CNFSM4IGJG36KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3ZDODQ#issuecomment-519190286>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAKCK7YY5BIZTU53UUJKJCTQDL7ULANCNFSM4IGJG36A> .

-- - Chris Carroll

[shaunagm]

I can't find evidence that Jupyter notebooks allow popups but I'll look harder tomorrow. If you're okay with cluttering the top of the notebooks a little bit, an alternative would be:

# hark installation code 
if HARK is not installed:
  raiseError("HARK needs to be installed to run this notebook.  To install HARK, please remove the hash symbol from the following line and re-run the notebook.  If you do not want to install HARK on your computer, try running this notebook on the cloud at the following URL: ")

Raising the error will cause the contents of the error to show up when you run the notebook. This will be annoying for some people but you can't customize for every user.

This also makes it harder to do the pinned version override. Up to you what you care about more and which experiences you're trying to optimize.

[llorracc]

Jupyter notebooks can definitely be interactive and your solution is exactly the kind of thing I had in mind.

And you are right that the key question is balancing simplicity versus other requirements.

Unfortunately, we already need a fair amount of preamble/setup stuff to make sure that our .ipynb Jupyter notebooks generate flawlessly functioning *.py files that can be executed via a ipython [x].py command. So adding one more item does not seem too awful.

[MridulS]

This issue is resolved with having pinned versioned released of the DemARK with HARK releases, further discussion in #108

@shaunagm @llorracc we should close this issue.