Feature request: Experiment's able to hint at their data file format

[jonathon-love]

hi,

i've been contemplating a jamovi integration with jatos (either through docker, or through https://cloud.jamovi.org), however it looks as though the results data format is up to the experiment (i.e. i use labjs a lot, and its results are jsonlines), meaning we'd need to rely on some sort of 'auto detection' if we wanted to allow people to open those data files.

a more robust solution would be for jatos to allow the experiment to record what sort of file format the experiment writes, and then jamovi could be provided that information ... it could even happen transparently, and could be based off the content-type of the POSTed data payload (although i appreciate there's a one-to-many relationship going on there ... the results are not a single file of a particular type, but rather many files concatenated together).

thoughts?

with thanks

[kristian-lange]

Hi,

You are correct, the result format is up the experiment, JATOS doesn't care for the format as long as it is text.

JATOS allows result data to be stored in the database or as files. The data stored in the database can overwrite eventually existing ones or append to them.

I can imagine to optionally store the content type for each result (e.g. as MIME types). But as you said: there are potentially many for each experiment, so it should be more of a list of content-types. Or, alternatively, we only store it once and expect / assume that all following data are stored in the same format (e.g. assume that once JSON, always JSON).

But then to specify the content type would be still voluntary and jatos.js would submit result data even without specifying it. Tools like lab.js will have to support it and old experiments won't have it anyway. I'm a bit unsure if it will be used.

But I have to admit I'm not a user of jamovi. And I'm just curious: why not just let the user import the data into jamovi and let them deal with the data format themselves (it their data and they know them best).

Best,
Kristian

[jonathon-love]

alternatively, we only store it once and expect / assume that all following data are stored in the same format (e.g. assume that once JSON, always JSON).

yeah, probably a reasonable assumption.

why not just let the user import the data into jamovi and let them deal with the data format themselves

i think both JATOS and jamovi exist to make things more straight forward for the user, and removing unnecessary steps is kinda what we both do, right?

(it their data and they know them best)

i think there's reasons why this isn't as true as it once was. as tools have become easier and easier to use, we've empowered a lot of less capable people to design their own experiments (labjs et al.) and deploy their own experiments (JATOS). ... at the same time, i think there's growth in the publishing and sharing of experiments, and people making use of other's work. a consequence is that people might "know their data", but they may not know "the format" of their data -- say the subtlety of the difference between json, and jsonlines, etc. etc.

surely, the dream for running an experiment is:

1. people browse for the experiment they want
2. customise it the way they want
3. deploy it (JATOS)
4. click a button, and bam, there's a summary of all the important measures from the experiment

i see opportunity here for streamlining the 3 -> 4 process

i'm thinking of it from jamovi's perspective, but there are countless other web based pipelines that this would enable too ... for close to zero work on your end.

Tools like lab.js will have to support it and old experiments won't have it anyway.

i don't think there'd be any reluctance on the side of @FelixHenninger to add it, it would only be a one-liner :)

thanks for engaging.

jonathon

[kristian-lange]

Hi jonathon,

i think both JATOS and jamovi exist to make things more straight forward for the user, and removing unnecessary steps is kinda what we both do, right?

Right

we've empowered a lot of less capable people to design their own experiments

Yes

at the same time, i think there's growth in the publishing and sharing of experiments, and people making use of other's work

I couldn't agree more.

a consequence is that people might "know their data", but they may not know "the format" of their data -- say the subtlety of the difference between json, and jsonlines, etc. etc.

I understand

surely, the dream for running an experiment is: 1) people browse for the experiment they want, 2) customise it the way they want, 3) deploy it (JATOS) 4) click a button, and bam, there's a summary of all the important measures from the experiment

Apart from the result data format, what other measures do you have in mind?

i see opportunity here for streamlining the 3 -> 4 process

Just an info: We are working on a public API for JATOS that can help integrating JATOS with other tools, like lab.js, e.g. deploy to a remote JATOS server by pressing a button. Or get the result data automatically into Matlab, R, Python.

i'm thinking of it from jamovi's perspective, but there are countless other web based pipelines that this would enable too ... for close to zero work on your end.

Here I can't follow you any more. Can you maybe give me some examples?

i don't think there'd be any reluctance on the side of FelixHenninger to add it, it would only be a one-liner :)
As long as it stays optional, I agree. From JATOS side it would be another field in the study or component properties.

Nice discussing with you :)

Best,
Kristian

[jonathon-love]

We are working on a public API for JATOS that can help integrating JATOS with other tools, like lab.js, e.g. deploy to a remote JATOS server by pressing a button. Or get the result data automatically into Matlab, R, Python.

yup, well this is basically what i have in mind ... and having "hints" about the format of the incoming data will be great.

but there's a bigger and more interesting question here, and that is how to join the software together in this pipeline ... i'm just thinking out loud here, and don't want to suggest that you haven't thought about it.

a simple approach is for people to be able to provide, say labjs, a JATOS url, and labjs exports directly to that service. similarly, to read the data out of JATOS, JATOS could provide a link with a token that the user could copy/paste into R, or python, or jamovi or whatever. that software downloads it, opens it.

another way (and this is the more compelling option imho) is for JATOS to assume a 'master role', and people can enable labjs, jamovi, etc. as plugins inside of JATOS. then a person can click '+ New Study', or '+New Study with labjs', and labjs opens within JATOS. similarly, on the results page, there's the option 'Option data with jamovi' (or something like it), and the data opens in jamovi within JATOS.

by default, JATOS (presumably) wouldn't ship with these plugins enabled, and it would be up to the user to enable them ... which sounds like a pain to configure ... but we'd just ship a helm chart, and people would tick off the features they want, and helm would bring it all on line. helm is a great way to bring an ensemble of services online and integrate them together.

anyhow, maybe you're thinking along these lines, and maybe you're not, but i think allowing for the tight integration of tools, with JATOS acting as the hub in the middle, is a really compelling user experience.

thoughts?

with thanks

[kristian-lange]

Sorry, for taken a couple days for answering, lot to do on my side.

a simple approach is for people to be able to provide, say labjs, a JATOS url, and labjs exports directly to that service. similarly, to read the data out of JATOS, JATOS could provide a link with a token that the user could copy/paste into R, or python, or jamovi or whatever. that software downloads it, opens it.

We thought along the same line, about tokens for authentication, similar to what GitHub is using, I think the term is API keys. Those tokens then can be used together with an API endpoint to e.g. get result data or upload a study.

another way (and this is the more compelling option imho) is for JATOS to assume a 'master role', and people can enable labjs, jamovi, etc. as plugins inside of JATOS. then a person can click '+ New Study', or '+New Study with labjs', and labjs opens within JATOS. similarly, on the results page, there's the option 'Option data with jamovi' (or something like it), and the data opens in jamovi within JATOS.

I try to wrap my head around this 'opens labjs or jamovi within JATOS'. What would a plugin say for jamovi actually do? I think I need more explanation.

by default, JATOS (presumably) wouldn't ship with these plugins enabled, and it would be up to the user to enable them ... which sounds like a pain to configure ... but we'd just ship a helm chart, and people would tick off the features they want, and helm would bring it all on line. helm is a great way to bring an ensemble of services online and integrate them together.

'helm chart', like this Kubernetes tool (I googled it, never used it myself). Wouldn't this mean that one has to install JATOS by using Kubernetes? I think this would limit JATOS to pretty professional setups and the small lab or single researcher just couldn't use it anymore. Or do I miss something?

Best,
Kristian

[jonathon-love]

What would a plugin say for jamovi actually do? I think I need more explanation.

let me see if i can whip up a proof of concept.

Wouldn't this mean that one has to install JATOS by using Kubernetes? I think this would limit JATOS to pretty professional setups and the small lab or single researcher just couldn't use it anymore.

so a plugin approach would mean that people could continue to use JATOS as they currently do ... however they have the option to enable plugins that may require additional configuration.

having said that, the eco-system emerging on top of kubernetes is really promising in terms of making deploying web services really easy. two prominent examples are portainer and k8s lens. here's a demo of k8s lens. i pick a "chart" from the list, select install, and then it presents me with the values.yaml ... which is the configuration (analogous to the config file that JATOS uses). then i can click 'install' and lens installs it. at any point i can return to the helm "release", tweak the values.yaml and the service will reconfigure itself. (further, docker desktop allows you to launch a kubernetes cluster locally ... so people can try out these things at home without needing to use a terminal).

https://youtu.be/J_grjEV3Pmw

however, uni's tend to be a few years behind the 8-ball ... so i concede -- in the short-term -- a helm chart will probably not see broad use.

however, even having said that, labjs is entirely (?) client side, and in the case of jamovi, there's the cloud version ... so there's routes to providing these things that don't involve anything more complicated than what it already being done by JATOS users. ... but let come back with my proof of concept!

cheers

jonathon

[jonathon-love]

OK, here's my very scant proof of concept (and maybe i'll cc @FelixHenninger in too, because i feel like he should be excited ... i certainly am):

Screen.Recording.2022-05-11.at.22.53.51.mov

[kristian-lange]

I like it. And it shouldn't be too much work from JATOS' side. How'd you open the lab.js window, in an iframe?

[jonathon-love]

And it shouldn't be too much work from JATOS' side

100%. way less work than stuffing around with authentication stuff ... and a way better ux.

How'd you open the lab.js window, in an iframe?

yup, iframe. although i couldn't get labjs to build, and ended up using chrome and the ignore-x-frame-headers addon and embedding https://labjs.felixhenninger.com/

so we'd have to either:

1. persuade felix to drop the x-frame headers ... shouldn't be a problem ... i can't think of a meaningful attack that could be performed with a sneaky embedded labjs instance.
2. ship labjs along with jatos ... in which case you'd probably want to do a plug-in thing.

but there's an API that needs to be spec'ed out here, that goes over the iframe boundary -- so labjs can push the changes back to jatos ...

... and of course we can do the same thing with the results files being able to be opened in an embedded jamovi ... in fact we can do one better ... the experiment can have a "jamovi template" embedded in it, and when opening the results from that experiment, jamovi automagically imports it into the template before presenting it to the user.

here's a vlog i did on labjs and jamovi templates a while back:

https://blog.jamovi.org/2019/03/27/import.html

this is my workflow at the moment, labjs => jatos => jamovi ... so this stuff is really scratching an itch for me at the moment :)

[jonathon-love]

hi,

some updates.

i've hit a snag with labjs: i'm not able to build it. i think the build process needs fixing by felix, but i haven't heard back (he's generally pretty responsive, so i assume he's stuck down a semester rabbit hole at the moment). (issue here: https://github.com/FelixHenninger/lab.js/issues/172). i've tried rolling back to an earlier version of labjs, but again, it errors. i did some work on labjs a couple of years ago, so i was clearly building it then ... so i guess i can roll back then if we get really desperate.

so i've had a bit of a nibble on what a jamovi integration would look like:

Screen.Recording.2022-05-16.at.11.22.02.mov

anyhow, let me know your thoughts -- don't let me spend too much time on this if you're thinking of going another direction! :P

to do these properly, jatos will invoke an iframe, and communicate via window.postMessage() ... and i'd suggest the next steps involve speccing out that API ... although i suppose there's some UI work to be done too (but that can be done in parallel).

cheers

jonathon

[kristian-lange]

Hi Jonathon,

Just to summarize (for me - I like summaries to clear my head)

• Integration with jamovi in JATOS' result pages
• Integration with experiment builder tools like lab.js in JATOS' in study page
• JATOS opens jamovi in iframe and communicate with it via window.postMessage

Just a thought: Maybe we can use a browser tab instead of an iframe. Somehow I personally consider a iframe as something less ... trustworthy, ... anoying ... - I don't know how to exactly put it. And a tab is a 'proper' window to the internet. It should be nearly the same coding-wise.

I looked into window.postMessage (never used it before).

1. JATOS does an window.open to open an iframe/tab with jamovi inside
2. jamovi uses window.postMessage to signal JATOS that it's ready to receive some data
3. JATOS uses window.postMessage to send the result data to jamovi

But I'm not sure if we can do without authentication / authorization using window.postMessage. Just read https://developer.mozilla.org/en-US/docs/Web/API/Window/postMessage#security_concerns. A JATOS instance can have multiple users. We have to prevent jamovi users from accessing result data from studies that are not theirs. As far as I understand any browser window object can use window.postMessage to talk to JATOS and e.g. ask for result data from JATOS. JATOS can verify sender's origin, that it is jamovi - but how does it verify that this jamovi instance is authorized to access this study's data? Maybe we need auth tokens too? Or do I miss something? I didn't use iframes that much so far. And if we have to use authentication too then there is no big difference to using the JATOS API and it would actually be less effort to have just one type of interface with external tools.

anyhow, let me know your thoughts -- don't let me spend too much time on this if you're thinking of going another direction! :P

I understand. I try to compare both approaches of interfacing JATOS with jamovi: 1) iframe/tab + window.postMessage, or 2) JATOS API. I just want to think this whole thing through before we start implementing and then discover half way it's not going to work :).

Btw. we can also use Slack to talk about the details: https://communityinviter.com/apps/jatosworkspace/jatos-slack

Best,
Kristian

[jonathon-love]

moved to #features-discussion in slack