web interface / gui for ANTs? #1677
Comments
|
This is interesting. I am not a frequent user of ANTs but I deal with data
organization and preprocessing using pipelines and can share a couple of
thoughts from my perspective.
First, uploading data anywhere may be a matter of high sensitivity in the
era of GDPR. We typically run questionnaires to make sure the institution
who gets the data has the resources and security measures. Sometimes
companies (or even academia) will not share the data if the receiving side
is lax on security. It may not be as simple as putting data on an ANTs
website to process them. Data use agreements may be restrictive on what you
can do with the data.
Second, any management and manipulation of data will likely need proper
tracking of what was run, when, and what was produced. We use Datalad for
data versioning, which helps, but just to share a thought, your idea will
probably need a comprehensive backend system, not just a GUI layer.
Third, ANTs has many variants, e.g. binaries, python, R. Maybe there is an
ANTs BIDS app too, which in my opinion would be the way to go for any
additional gui variant. So, I am not sure what exactly would be the GUI
layer interfacing too. It certainly would need to be flexible in it's
function to handle both data and ANTs processes in the backend. It can
perhaps be done if combined with other existing open source solutions: e.g.
input data is in BIDS format, data is tracked with Datalad, ANTs
functions/options in gui are extracted from ANTs itself rather than being
hard coded in the gui, etc.
Again, maybe you were not looking for my response, but got excited from
your idea and wanted to share some thoughts.
…On Sun, Feb 11, 2024, 4:13 AM Nick Cullen ***@***.***> wrote:
*Is your feature request related to a problem?*
No
*Proposed new features*
It would be nice to be able to interact with ANTs via a user interface on
the web. This would allow users to more easily visualize and compare
results. It would also be nice to have a place to create deep learning
models using ANTs-specific pipelines. To be clear, the idea is a more
persistent application where results could be saved and shared, and not
like a toy Shiny app to demonstrate algorithms. A hosted ANTs could also be
a way to raise funds to support the project into the future.
*Alternatives you've considered*
The first main issue I see is how users would actually get their images
onto such a web app, since just uploading them seems possible but not
ideal. I wonder if there is already a common web host like flywheel which
has an API that images could be pulled from. The second main issue would be
high memory demands to run ANTs, although this probably just means some
kind of aws solution is needed rather than a small server.
*Additional context*
I was wondering @cookpa <https://github.com/cookpa> what the thoughts are
about this? I know this has been discussed in the past but I wonder if it
hasn't be tackled due to lack of time/interest on the devs part or
perceived lack of demand from the community. Would people use this if it
existed?
I have some time and some great web development experience where I think I
could take a shot at building a first MVP. I may try regardless but just
wanted to see what the thoughts were.
—
Reply to this email directly, view it on GitHub
<#1677>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACFJU7NMILBVEANGWT2RKLDYTCDSPAVCNFSM6AAAAABDDMMXWSVHI2DSMVQWIX3LMV43ASLTON2WKOZSGEZDQOJQGYZTAMY>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
|
Thanks, Dorian! That is great insight. The key value proposition would be exactly as you say -- not just running ANTs without writing any code, but being able to store and version results which persist across user sessions. I am pretty convinced that using ANTsPy would be the way to go, as I imagine supporting deep learning pipelines is something that would need to be done. The key really is how to interface with the data, so I appreciate hearing the way people store and version images. Like you said, users uploading images seems unlikely to be the main use-case. Getting read-only access to an S3 bucket where images are stored seems reasonable. For labs that have images stored on a university cluster, however, I'm not sure how that would work. Images would have to be downloaded from the user's image directory onto the server running ANTs, so maybe that's not super efficient for quick functions. In short, it would be nice to decide on what a bare-bones MVP would need to look like for people to say "OK, yes this is worth pursuing further." |
|
I would recommend starting with a simple visualization task. e.g.
just PoC stuff that doesn't depend on any particular back end or data organization. |
|
I will add more later (if/when paper is submitted) but there is some work like this being done by Faraz Faghri's group. |
|
I think a service that interfaces with an existing back end like Datalad or Flywheel would be the way to go. Managing a provenance database is a huge amount of work in itself. In terms of use cases, just based on community feedback I think the most interesting cases would be:
|
|
Thanks for the input, Phil. I am with you on all those points. |
|
@dorianps I was working a bit on datalad integration and was wondering if you could expand a bit on how you work with it. Do you host the datasets on github and then clone them onto a server or local computer when you run an analysis? How do you handle authentication if they're private? |
|
I use Datalad mainly in a local workstation, but Datalad can use Github,
Amazon S3, etc. E.g., you can pull templates easily from the templateflow
repository. I have also used Datalad with a ssh remote in the past, to pull
and push some files. In my case, it's all in a linux machine with ACL
control permission; users clone the dataset, make changes and ask me to
merge, just like with Git. I have not needed special authentication so far,
but I think some of the big datasets (UK biobank maybe?) provide access
tokens that can be input in Datalad to clone and get the data.
Happy to answer more questions, or even jump in a call if you prefer. The
Datalad online book and documentation was a good starter for me. I think
this tool is excellent for scientists given how much flexibility it has to
do tasks.
…On Tue, Feb 20, 2024 at 10:32 AM Nick Cullen ***@***.***> wrote:
@dorianps <https://github.com/dorianps> I was working a bit on datalad
integration and was wondering if you could expand a bit on how you work
with it. Do you host the datasets on github and then clone them onto a
server or local computer when you run an analysis? How do you handle
authentication if they're private?
—
Reply to this email directly, view it on GitHub
<#1677 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACFJU7O4VU5R77MTRJT5QS3YUS6XRAVCNFSM6AAAAABDDMMXWSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJUGQ3TENJRG4>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
Forgot to add something relevant to your use. Datalad has a system to run pipelines directly from datalad. The idea is to keep track of absolutely everything, the script you ran, the data it produced, and the command that was fired, etc. There is an example in the handbook: https://handbook.datalad.org/en/latest/basics/101-130-yodaproject.html However, I do not use this system because I need to run jobs in parallel using a job scheduling system. Datalad tended to struggle with this scenario because it tries to save dataset from multiple instances. Not sure if they fixed this behavior later. I just run pipelines separately and then save the outputs with datalad. |
|
Thanks, that's really helpful. I would love to talk later next week - I will reach out to set up a time. |
Is your feature request related to a problem?
No
Proposed new features
It would be nice to be able to interact with ANTs via a user interface on the web. This would allow users to more easily visualize and compare results. It would also be nice to have a place to create deep learning models using ANTs-specific pipelines. To be clear, the idea is a more persistent application where results could be saved and shared, and not like a toy Shiny app to demonstrate algorithms. A hosted ANTs could also be a way to raise funds to support the project into the future.
Alternatives you've considered
The first main issue I see is how users would actually get their images onto such a web app, since just uploading them seems possible but not ideal. I wonder if there is already a common web host like flywheel which has an API that images could be pulled from. The second main issue would be high memory demands to run ANTs, although this probably just means some kind of aws solution is needed rather than a small server.
Additional context
I was wondering @cookpa what the thoughts are about this? I know this has been discussed in the past but I wonder if it hasn't be tackled due to lack of time/interest on the devs part or perceived lack of demand from the community. Would people use this if it existed?
I have some time and some great web development experience where I think I could take a shot at building a first MVP. I may try regardless but just wanted to see what the thoughts were.
The text was updated successfully, but these errors were encountered: