Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Some docker fixes #3

Merged
merged 2 commits into from Aug 28, 2019

Conversation

scottwittenburg
Copy link
Contributor

@scottwittenburg scottwittenburg commented Jul 12, 2019

This still needs a few changes before we can merge it, but it's currently working for me. Some of the changes this PR makes include:

  • make the base container (bionic-python) truly Ubuntu 18.04 rather than Debian stretch
  • provides options to build the compute/visualize containers based on nvidia docker
  • creates a new EGL build of ParaView
  • provides docker-compose override file to allow selection of nvidia containers at runtime

Some things still left to do before merging:

  • Clean up commit history
  • Rebase on latest master (which brings changes for Girder 3 migration)
  • Minor cleanups of container stack
  • Update the compute-parflow Dockerfile to allow basing on nvidia stack

@scottwittenburg
Copy link
Contributor Author

@patrickoleary @aronhelser While this isn't 100% ready to go, it's getting pretty close except for the short TODO list I put in the PR description.

Feel free to take a look or try it out (obviously need an NVidia graphics card with latest drivers in order to try the gpu-accelerate path).

unzip \
wget \
--no-install-recommends && \
apt-get install -y --no-install-recommends \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was dropping build-essential intentional?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just moved it to the base image (bionic-python).

Makes the base image (bionic-python) a true Ubuntu 18.04 image, and
fixes the containers that base off it to work in that environment.

Also adds a path allowing creation of EGL images for the compute and
visualization parts of the workflow.
@scottwittenburg
Copy link
Contributor Author

ping @patrickoleary @aronhelser This is ready for you guys to take a look.

@aronhelser
Copy link
Member

Looks good to me! +1
The only things I noticed is some places doing the 'package cleanup' might have been removed - but I'm not sure if that step is even important?

@scottwittenburg
Copy link
Contributor Author

scottwittenburg commented Aug 12, 2019

Thanks for mentioning that, I meant to bring it up with you guys at some point. I never removed package cleanup, but rather any time it is needed (if apt-get update is run and pkgs installed) it all needs to be done within the same RUN command. Doing it at the end still results in the final image not having the cleaned up files, but they'll have to be downloaded with the layer that brought them regardless. You can read about that here, near the bottom of the RUN section.

@aronhelser
Copy link
Member

Were you going to merge this? I think we should start using it!

@scottwittenburg
Copy link
Contributor Author

I think we were holding off because @patrickoleary had a demo last week, but we could probably merge it now. I don't suppose you have tried the branch? I guess if we merge it, and then it turns out only to work for me, I can revisit in a follow-on topic.

@patrickoleary
Copy link
Member

patrickoleary commented Aug 27, 2019 via email

@aronhelser
Copy link
Member

I tried it out, and had a problem building some of the paraflow images - Scott is looking at it. I was able to build all the images used by docker-compose, and I did the complete workflow for pyfr through visualizer - so that part is good to go.

@patrickoleary
Copy link
Member

patrickoleary commented Aug 27, 2019 via email

@aronhelser
Copy link
Member

Not yet - nvidia doesn't work on windows. I'm going to try it out on my linux box next, and try it with the nvidia runtime.

@aronhelser
Copy link
Member

On Linux it works. I started with Docker 19.03, and had to follow these suggestions to enable the 'runtime' keyword in docker-compose. After that, I could run the pyfr workflow all the way through visualization, and the animation was much faster.

@patrickoleary
Copy link
Member

patrickoleary commented Aug 28, 2019 via email

@aronhelser
Copy link
Member

I'll add the readme in a separate commit....

@aronhelser aronhelser merged commit 3f57144 into Kitware:master Aug 28, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants