ShapeWorks 6.1 Testing

[jadie1]

Please edit and add a ✅ indicating success and ❌ indicating failure or 🕒 for a test in progress with your username when you complete a task for a given platform.

When a test fails, please add a github issue, add it to the 6.1 release, and link it (*the issue when it's fixed and ready to test again). Also, go ahead and add new tasks that might not already be on here.

Please use the most recent release candidate for all testing (be careful which shapeworks is in your $PATH). The most recent is RC14:
https://github.com/SCIInstitute/ShapeWorks/releases/tag/v6.1.0-rc14

Example:

	Windows	Mac	Linux
Notebooks		🕒 (@archanasri - A)
Usecase: Ellipsoid	✅ (@akenmorris - A)
Usecase: All tiny-test	❌ (#1073)	✅ (@cchriste - A)

Systems chart

Username	System
(@HeavenlyBerserker - A)	Ubuntu 20.04.2 LTS (Linux) [Binaries]
(@HeavenlyBerserker - B)	Microsoft Windows 10 Education (Windows) [Release Branch]
(@iyerkrithika21 - A)	Microsoft Windows 10 Home OS Build 19042.1052 (Windows) [Binaries]
(@archanasri - A)	macOS Big Sur 11.4 (OSX) [Binaries]
(@cchriste - A)	maxOS Big Sur 11.3.1 (OSX)
(@cchriste - B)	openSUSE 15.1 (Linux)
(@cchriste - C)	(Windows)
(@akenmorris - A)	Windows 10
(@jadie1 - A)	openSUSE 15.0 (Linux)
(@username - B)	System (System Type)

Instructions for running and testing items

Testing modes:

1. Binary downloads

• Go to https://github.com/SCIInstitute/ShapeWorks, click releases, click the system you are testing, download and unzip the first asset.
• Set your paths to the release binaries instead of your regular paths. Make sure python scripts are running them.

2. Release/master branch

• Fetch, checkout master, and pull; Make sure it builds. If you're on your usual repository directory, you might need to delete the dependencies directory and rerun source build_dependencies.sh.

3. Clean installation

• Find a different machine or create a VM and download the binaries (or clone branches and build if you're feeling industrious).
• Run 1 or 2 items in each category.

General instructions

• Before testing anything, activate the shapeworks Anaconda environment via conda activate shapeworks. You might have to update it via source conda_installs.sh.
• Now follow the specific instructions for each item type below.

Notebooks

1. Go to ShapeWorks/Examples/Python/notebooks/tutorials on terminal/CMD.
2. Run the jupyter notebook command.
3. Open whichever file you wish to test.
4. Make sure it runs to completion and each input matches what the explanation says.

UseCases

1. Go to ShapeWorks/Examples/Python/ on terminal/CMD or on your favorite IDE that runs python.
2. Run the python RunUseCase.py --use_case <use_case> command, where <use_case> is replaced by the command given. For instance python RunUseCase.py --use_case femur, or python RunUseCase.py --use_case femur --groom_images.
3. Wait until it finishes. A Studio Analyze window should pop up. Make sure sampling, correspondence, and analyze->PCA->animate are reasonable.

Studio

1. Go to ShapeWorks/Examples/Python/<item> on terminal/CMD, where "" is the item you're testing.
2. Run the command ShapeWorksStudio <item>.xlsx.
3. Click on "Run Groom" and make sure it's reasonable. Click on the "Optimize" tab, and click "Run Optimize". Go to the "Analyze" tab and make sure sampling, correspondence, and analyze->PCA->animate are reasonable.

Studio: MeshGrooming

1. Download test_meshes.zip and unzip.
2. Run command ShapeWorksStudio, or open ShapeWorksStudio somehow.
3. "Start New Project" -> "+ to import shapes" -> Select all 4 downloaded shapes and open.
4. Play with the mesh groom parameters and the views. Adjust parameters and make sure it still makes sense; e.g. check Groom->Smooth and adjust laplacian, then "Run Groom"; e.g. move around, click Center, increase viewers, run groom with other parameters, etc.

Ok, now the real thing!

	Windows	Mac	Linux
Clean installation	❎ (@cchriste - C) both Anaconda and git-bash, default and user-selected install dirs, #1296 🕒(@HeavenlyBerserker - B dev instructions)	❎ (@cchriste - A) #1296) ✅ (@archanasri - A) (user instructions)	✅ (@jadie1 - A (user instructions - RC12))
Notebooks: getting-started-with-notebooks	✅ (@iyerkrithika21 - A)	✅ (@archanasri - A)	✅ (@HeavenlyBerserker - A) ✅ (@jadie1 - A (RC8))
Notebooks: getting-started-with-segmentations	✅ (@iyerkrithika21 - A)	✅ (@archanasri - A)	✅ (@HeavenlyBerserker - A) ❌(#1314)
Notebooks: getting-started-with-exploring-segmentations	✅ (@iyerkrithika21 - A)	✅ (@archanasri - A)	✅ (@HeavenlyBerserker - A) ❌(#1314)
Notebooks: getting-started-with-meshes	✅ (@iyerkrithika21 - A)	✅ (@archanasri - A)	✅ (@HeavenlyBerserker - A) ❌(#1314)
Notebooks: getting-started-with-data-augmentation	✅ (@iyerkrithika21 - A)	✅ (@archanasri - A) (with num_samples=5) ✅ (@cchriste - A)	✅ (@HeavenlyBerserker - A) ✅ (@jadie1 - A (RC8))
Notebooks: getting-started-with-shape-cohort-generation	✅ (@iyerkrithika21 - A)	✅ (@archanasri - A)	✅ (@HeavenlyBerserker - A) ✅ (@jadie1 - A (RC8))
Usecase: ellipsoid	✅ (@iyerkrithika21 - A)	✅ (@archanasri - A)	✅ (@HeavenlyBerserker - A) ✅ (@jadie1 - A (RC8))
Usecase: ellipsoid_cut	✅ (@iyerkrithika21 - A)	✅ (@archanasri - A)	✅ (@HeavenlyBerserker - A) ✅ (@jadie1 - A (RC8))
Usecase: ellipsoid_evaluate	✅ (@iyerkrithika21 - A)	✅ (@archanasri - A)	✅ (@HeavenlyBerserker - A) ✅ (@jadie1 - A (RC8))
Usecase: ellipsoid_pca	✅ (@iyerkrithika21 - A)	✅ (@archanasri - A)	✅ (@HeavenlyBerserker - A) ✅ (@jadie1 - A (RC8))
Usecase: ellipsoid_fd	✅ (@iyerkrithika21 - A)	✅ (@archanasri - A)	✅ (@HeavenlyBerserker - A) ✅ (@jadie1 - A (RC8))
Usecase: ellipsoid_mesh	✅ (@iyerkrithika21 - A)	✅ (@archanasri - A)	✅ (@HeavenlyBerserker - A) ✅ (@jadie1 - A (RC8))
Usecase: ellipsoid_multiple_domain	✅ (@iyerkrithika21 - A)	✅ (@archanasri - A)	✅ (@HeavenlyBerserker - A) ✅ (@jadie1 - A (RC8))
Usecase: ellipsoid_multiple_domain_mesh	✅ (@iyerkrithika21 - A)	✅ (@archanasri - A)	✅ (@HeavenlyBerserker - A)
Usecase: femur	✅ (@akenmorris - A)	✅ (@archanasri - A)	✅ (@HeavenlyBerserker - A)
Usecase: femur --groom_images	✅ (@akenmorris - A)	✅ (@archanasri - A)	✅ (@HeavenlyBerserker - A)
Usecase: femur_mesh	✅ (@iyerkrithika21 - A)	✅ (@archanasri - A)	✅ (@HeavenlyBerserker - A)
Usecase: femur_cut	✅ (@akenmorris - A)	✅ (@archanasri - A)	✅ (@HeavenlyBerserker - A )
Usecase: femur_cut --groom_images	✅ (@akenmorris - A)	✅ (@archanasri - A)	✅ (@HeavenlyBerserker - A)
Usecase: left_atrium	✅ (@iyerkrithika21 - A)	✅ (@archanasri - A)	✅ (@HeavenlyBerserker - A)
Usecase: left_atrium --groom_images	✅ (@cchriste - C)	✅ (@archanasri - A)	✅ (@HeavenlyBerserker - A )
Usecase: lumps	✅ (@iyerkrithika21 - A)	✅ (@archanasri - A)	✅ (@HeavenlyBerserker - A)
Usecase: thin_cavity_bean	✅ (@iyerkrithika21 - A)	✅ (@archanasri - A)	✅ (@HeavenlyBerserker - A)
Usecase: supershapes_1mode_contour	✅ (@iyerkrithika21 - A)	✅ (@archanasri - A)	✅ (@HeavenlyBerserker - A)
Usecase: deep_ssm*	✅ (@cchriste - C)	✅ (@archanasri - A)	✅ (@HeavenlyBerserker - A )
Studio: ellipsoid	✅ (@iyerkrithika21 - A)	✅ (@archanasri - A))	✅ (@HeavenlyBerserker - A )
Studio: FeatureMap	✅ (@iyerkrithika21 - A)	✅ (@archanasri - A)	✅ (@HeavenlyBerserker - A )
Studio: MeshGrooming	✅ (@iyerkrithika21 - A)	✅ (@archanasri - A)	✅ (@HeavenlyBerserker - A )
...

* Edit deep_ssm.py so that num_samples = 3 on line 76 and "epochs": 3, on line 132, otherwise it will take upwards of two days.

Please keep comments on this issue to a minimum. Let's try to keep the status in the table and not in the comments.

[HeavenlyBerserker]

@sheryjoe I think we should assign specific testing tasks to each person. Otherwise, we'll witness diffusion of responsibility in play here.

[sheryjoe]

@HeavenlyBerserker As we did in the 6.0 release (#1073), the table in this issue should help each one to "proactively" start testing depending on the system(s) he/she has access to.

[HeavenlyBerserker]

How do we tell correctness?

• I'm assuming that for Studio, we run grooming and optimize with default parameters and make sure the particle sampling, correspondence, and animate are reasonable?
• For use cases, we just make sure sampling, correspondence, and animate are reasonable, right?

Anything else we need to test for?

Also, we should include instructions for what to look for in tests in the future. Otherwise, I can only make sure it runs and the results are somewhat reasonable.

Edit: I added instructions to the first comment. Please let me know if we have to look for anything else.

[cchriste]

Correctness should be typically objective and measurable, especially wrt unit testing. But we currently have many tests deemed successful if they simply finish without crashing. Everyone: please file specific issues for things like this as you test.

…

On Mon, Jun 14, 2021 at 12:45 PM Hong Xu ***@***.***> wrote: How do we tell correctness? - I'm assuming that for Studio, we run grooming and optimize with default parameters and make sure the particle sampling, correspondence, and animate are reasonable? - For use cases, we just make sure sampling, correspondence, and animate are reasonable, right? Anything else we need to test for? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1273 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJT3ELIRGQ4HR6ST32BXPDTSZE4FANCNFSM46KWIFZQ> .

[jadie1]

@akenmorris RC12 worked for me as well

[akenmorris]

What's left?

[sheryjoe]

I added more docs on the getting-started/shape to cover good and bad surfaces and what is shape modeling.

[sheryjoe]

@akenmorris I added the tour video and the workshop videos (partitioned into three parts) to the docs. Nothing else for my side. Other than #1354, any thing else left for the release?

[akenmorris]

RC14 here:

https://github.com/SCIInstitute/ShapeWorks/releases/tag/v6.1.0-rc14

[sheryjoe]

Thanks all for your great job in getting the 6.1 release out! Happy to hit the close button for this issue.