sciview hip 2021 tutorial prep

[kephale]

I realized it makes sense to consolidate the prep conversation to some place a bit more permanent.

Tutorial link https://docs.scenery.graphics/sciview/workshops/hip-2021

[kephale]

My testing results are here.

Tests

fiji + sciview-buttercup

Follow fiji install instructions here https://docs.scenery.graphics/sciview/installation/installing-fiji
Follow update site instructions here https://docs.scenery.graphics/sciview/installation/installing-the-development-sciview-plugin-for-fiji
Download the data described here https://docs.scenery.graphics/sciview/workshops/hip-2021

1. Open Fiji
2. Run sciview
3. Test these demos:

• Demos>Basic>Lines3D
• Demos>Basic>Mesh (broken: sciview hip 2021 tutorial prep #405 (comment))
• Demos>Animation>Volume Timeseries
• Demos>Advanced>Show segmentation
• Demos>Advanced>Load Cremi dataset and neuron labels

intellij + demo command

Follow these instructions
https://docs.scenery.graphics/sciview/development/first-demo-command

Once the sciview window is open, also try running Volume Timeseries

pyimagej + sciview

Explanation here #405 (comment)

Systems

Windows 10:

• fiji + sciview-buttercup: works for everything except Mesh demo
• intellij + demo command: works for primitive objects, fails for volumes

Linux (Ubuntu 18.04):

• fiji + sciview-buttercup:
• intellij + demo command: works for primitive objects, might work for volumes but same GPU memory issue on @kephale's laptop
• pyimagej + sciview: seems to work (runs out of gpu memory on @kephale 's laptop at the moment)

Macos (catalina):

• fiji + sciview-buttercup: works for everything except Mesh demo
• intellij + demo command: fails, see sciview hip 2021 tutorial prep #405 (reply in thread)

[kephale]

Error from intellij + demo on macos:

2021-09-27 11:52:12.920 java[33799:2762219] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'NSWindow drag regions should only be invalidated on the Main Thread!'
*** First throw call stack:
(
	0   CoreFoundation                      0x00007fff30166437 __exceptionPreprocess + 250
	1   libobjc.A.dylib                     0x00007fff68e6f5bf objc_exception_throw + 48
	2   CoreFoundation                      0x00007fff3018ec1c -[NSException raise] + 9
	3   AppKit                              0x00007fff2d38bc1c -[NSWindow(NSWindow_Theme) _postWindowNeedsToResetDragMarginsUnlessPostingDisabled] + 310
	4   AppKit                              0x00007fff2d373682 -[NSWindow _initContent:styleMask:backing:defer:contentView:] + 1416
	5   AppKit                              0x00007fff2d3730f3 -[NSWindow initWithContentRect:styleMask:backing:defer:] + 42
	6   libnativewindow_macosx.jnilib       0x00000001baff93fe Java_jogamp_nativewindow_macosx_OSXUtil_CreateNSWindow0 + 398
	7   ???                                 0x000000011685ca10 0x0 + 4672834064
)
libc++abi.dylib: terminating with uncaught exception of type NSException

[kephale]

pyimagej + sciview

A pyimagej test script that we're working on to resolve linux issues:

# Run this inside the sciview environment

from ctypes import *
from ctypes.util import find_library
import imagej

# import ctypes
# x11 = ctypes.DLL("libX11.so")
# x11.XInitThreads()

libpath = find_library('X11')
if not libpath:
    raise ImportError('Could not find X11 library')
x11 = cdll.LoadLibrary(libpath)
del libpath

x11.XInitThreads()

ij = imagej.init('/home/kharrin/IJ/sciview-hip-2021/Fiji.app', headless=False)
ij.ui().showUI()

For an environment file like:

name: sciview
channels:
  - conda-forge
  - defaults
dependencies:
  - pyyaml
  - python=3.6
  - scyjava
  - openjdk=8.0.152
  - pyimagej

[frauzufall]

works for me with the prepackaged Linux Fiji https://www.dropbox.com/s/yjorzgsl39u03g2/Fiji.app-sciview-i2k.zip?dl=0

[kephale]

solved

MeshDemo seems to be broken

[kephale]

Something is going wrong with ioservice and opening meshes

[kephale]

Here is my workaround bd638b9

[kephale]

done

Mesh processing (https://docs.scenery.graphics/sciview/basics/mesh-processing) needs:

• a proper section on importing and exporting meshes
• a section on mesh smoothing

[frauzufall]

solved

This script (copied from https://docs.scenery.graphics/sciview/basics/volumetric-data) with an additional sv.addVolume fails with Caused by: java.lang.IllegalStateException: Unknown volume type class , adding it as described in the documentation via the Add Volume button fails silently / does nothing:

# @SciView sv
# @UIService ui

import net.imglib2.img.array.ArrayImgs as ArrayImgs

w = 100
h = w
d = w

img = ArrayImgs.unsignedBytes(w, h, d)
img_ra = img.randomAccess()

for x in range(0, w):
	for y in range(0, h):
		for z in range(0, d):
			img_ra.setPosition([x, y, z])

			x0 = x - 50
			y0 = y - 50
			z0 = z - 50
			val = 255 - (x0**2 + y0**2 + z0**2) ** 0.65

			img_ra.get().setReal(val)

ui.show(img)
sv.addVolume(img)

[frauzufall]

It works with the newest Fiji bundle @kephale made for Linux 🤷 🙌 https://www.dropbox.com/s/sjpuqty5w97pg9h/Fiji.app-hip-summer-school-2021.zip?dl=0

[frauzufall]

ISSUE

I added two volumes via the script editor and then tried to delete one from sciview via right click - the right click was not showing up for a while, I tried a couple of times, then it did but the console threw errors, I removed one volume and now the other one is only rendering the bounding box, nothing else, removed it again, added it again, still only the bounding box.

U8 texture does not exist, creating new one (tpietzsch.example2.VolumeTextureU8@21491136)
[ERROR] Call to allocate image staging memory of size 492011520 failed: A device memory allocation has failed.
[WARN] Could not load texture for VolumeManager: java.lang.RuntimeException: Call to allocate image staging memory of size 492011520 failed: A device memory allocation has failed.
VolumeManager has uninitialised UBO offset, skipping for rendering
[WARN] graphics.scenery.backends.vulkan.VulkanObjectState@5310df0e: Could not find descriptor set for geometry and texture set volumeCache
[WARN] DS are: geometry: transferFunction_x_118_x_, colorMap_x_119_x_, volume_x_117_x_, ObjectTextures, lights: ObjectTextures, colorMap_x_119_x_, volume_x_117_x_, transferFunction_x_118_x_
[ERROR] Internal consistency error for node VolumeManager: Descriptor set volumeCache not found in renderpass ForwardShading, skipping node for rendering.
[WARN] graphics.scenery.backends.vulkan.VulkanObjectState@5310df0e: Could not find descriptor set for geometry and texture set volumeCache
[WARN] DS are: geometry: transferFunction_x_118_x_, colorMap_x_119_x_, volume_x_117_x_, ObjectTextures, lights: ObjectTextures, colorMap_x_119_x_, volume_x_117_x_, transferFunction_x_118_x_
[ERROR] Internal consistency error for node VolumeManager: Descriptor set volumeCache not found in renderpass ForwardShading, skipping node for rendering.
[WARN] graphics.scenery.backends.vulkan.VulkanObjectState@5310df0e: Could not find descriptor set for geometry and texture set volumeCache
[WARN] DS are: geometry: transferFunction_x_118_x_, colorMap_x_119_x_, volume_x_117_x_, ObjectTextures, lights: ObjectTextures, colorMap_x_119_x_, volume_x_117_x_, transferFunction_x_118_x_
[ERROR] Internal consistency error for node VolumeManager: Descriptor set volumeCache not found in renderpass ForwardShading, skipping node for rendering.
[WARN] graphics.scenery.backends.vulkan.VulkanObjectState@5310df0e: Could not find descriptor set for geometry and texture set volumeCache
[WARN] DS are: geometry: transferFunction_x_118_x_, colorMap_x_119_x_, volume_x_117_x_, ObjectTextures, lights: ObjectTextures, colorMap_x_119_x_, volume_x_117_x_, transferFunction_x_118_x_
[ERROR] Internal consistency error for node VolumeManager: Descriptor set volumeCache not found in renderpass ForwardShading, skipping node for rendering.
[WARN] graphics.scenery.backends.vulkan.VulkanObjectState@5310df0e: Could not find descriptor set for geometry and texture set volumeCache
[WARN] DS are: geometry: transferFunction_x_118_x_, colorMap_x_119_x_, volume_x_117_x_, ObjectTextures, lights: ObjectTextures, colorMap_x_119_x_, volume_x_117_x_, transferFunction_x_118_x_
[ERROR] Internal consistency error for node VolumeManager: Descriptor set volumeCache not found in renderpass ForwardShading, skipping node for rendering.

[kephale]

Do they have the same name?

[kephale]

And this works for me... https://github.com/scenerygraphics/sciview/blob/bf8eee3e40216822d5df8e52ddc953034f598862/src/test/java/sc/iview/VolumeScriptTest.java I can delete one volume, then the other

[kephale]

Timing of workshop

~2hs total

Syllabus:

• 5-10min, Introductions (Kyle, Deborah, Giuseppe, Justin, and everyone attending)
• 5min, Getting your sciview and development environment setup
• 25min, Overview of the sciview project
• 5min, Justin's protein roller coaster
• 5min break
• 10min, Making your first scene
• ===== hr mark =====
• 15min, Working with volumetric data
• 15min, Working with meshes
• 30min, Creating sciview scripts
• ===== hr mark =====
• 15min, Making your first sciview demo command
• 15min, Custom point clouds
• 15min, 3D segmentation data

[kephale]

For bug in kapt on windows:
https://youtrack.jetbrains.com/issue/KT-40750

Nick Papirniy commented 5 Apr 2021 12:44

My solution was just adding those lines to gradle.properties

kapt.use.worker.api=false
kapt.incremental.apt=false

[kephale]

Then I get

LinkageError occurred while loading main class sc.iview.commands.demo.basic.PointCloudDemo