Working with Python

Python is an extremely popular programming language, for both full applications and quick scripts. With the help of the Scipy stack, and various deep learning libraries, it is also becoming increasingly popular for scientific programming and image classification.

Scripting with Jython

The easiest way to use Python with QuPath is through Jython. This an implementation of Python for the Java Virtual Machine. What this means in practice is that it lacks support for extensions written for the version of Python written in C, i.e. 'CPython'. Unfortunately, these extensions include very useful things like Numpy and Scipy. You can't use these with QuPath via Jython.

Consequently, Jython is of somewhat limited use in terms of linking QuPath with the outside world. Still, it provides good scripting and file handling functionality for anyone already familiar with Python.

To get started with Jython, it may already be available under Automate → Show script editor, and then change the language to Jython under the Language → menu.

If Jython isn't available, you can install it simply by downloading Jython-standalone.jar from the Jython downloads page and drag it onto QuPath while it is running. The next time QuPath is restarted and the script editor is opened, the Jython menu option should appear.

QuPath and CPython

There are several options for connecting a Java application (such as QuPath) and a full CPython installation, to enable the best of both worlds.

At this point, none of these solutions have been fully developed for QuPath/CPython integration, but below there are some suggestions regarding how to get started. If anyone has a better solution, or has time to work on more complete integration, please do get in touch (through one of these methods) to share your method.

JEP - Java Embedded Python

JEP is currently the best way I know of to access CPython from QuPath. It is apparently not 100% reliable with all CPython extensions - but performance can be very good.

Conceivably, it might be used for applications such as cell segmentation using image processing operations coded in Python, although the wrappers etc. required to make this a straightforward process have not been written yet.

Setup

Setting up QuPath to work with JEP is not entirely straightforward, since it's necessary to ensure that the required libraries (both a native library and a Jar) are found by Java. These should all be available within your JEP installation.

The Jar will have a name something like jep-3.6.1.jar. It needs to be added to the Java classpath. There are a few ways this can be done:

• Add it to the classpath by modifying your QuPath.cfg file
• Launch QuPath from the command line, with the Jar included on the classpath (along with everything else QuPath requires)
• Copy the Jar to your QuPath extensions directory - this can be done by simply dragging it onto QuPath while it is running
• Create a symbolic link to the JEP directory within your QuPath extensions directory

The native library will have an extension that depends upon the platform being used, e.g. .dll (Windows), .jnilib (Mac), or .so (Linux). This needs to be added to the Java library path. Options for doing this include:

• Copy the library to the main QuPath installation, alongside where all the other required native libraries are to be found
• Add -Djava.library.path=\path\to\JEP\directory:.:.. or similar to the [JVMOptions] inside your QuPath.cfg file
• Launch QuPath from the command line, with the appropriate -Djava.library.path setting

Setup on Linux has some extra troubles because of the way Python needs to be loaded. See this post for a potential solution.

Test script #1: Computing the mean pixel value with Numpy

Assuming the paths have been set appropriately, the following Groovy script can then be used to get started. It takes a thumbnail from an open image, then calculates the mean value for each RGB channel - first using Java, then secondly using Python.

/**
 * Demonstration of using QuPath with JEP + Numpy.
 *
 * @author Pete Bankhead
 */

import jep.Jep
import jep.JepConfig
import jep.NDArray
import qupath.lib.analysis.stats.StatisticsHelper
import qupath.lib.gui.QuPathGUI
import qupath.lib.scripting.QPEx

import java.awt.image.BufferedImage


// Get an RGB thumbnail from the current image
def img = QPEx.getCurrentImageData().getServer().getBufferedThumbnail(200, -1, 0);

// Extract pixels for each channel as float arrays
int w = img.getWidth();
int h = img.getHeight();
float[] red = img.getData().getSamples(0, 0, w, h, 0, (float[])null);
float[] green = img.getData().getSamples(0, 0, w, h, 1, (float[])null);
float[] blue = img.getData().getSamples(0, 0, w, h, 2, (float[])null);

// Print mean values for each channel from Java
println("Mean red (from Java): " + StatisticsHelper.computeRunningStatistics(red).getMean());
println("Mean green (from Java): " + StatisticsHelper.computeRunningStatistics(green).getMean());
println("Mean blue (from Java): " + StatisticsHelper.computeRunningStatistics(blue).getMean());

// Create JEP instance
Jep jep = null;
try {
   jep = new Jep(new JepConfig()
            .setInteractive(false)
            .addSharedModule("numpy")
            .setClassLoader(QuPathGUI.getClassLoader()));
} catch (Throwable t) {
    // This is clumsy, but will show if there is library-locating trouble
    println(t)
    t.printStackTrace();
    return;
}
print("Started JEP: " + jep.toString())

// Send float arrays to JEP
def nd = new NDArray<>(red, h, w);
jep.set("r", nd);
nd = new NDArray<>(green, h, w);
jep.set("g", nd);
nd = new NDArray<>(blue, h, w);
jep.set("b", nd);

// Print mean values for each channel using JEP + Numpy
// Note!  This won't appear in QuPath's log window, since the printing goes to the 'wrong' place...
def script = """
print('Mean red value: ' + str(r.mean()))
print('Mean green value: ' + str(g.mean()))
print('Mean blue value: ' + str(b.mean()))
"""

// Move through script, line by line
// See https://github.com/mrj0/jep/issues/55
// Also, consider jep.runScript(String path) instead
for (s in script.readLines()) {
    // Skip empty lines (they might interfere with blocks?)
    if (s.trim().isEmpty())
        continue;
    // Evaluate the line
    jep.eval(s)
}

// Print the same results using a different approach
// Here, we request only the results from JEP, then print in a 'standard' Groovy way
println("Mean red from Python: " + jep.getValue("r.mean()"))
println("Mean green from Python: " + jep.getValue("g.mean()"))
println("Mean blue from Python: " + jep.getValue("b.mean()"))

// Clean up
jep.close();

Test script #2: Clustering with scikit-learn

One potentially very useful reason to linking QuPath with Python is the ability to start applying some of the many machine learning, clustering and feature selection possibilities within scikit-learn.

The following script shows the basic idea by packaging up the features for all QuPath detection objects in an image, and then sending them to scikit-learn for some k-means clustering.

/**
 * Code to illustrate the use of scikit-learn with QuPath, via JEP.
 *
 * This will apply k-means clustering to detection objects within QuPath based
 * on their feature measurements.
 *
 * @author Pete Bankhead
 */


import jep.*
import qupath.lib.classifiers.PathClassificationLabellingHelper
import qupath.lib.common.ColorTools
import qupath.lib.objects.classes.PathClassFactory
import qupath.lib.scripting.QPEx

//----------------------------------------------------------------------
// Parameters & code for Python

int nClusters = 5;

def pythonSharedModules = ["numpy", "sklearn", "sklearn.cluster"]

def pythonCode = String.format("""
from sklearn.cluster import KMeans
predictions = KMeans(n_clusters=%s, random_state=1).fit_predict(features)
""", nClusters)

def pythonResultVariable = "predictions"

def printCode = true

//----------------------------------------------------------------------

// Request all detections
def detections = QPEx.getDetectionObjects()

// Create an array of features
// Note: JEP requires that this must be a 1-D array in Java
def measurements = PathClassificationLabellingHelper.getAvailableFeatures(detections)
float[] features = new float[detections.size() * measurements.size()]
int i = 0
for (detection in detections) {
    for (m in measurements) {
        float val = (float) detection.getMeasurementList().getMeasurementValue(m)
        // Unfortunately, JEP doesn't seem to like NaNs or infinite values... for now, replace with zero
        // (it would probably be best to do this after mean-centering)
        if (!Float.isFinite(val))
            val = 0
        features[i] = val
        i++
    }
}

// Try the JEP-specific work in a try/catch block
Jep jep = null
def result = null
try {
    // Create JEP with shared modules
    def config = new JepConfig()
    for (module in pythonSharedModules)
        config.addSharedModule(module)
    jep = new Jep(config);

    // Send the required data
    NDArray<float[]> nd = new NDArray<>(features, detections.size(), measurements.size())
    jep.set("features", nd);

    // Run the code
    for (line in pythonCode.readLines()) {
        if (printCode)
            println(line)
        // Evaluate any non-empty line
        if (!line.trim().isEmpty())
            jep.eval(line)
    }

    // Request the result
    result = (NDArray)jep.getValue(pythonResultVariable)

} catch (Throwable t) {
    println(t)
    t.printStackTrace();
    return;
} finally {
    if (jep != null)
        jep.close()
}

// Shouldn't happen... but should probably check anyway
if (result == null) {
    print("No clusters found")
    return
}

// Set any clusters
for (int c = 1; c <= nClusters; c++) {
    def pathClass = PathClassFactory.getPathClass("Cluster " + c)
    pathClass.setColor(ColorTools.makeRGB(
            (int)(Math.random()*255),
            (int)(Math.random()*255),
            (int)(Math.random()*255)))
}

// Apply the clusters
int[] clusterCounts = new int[nClusters]
for (i = 0; i < detections.size(); i++) {
    int cluster = (int)result.data[i]
    clusterCounts[cluster]++
    def classification = PathClassFactory.getPathClass("Cluster " + cluster)
    detections.get(i).setPathClass(classification)
}

// Fire an update
QPEx.fireHierarchyUpdate()

// Print some info
for (int c = 0; c < nClusters; c++) {
    println("Number in cluster " + (c+1) + ": " + clusterCounts[c])
}

Py4J - A Bridge between Python and Java

As the Py4J website states,

Py4J enables Python programs running in a Python interpreter to dynamically access Java objects in a Java Virtual Machine. Methods are called as if the Java objects resided in the Python interpreter and Java collections can be accessed through standard Python collection methods. Py4J also enables Java programs to call back Python objects.

To get started, you will need to follow the Py4J download instructions. Then ensure the py4j-0.x.jar file can be seen by QuPath - if it isn't already there, drag this onto QuPath to install it as an extension.

Then you can run the following script within QuPath (once!) to start the GatewayServer used to communicate between Python and Java:

import py4j.GatewayServer;
import qupath.lib.gui.QuPathGUI;

QuPathGUI qupath = QuPathGUI.getInstance();
GatewayServer gatewayServer = new GatewayServer(qupath);
gatewayServer.start();
print('GatewayServer address: ' + gatewayServer.getAddress())

After doing this, you can go into Python and being interacting with QuPath as follows:

from py4j.java_gateway import JavaGateway
gateway = JavaGateway()
qupath = gateway.entry_point
image_data = qupath.getImageData()
path = image_data.getServer().getPath()
print('Current image path: ' + path)
n_objects = image_data.getHierarchy().nObjects()
print('Current hierarchy contains %d objects' %  n_objects)

This works nicely for me, except that I ran into major performance issues when trying to pass an image or anything reasonably large... making it rather limited in its usefulness.

I'd be happy if anyone has a better way, or can show something I did wrong.

JavaBridge

An interesting (but untested) alternative is JavaBridge.

This enables the excellent CellProfiler to run Java code from Python, although also offers the ability to call Python from Java. However I have not yet had any success using this with QuPath.