First execution works but the next ones don't do anything #258

MLobatoR · 2023-06-01T18:53:11Z

MLobatoR
Jun 1, 2023

Hi,

I'm struggling with how to configure TaskGraph for this to work properly. In the first execution everything works fine, the problem is with the next ones. I already tries to replace parameters before executing but it did not fix the problem.

TaskGraph morpheus = new TaskGraph("s")
				.lockObjectsInMemory(graph.Va, graph.Ea) 
				.transferToDevice(DataTransferMode.FIRST_EXECUTION, graph.Va, graph.Ea, results, updatedMark)
				.transferToDevice(DataTransferMode.EVERY_EXECUTION, costsToConsiderPerEdge, mark)
				.task("t", NeoAlgorithmLinesGPU::enterTheMatrix, graph.Va, graph.Ea, costsToConsiderPerEdge, results, mark, updatedMark)
				.transferToHost(results, updatedMark);

		morpheus.execute();

The parameter results and updatedMark will be modified during the execution. When I inspect this two parameters they are correct, the values are as expect after "morpheus.execute();". Then I update mark, and replace the parameters end execute again, but nothing happens. Can you hepl me please? Could it be because I have this?

for(@Parallel int i = 0; i < Va.getLength(); i++) {
			for(int j = 0; j < Va.getLength(); j++) {
				//expand
				if(mark.get(i, j) == 1) {
					Kernel.hackLineFromMatrix(Va, Ea, results.get(i, j), results, costPerEdge, j, i, updatedMark);
				}
			}
		}

The "if" is messing this?

Thank you for your help with this,
ML

Answered by jjfumero

Jun 24, 2023

The replaceParameters method within the TaskSchedule was removed in version 0.15. We do not need this call because we can use multiple task-graphs and move data across them. I think you should create multiple task-graphs (as many as you need) and multiple execution plans. And move data around as we point out in the examples.

If this approach does not fit your needs, feel free to create a simple test-case, and send us a proposal with the feature/s you would like to add in TornadoVM. It is important to understand why your use case could not use multiple execution plans instead, and how we could approach your patterns/examples. The way to open proposals is via GitHub discussions.

View full answer

stratika · 2023-06-02T10:04:53Z

stratika
Jun 2, 2023
Maintainer

Hi @MLobatoR, the use of the morpheus.execute() is wrong. Since TornadoVM v0.15, and the refactoring of the API to TaskGraph, a Java programmer should not invoke the execute method directly. You can find more information in the TornadoVM blog for the migration to the new API. So, your version of code is not fully compliant with the way that the programming model is designed. Can you try the following to see if the same problem occurs?

In a nutshell, the TaskGraph should be converted to an ImmutableTaskGraph object, by taking a snapshot (Step 3 of the blog):
java ImmutableTaskGraph itg = morpheus.snapshot();

And then an TornadoExecutionPlan should be created, which will enable you to invoke the execute method (Step 4 of the blog):
java TornadoExecutionPlan executionPlan = new TornadoExecutionPlan(itg);

Finally, you can execute:
java executionPlan.execute();

8 replies

jjfumero Jun 15, 2023
Maintainer

Hi @MLobatoR , we recently upgraded the maven version and other dependencies. Are you using the latest version from the master branch?
5a93a7c

MLobatoR Jun 15, 2023
Author

Hi @jjfumero,

I followed the instruction on readthedocs. There you have the following:

Clone the latest TornadoVM source code from the GitHub [repository](https://github.com/beehive-lab/TornadoVM) using <MSYS2>/mingw64.exe:

cd D:/MyProjects
git clone https://github.com/beehive-lab/TornadoVM.git
cd TornadoVM

We will refer hereafter the directory with TornadoVM sources as <TornadoVM>.

So I suppose this is the latest version available.

jjfumero Jun 15, 2023
Maintainer

I just checked on Windows 11 and MSys64. It works for me. If it helps, this is my source.sh file:

$ cat sources.sh
#!/bin/bash

# UPDATE PATH TO ACTUAL LOCATION OF THE JDK OR GRAAL (REMEMBER OF UNIX_STYLE SLASHES AND SPACES!!!)
export JAVA_HOME="C:\Users\jjfum\Documents\bin\jvms\graalvm-ce-java17-windows-amd64-22.3.2\graalvm-ce-java17-22.3.2"

## NEXT TWO LINES NECESSARY TO BUILD PTX (NVIDIA CUDA) BACKEND
## COMMENT THEM OUT OR JUST IGNORE IF YOU ARE NOT INTERESTED IN PTX BUILD
## OTHERWISE UPDATE 'CUDA_PATH' WITH ACTUAL VALUE (REMEMBER OF UNIX_STYLE SLASHES AND SPACES!!!)
export CUDA_PATH="C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.6"
export PTX_LDFLAGS=-L\"$CUDA_PATH/lib/x64\"

# LEAVE THE REST OF FILE 'AS IS'
# DON'T ALTER!
export PATH=$PWD/bin/bin:$PATH               ## This directory will be automatically generated during Tornado compilation
export TORNADO_SDK=$PWD/bin/sdk              ## This directory will be automatically generated during Tornado compilation
CMAKE_FILE=$(where cmake | head -n 1)
export CMAKE_ROOT=${CMAKE_FILE%\\*\\*}

Then I just did:

$ . sources.sh
$ make graal-jdk-11-plus 
$ tornado --version
  version=0.15.2-dev
  branch=master
  commit=5a93a7c
  Backends installed:
         - opencl

$ tornado --devices 

Number of Tornado drivers: 1
Driver: OpenCL
  Total number of OpenCL devices  : 4
  Tornado device=0:0  (DEFAULT)
        OPENCL --  [Intel(R) OpenCL HD Graphics] -- Intel(R) UHD Graphics 770
                Global Memory Size: 12.7 GB
                Local Memory Size: 64.0 KB
                Workgroup Dimensions: 3
                Total Number of Block Threads: [256]
                Max WorkGroup Configuration: [256, 256, 256]
                Device OpenCL C version: OpenCL C 1.2

  Tornado device=0:1
        OPENCL --  [NVIDIA CUDA] -- NVIDIA GeForce RTX 3070
                Global Memory Size: 8.0 GB
                Local Memory Size: 48.0 KB
                Workgroup Dimensions: 3
                Total Number of Block Threads: [1024]
                Max WorkGroup Configuration: [1024, 1024, 64]
                Device OpenCL C version: OpenCL C 1.2

  Tornado device=0:2
        OPENCL --  [Intel(R) OpenCL] -- 12th Gen Intel(R) Core(TM) i7-12700K
                Global Memory Size: 31.7 GB
                Local Memory Size: 32.0 KB
                Workgroup Dimensions: 3
                Total Number of Block Threads: [8192]
                Max WorkGroup Configuration: [8192, 8192, 8192]
                Device OpenCL C version: OpenCL C 3.0

  Tornado device=0:3
        OPENCL --  [Intel(R) FPGA Emulation Platform for OpenCL(TM)] -- Intel(R) FPGA Emulation Device
                Global Memory Size: 31.7 GB
                Local Memory Size: 256.0 KB
                Workgroup Dimensions: 3
                Total Number of Block Threads: [67108864]
                Max WorkGroup Configuration: [67108864, 67108864, 67108864]
                Device OpenCL C version: OpenCL C 1.2

MLobatoR Jun 15, 2023
Author

@jjfumero,

Thank you, it worked. If I run test on cmd window they are successful. But I need a little bit more help, please.

I'm running Matrix multiplication on Eclipse IDE. And I'm getting this error:

Computing MxM of 512x512
Exception in thread "main" java.lang.ExceptionInInitializerError
	at tornado.drivers.opencl@0.15.2-dev/uk.ac.manchester.tornado.drivers.opencl.OCLDriver.<init>(OCLDriver.java:65)
	at tornado.drivers.opencl@0.15.2-dev/uk.ac.manchester.tornado.drivers.opencl.OCLTornadoDriverProvider.createDriver(OCLTornadoDriverProvider.java:50)
	at tornado.runtime@0.15.2-dev/uk.ac.manchester.tornado.runtime.TornadoCoreRuntime.loadDrivers(TornadoCoreRuntime.java:173)
	at tornado.runtime@0.15.2-dev/uk.ac.manchester.tornado.runtime.TornadoCoreRuntime.<init>(TornadoCoreRuntime.java:107)
	at tornado.runtime@0.15.2-dev/uk.ac.manchester.tornado.runtime.TornadoCoreRuntime.<clinit>(TornadoCoreRuntime.java:83)
	at tornado.runtime@0.15.2-dev/uk.ac.manchester.tornado.runtime.tasks.LocalObjectState.<init>(LocalObjectState.java:43)
	at tornado.runtime@0.15.2-dev/uk.ac.manchester.tornado.runtime.graph.TornadoExecutionContext.insertVariable(TornadoExecutionContext.java:113)
	at tornado.runtime@0.15.2-dev/uk.ac.manchester.tornado.runtime.graph.TornadoExecutionContext.getObjectState(TornadoExecutionContext.java:259)
	at tornado.runtime@0.15.2-dev/uk.ac.manchester.tornado.runtime.tasks.TornadoTaskGraph.transferToDevice(TornadoTaskGraph.java:873)
	at tornado.api@0.15.2-dev/uk.ac.manchester.tornado.api.TaskGraph.transferToDevice(TaskGraph.java:723)
	at gpuproject.MatrixMultiplication.main(MatrixMultiplication.java:61)
Caused by: uk.ac.manchester.tornado.api.exceptions.TornadoRuntimeException: OpenCL JNI Library not found
	at tornado.drivers.opencl@0.15.2-dev/uk.ac.manchester.tornado.drivers.opencl.OpenCL.<clinit>(OpenCL.java:68)
	... 11 more

I already added the arguments to VM, but nothing seems to work. Do I need to add some specific libraries somewhere?

Thank you in advance for all your help

MLobatoR Jun 15, 2023
Author

@jjfumero,

I found out the problem. It is working now. Now I will check if with the code updated the original problem is fixed. I'll send you some feedback later.

Thank you

MLobatoR · 2023-06-15T17:21:27Z

MLobatoR
Jun 15, 2023
Author

@jjfumero,

I don't understand what is the function that replaces "replaceParameter" of TaskGraph.

		TaskGraph taskGraph = new TaskGraph("s")
				.transferToDevice(DataTransferMode.FIRST_EXECUTION, graph.Va, graph.Ea)
				.transferToDevice(DataTransferMode.EVERY_EXECUTION, edgesToExpand, costsToConsiderPerEdge, pathCost, nbEdgesToExpand)
				.task("t", NeoAlgorithm::expand, graph.Va, graph.Ea, costsToConsiderPerEdge, pathCost, edgesToExpand, results, nbEdgesToExpand)
				.transferToHost(DataTransferMode.EVERY_EXECUTION, results);

This was running multiple times and every time it was running I was doing this and it was working fine:

			taskGraph.replaceParameter(nbEdgesToExpand, newNbEdgesToExpand);
			taskGraph.replaceParameter(edgesToExpand, newEdgesToExpand);
			taskGraph.replaceParameter(pathCost, newPathCost);
			taskGraph.replaceParameter(costsToConsiderPerEdge, newCostsToConsiderPerEdge);
			taskGraph.replaceParameter(results, newResults);

But now the results are not being updated and I think is because I'm not replacing old parameters with the new ones.
How can I replace original parameters to make this work properly?

Thank you,
ML

7 replies

jjfumero Jun 15, 2023
Maintainer

Sorry. wrong repo. This should be the one: https://github.com/beehive-lab/TornadoVM/blob/master/tornado-unittests/src/main/java/uk/ac/manchester/tornado/unittests/executor/TestExecutor.java#L186-L235

MLobatoR Jun 15, 2023
Author

@jjfumero,

I don't think this is what I need. Depending in the result, some other variable will change their value. I was using replace parameter precisely to do that. I don't want to instantiate multiple task graphs, at least in the earliest version i didn't need to do that. I just replaced the parameters in every iteration.

Here's the code as it was in previous version, and it was working fine:

Matrix2DInt costsToConsiderPerEdge = costsToConsiderPerEdgeToExpand(fastest, graph, pathCost, edgesToExpand);
		TaskGraph taskGraph = new TaskGraph("s")
				.lockObjectsInMemory(graph.Va, graph.Ea) 
				.transferToDevice(DataTransferMode.FIRST_EXECUTION, graph.Va, graph.Ea)
				.transferToDevice(DataTransferMode.EVERY_EXECUTION, edgesToExpand, costsToConsiderPerEdge, pathCost, nbEdgesToExpand)
				.task("t", NeoAlgorithm::expand, graph.Va, graph.Ea, costsToConsiderPerEdge, pathCost, edgesToExpand, results, nbEdgesToExpand)
				.transferToHost(results);

		taskGraph.execute();

		update(pathCostArray, mark, nbEdgesToExpand, results);

		while(!stop(mark)) {
			VectorInt newPathCost = new VectorInt(pathCostArray);
			VectorInt newEdgesToExpand = getEdgesToExpand(mark);
			int newNbEdgesToExpand = newEdgesToExpand.size();
			Matrix2DInt newCostsToConsiderPerEdge = costsToConsiderPerEdgeToExpand(fastest, graph, newPathCost, newEdgesToExpand);
			Matrix2DInt newResults = copyArray(pathCostArray, newEdgesToExpand.size());

			taskGraph.replaceParameter(nbEdgesToExpand, newNbEdgesToExpand);
			taskGraph.replaceParameter(edgesToExpand, newEdgesToExpand);
			taskGraph.replaceParameter(pathCost, newPathCost);
			taskGraph.replaceParameter(costsToConsiderPerEdge, newCostsToConsiderPerEdge);
			taskGraph.replaceParameter(results, newResults);

			taskGraph.execute();

			update(pathCostArray, mark, newEdgesToExpand.size(), newResults);

			nbEdgesToExpand = newNbEdgesToExpand;
			edgesToExpand = newEdgesToExpand;
			pathCost = newPathCost;
			costsToConsiderPerEdge = newCostsToConsiderPerEdge;
			results = newResults;
		}

Can you help me comprehend what to do to use the latest version in order for this to work properly please? Because if I remove replace parameters and add the ImmutableTaskGraph this stops working and I don't understand how to make this work again in the latest version.

Thank you,
ML

jjfumero Jun 16, 2023
Maintainer

Hi @MLobatoR . In the previous version, every time we invoke replaceParameter we will trigger recompilation and new buffer allocation. This is not efficient when running on GPUs (or any compatible accelerator) with discrete memory.

Thus. we introduced the TornadoExecutionPlan and we split the task-graph into two parts:
a) Definition: This is the task-graph in which we define data and code to be accelerated.
b) Execution: This is done through execution plans. One execution plan can have one or many immutable task-graphs associated with it.

You can define different task graphs associated with different sets of data and pass data around these graphs through different execution plans (as we show in the unit test).

You can find more details in our documentation: https://tornadovm.readthedocs.io/en/latest/programming.html

We had a similar use case to yours in our Quantum Compute Simulator with TornadoVM:

In this case, data is copied from the host side using System.arraycopy:

https://github.com/beehive-lab/TornadoQSim/blob/main/TornadoQSim/src/main/java/uk/ac/manchester/tornado/qsim/simulator/unitary/UnitarySimulatorAccelerated.java#L148-L168

A side note: we are improving TornadoVM with new types to avoid extra copies between Java heap <-> native memory. Thus, in the near future, this part will be also optimized.

jjfumero Jun 16, 2023
Maintainer

We also have documented how to transition from the previous API to the new Execution Plan:
https://tornadovm.readthedocs.io/en/latest/programming.html#migration-to-tornadovm-v0-15

MLobatoR Jun 16, 2023
Author

@jjfumero,

Thank you for all your help. I'll try using what you suggested.

ML

MLobatoR · 2023-06-22T15:10:36Z

MLobatoR
Jun 22, 2023
Author

Hi @jjfumero,

Followinh your suggestion I updated my code. Now I get this warning when transfer data to host:
Type int[][] of the last argument to method transferToHost(int, Object...) doesn't exactly match the vararg parameter type. Cast to Object[] to confirm the non-varargs invocation, or pass individual arguments of type Object for a varargs invocation.

The object I'm trying to transfer is int[][], nothing special, in my opinion.

Because I'm ignoring this warning when I execute I have an error.

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  uk.ac.manchester.tornado.drivers.opencl.OCLCommandQueue.writeArrayToDevice(J[BJZJJJ[J)J+0 tornado.drivers.opencl@0.15.2-dev
j  uk.ac.manchester.tornado.drivers.opencl.OCLCommandQueue.enqueueWrite(JZJJ[BJ[J)J+35 tornado.drivers.opencl@0.15.2-dev
j  uk.ac.manchester.tornado.drivers.opencl.OCLDeviceContext.enqueueWriteBuffer(JJJ[BJ[I)I+44 tornado.drivers.opencl@0.15.2-dev
j  uk.ac.manchester.tornado.drivers.opencl.mm.OCLByteBuffer.enqueueWrite([I)I+25 tornado.drivers.opencl@0.15.2-dev
j  uk.ac.manchester.tornado.drivers.opencl.mm.OCLArrayWrapper.enqueueWrite(Ljava/lang/Object;JJ[IZ)Ljava/util/List;+57 tornado.drivers.opencl@0.15.2-dev
j  uk.ac.manchester.tornado.drivers.opencl.runtime.OCLTornadoDevice.streamIn(Ljava/lang/Object;JJLuk/ac/manchester/tornado/api/memory/TornadoDeviceObjectState;[I)Ljava/util/List;+31 tornado.drivers.opencl@0.15.2-dev
j  uk.ac.manchester.tornado.runtime.TornadoVM.transferHostToDeviceAlways(Ljava/lang/StringBuilder;IIJIJ[I)I+155 tornado.runtime@0.15.2-dev
j  uk.ac.manchester.tornado.runtime.TornadoVM.execute(Z)Luk/ac/manchester/tornado/api/common/Event;+455 tornado.runtime@0.15.2-dev
j  uk.ac.manchester.tornado.runtime.TornadoVM.execute()Luk/ac/manchester/tornado/api/common/Event;+2 tornado.runtime@0.15.2-dev
j  uk.ac.manchester.tornado.runtime.tasks.TornadoTaskGraph.scheduleInner()V+41 tornado.runtime@0.15.2-dev
j  uk.ac.manchester.tornado.runtime.tasks.TornadoTaskGraph.schedule()Luk/ac/manchester/tornado/api/TornadoTaskGraphInterface;+119 tornado.runtime@0.15.2-dev
j  uk.ac.manchester.tornado.api.TaskGraph.execute()V+4 tornado.api@0.15.2-dev
j  uk.ac.manchester.tornado.api.ImmutableTaskGraph.execute()V+4 tornado.api@0.15.2-dev
j  uk.ac.manchester.tornado.api.TornadoExecutionPlan$TornadoExecutor$$Lambda$117.accept(Ljava/lang/Object;)V+4 tornado.api@0.15.2-dev
j  java.util.ArrayList.forEach(Ljava/util/function/Consumer;)V+46 java.base@11.0.19
j  uk.ac.manchester.tornado.api.TornadoExecutionPlan$TornadoExecutor.execute()V+9 tornado.api@0.15.2-dev
j  uk.ac.manchester.tornado.api.TornadoExecutionPlan.execute()Luk/ac/manchester/tornado/api/TornadoExecutionResult;+54 tornado.api@0.15.2-dev
j  neoversion02.rungpu.NeoAlgorithmGPU.run(ZI)[I+268
j  neoversion02.TestRunner.runOnGPU(ZI)[I+19
j  neoversion02.TestRunner.main([Ljava/lang/String;)V+12
v  ~StubRoutines::call_stub

siginfo: EXCEPTION_ACCESS_VIOLATION (0xc0000005), reading address 0x000000000000003f

My question is what are the objects you support on transfer to host. The examples I saw where float[].

What can I do here?

Thank you in advance for your help,
ML

5 replies

jjfumero Jun 22, 2023
Maintainer

Hi @MLobatoR . int[][] or arrays of arrays data structures are not supported in TornadoVM. Instead, TornadoVM exposes types for allocating internal memory in consecutive device memory regions and avoids extra marshalling/unmarshalling of objects.

For example https://github.com/beehive-lab/TornadoVM/blob/master/tornado-api/src/main/java/uk/ac/manchester/tornado/api/collections/types/Matrix2DFloat.java

We have examples in the unit-tests and in the example module of TornadoVM:

MLobatoR Jun 22, 2023
Author

@jjfumero if I use those and I'm not able to use replace parameter anymore, how I'm supposed to follow your previous suggestion and use System.arraycopy to replace some parameters?

Thank you,
ML

jjfumero Jun 24, 2023
Maintainer

The replaceParameters method within the TaskSchedule was removed in version 0.15. We do not need this call because we can use multiple task-graphs and move data across them. I think you should create multiple task-graphs (as many as you need) and multiple execution plans. And move data around as we point out in the examples.

If this approach does not fit your needs, feel free to create a simple test-case, and send us a proposal with the feature/s you would like to add in TornadoVM. It is important to understand why your use case could not use multiple execution plans instead, and how we could approach your patterns/examples. The way to open proposals is via GitHub discussions.

Answer selected by jjfumero

MLobatoR Jul 26, 2023
Author

@jjfumero it is working fine, thank you for all your help :)

jjfumero Jul 26, 2023
Maintainer

Thank you for the follow-up. I will close this issue.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

First execution works but the next ones don't do anything #258

{{title}}

Replies: 3 comments 20 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

First execution works but the next ones don't do anything #258

MLobatoR Jun 1, 2023

Replies: 3 comments · 20 replies

stratika Jun 2, 2023 Maintainer

jjfumero Jun 15, 2023 Maintainer

MLobatoR Jun 15, 2023 Author

jjfumero Jun 15, 2023 Maintainer

MLobatoR Jun 15, 2023 Author

MLobatoR Jun 15, 2023 Author

MLobatoR Jun 15, 2023 Author

jjfumero Jun 15, 2023 Maintainer

MLobatoR Jun 15, 2023 Author

jjfumero Jun 16, 2023 Maintainer

jjfumero Jun 16, 2023 Maintainer

MLobatoR Jun 16, 2023 Author

MLobatoR Jun 22, 2023 Author

jjfumero Jun 22, 2023 Maintainer

MLobatoR Jun 22, 2023 Author

jjfumero Jun 24, 2023 Maintainer

MLobatoR Jul 26, 2023 Author

jjfumero Jul 26, 2023 Maintainer

MLobatoR
Jun 1, 2023

Replies: 3 comments 20 replies

stratika
Jun 2, 2023
Maintainer

jjfumero Jun 15, 2023
Maintainer

MLobatoR Jun 15, 2023
Author

jjfumero Jun 15, 2023
Maintainer

MLobatoR Jun 15, 2023
Author

MLobatoR Jun 15, 2023
Author

MLobatoR
Jun 15, 2023
Author

jjfumero Jun 15, 2023
Maintainer

MLobatoR Jun 15, 2023
Author

jjfumero Jun 16, 2023
Maintainer

jjfumero Jun 16, 2023
Maintainer

MLobatoR Jun 16, 2023
Author

MLobatoR
Jun 22, 2023
Author

jjfumero Jun 22, 2023
Maintainer

MLobatoR Jun 22, 2023
Author

jjfumero Jun 24, 2023
Maintainer

MLobatoR Jul 26, 2023
Author

jjfumero Jul 26, 2023
Maintainer