You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I would like to submit to Spark a 32-bit driver program. The driver program is the .NET Framework console app:
the app is rather simple and features a very simple testing code:
public static void Main(string[] args)
{
var conf = new SparkConf();
var sparkContext = new SparkContext(conf);
var rdd = sparkContext.Parallelize(new List<string>() { "a", "b", "c", "d" });
var response = rdd.Map(s => s).Collect();
}
After I properly submit this app to sparkclr (that is, to mobius which then passes it to Spark itself), I get the following exception:
System.Exception: JVM method execution failed: Static method collectAndServe failed for class
org.apache.spark.api.python.PythonRDD when called with 1 parameters ([Index=1,
Type=JvmObjectReference, Value=11], )
at Microsoft.Spark.CSharp.Interop.Ipc.JvmBridge.CallJavaMethod(Boolean isStatic, Object
classNameOrJvmObjectReference, String methodName, Object[] parameters)
at Microsoft.Spark.CSharp.Interop.Ipc.JvmBridge.CallStaticJavaMethod(String className, String
methodName, Object[] parameters)
at Microsoft.Spark.CSharp.Proxy.Ipc.RDDIpcProxy.CollectAndServe()
at Microsoft.Spark.CSharp.Core.RDD`1.Collect()
The exception goes away when I build the app in x64. Also, the exception is not there when the app is built in x86 but does nothing (i.e. has no
var rdd = sparkContext.Parallelize(new List<string>() { "a", "b", "c", "d" });
var response = rdd.Map(s => s).Collect();
lines).
Is there any workaround to submit 32bit app to Spark?
(In the original question I thought the issue could be with Java, but later found this was not the case).
With kind regards,
Lesia
The text was updated successfully, but these errors were encountered:
I've asked this question on stackoverflow (https://stackoverflow.com/questions/48323609/submitting-a-32-bit-application-to-spark-via-mobius):
I would like to submit to Spark a 32-bit driver program. The driver program is the .NET Framework console app:
the app is rather simple and features a very simple testing code:
After I properly submit this app to sparkclr (that is, to mobius which then passes it to Spark itself), I get the following exception:
The exception goes away when I build the app in x64. Also, the exception is not there when the app is built in x86 but does nothing (i.e. has no
lines).
Is there any workaround to submit 32bit app to Spark?
(In the original question I thought the issue could be with Java, but later found this was not the case).
With kind regards,
Lesia
The text was updated successfully, but these errors were encountered: