Common Exceptions and possible reasons

java.lang.NoClassDefFoundError

This is probably when you are missing a dependency, or have a problem with the binary version of a dependency. Some libraries required at run time, such as Hadoop and many of Hadoop's includes, are expected to be on the classpath and are not bundled by Scalding itself. So check for that.

java.lang.AbstractMethodError

This probably is also a jar version problem.

java.lang.NoSuchMethodError

This probably is also a jar version problem.

java.lang.ClassNotFoundException

1. Your classpath or jar is missing a needed class. Check your build, or perhaps the name of the job you are trying to run.
2. Make sure you specified the fully packaged name of the class you wish to run package.JobName and not just JobName.

java.lang.UnsupportedClassVersionError

You have compiled your code with a later version of javac/scalac than is supported by the java vm you are attempting to run with. You need to either downgrade the compiler or upgrade the java installation.

java.util.zip.ZipException

: invalid CEN header (bad signature)

This is the jar file exceeding 64k problem. The solution is to use --bundle option of scald.rb.

java.lang.NullPointerException

Caused by: java.lang.NullPointerException
at cascading.scheme.Scheme.setSourceFields(Scheme.java:166)
at cascading.scheme.Scheme.<init>(Scheme.java:108)
at ...my class initialization ...
at ...my class initialization ...
at ...my class initialization ...
... 10 more

This null pointer occurs during initialization of classes: Scheme<init> refers to the initialization of static members in the Scheme class. A NullPointerException during the initialization of a class often happens because it references static fields in another class, which has not been initialised yet. This can be difficult to solve because Java has no way to specify the order in which classes are initialized. In my case I had:

import cascading.tuple.Fields;
var fields = Fields.ALL

And I was passing the fields into the Scheme initialization. Writing directly Fields.ALL instead of using the variable fields solved the problem.

cascading.flow.planner.PlannerException

Cascading requires all sources to have final sinks on disk. This exception happens when you miss an output for an input.

It also could signify an attempt to write an unserializable datatype.

This exception could also be triggered by creating a non-lazy Finagle client.

com.twitter.scalding.InvalidSourceException

This happens when data is missing from the path you provided.

java.lang.RuntimeException: Please only provide a single value for --some input key here

Try putting quotes around your input value

cascading.flow.FlowException: step failed: (1/2) .../algos/4/data/userIds.tsv, with job id: ...

Common exception when running hadoop commands with sudo. Check that you have given permissions for the user (root) in hdfs sudo addgroup supergroup; sudo adduser root supergroup