common-protos for ScalaPB 0.11

[Tochemey]

No description provided.

[thesamet]

Hi @Tochemey , thanks for reporting. The release of common-protos for ScalaPB 0.11 is blocked by googleapis/java-common-protos#134 as currently the Scala-Java converters do not compile due to a mismatch between the Java classes and protos shipped with common-protos.

[thesamet]

This is currently blocked due to googleapis/java-common-protos#173

[Tochemey]

@thesamet it seems the blocker has been resolved googleapis/java-common-protos#173.

[thesamet]

This is currently blocked due to googleapis/java-common-protos#179

[juliano]

@thesamet it's finally available! https://github.com/googleapis/java-common-protos/releases/tag/v2.4.1

From: googleapis/java-common-protos#179

[thesamet]

Yes. However, it comes with a different inconsistency between the protos and Java code (googleapis/java-common-protos#210). I think that this time the impact is minor enough that we can just exclude this proto from our build and list this as a known issue for ScalaPB/common-protos.

[thesamet]

Released 2.4.1-0 for google-common-protos and 1.96.2-0 for google cloud pubsub.

[Ic3fr0g]

@thesamet since this is an important release (finally able to work with Spark 3.1.2, sparksql-scalapb and common-protos) can you update the documentation in https://scalapb.github.io/docs/ ?

[Ic3fr0g]

FYI the workaround is not required now - googleapis/java-common-protos#214

[thesamet]

@Ic3fr0g Updated the docs. Also released 2.5.0 for proto-google-common-protos which indeed doesn't require the workaround.

[Visorgood]

Hello @thesamet !
Could you please clarify, which exact library dependency should we use with Spark 3.1.2?
I am trying

"com.thesamet.scalapb" %% "sparksql31-scalapb0_11" % "1.0.0"

but I get the error:

found version conflict(s) in library dependencies; some are suspected to be binary incompatible:
[error] 
[error] 	* org.typelevel:frameless-core_2.12:0.12.0 (early-semver) is selected over 0.10.1
[error] 	    +- org.typelevel:frameless-dataset-spark31_2.12:0.12.0 (depends on 0.12.0)
[error] 	    +- org.typelevel:frameless-dataset_2.12:0.10.1        (depends on 0.10.1)

[thesamet]

Hi @Visorgood , it looks like your question is unrelated to common-protos (which is the issue you have posted it on). Can you file a new issue, also can you investigate how frameless-dataset version 0.10.1 gets into your build? It appears to be causing the error. You are using the correct dependency for the spark version you use.

[Visorgood]

Thank you @thesamet ! You are right, the problem is a different one. I've solved it by adding exclusion for frameless-dataset library in this way:

excludeDependencies ++= Seq(
  ExclusionRule("org.typelevel", "frameless-dataset_2.12")
)