Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Null Pointer Exception with slf4j-log4j12-1.7.30 (Fluent-logger incompatible with the new version) #89

Open
anusricorp opened this issue Aug 22, 2020 · 3 comments
Open

Null Pointer Exception with slf4j-log4j12-1.7.30 (Fluent-logger incompatible with the new version) #89

anusricorp opened this issue Aug 22, 2020 · 3 comments

Comments

@anusricorp
Copy link

anusricorp commented Aug 22, 2020 •
edited

When running spark-submit with slf4j-log4j12-1.7.18.jar in the ClassPath, FluentLogger works with no issues.
When we use version 1.7.30 which comes with Spark3.0, Fluentlogger.getLogger throws a null pointer exception.

This is the offending commit introduced in 1.7.19  which was done to fix a NullPointerException. Its force initializing rootLogger to fix a NPE in multithreaded scenarios looks like. The irony is this is the commit which is causing NullPointerException for us
qos-ch/slf4j@111b8e5

Can we upgrade the version of slf4j-api in FluentLogger to latest 1.7.30 and make it compatible?

hdiuser@hn0-d0140a:/usr/hdp/4.1-0/spark2/jars$ spark-submit --version
SPARK_MAJOR_VERSION is set to 2, using Spark2
Unexpected problem occured during version sanity check
Reported exception:
java.lang.NullPointerException
at org.slf4j.LoggerFactory.versionSanityCheck(LoggerFactory.java:267)
at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:126)
at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:412)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:357)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383)
at org.fluentd.logger.sender.RawSocketSender.(RawSocketSender.java:34)
at org.fluentd.logger.FluentLoggerFactory.getLogger(FluentLoggerFactory.java:72)
at org.fluentd.logger.FluentLoggerFactory.getLogger(FluentLoggerFactory.java:51)
at org.fluentd.logger.FluentLogger.getLogger(FluentLogger.java:40)
at com.mdsdclient.MdsLogger.(Unknown Source)
at com.log4jappender.common.logger.HdInsightLoggerLinux.(Unknown Source)
at com.log4jappender.common.logger.HdInsightLoggerFactory.getLogger(Unknown Source)
at com.log4jappender.common.appendercore.AnonymizeLogAppenderCommon.getLogger(Unknown Source)
at com.log4jappender.common.appendercore.AnonymizeLogAppenderCommon.(Unknown Source)
at com.microsoft.log4jappender.AnonymizeLogAppender.activateOptions(Unknown Source)
at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842)
at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
at org.apache.log4j.LogManager.(LogManager.java:127)
at org.slf4j.impl.Log4jLoggerFactory.(Log4jLoggerFactory.java:66)
at org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:72)
at org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:45)
at org.apache.spark.internal.Logging$.org$apache$spark$internal$Logging$$isLog4j12(Logging.scala:222)
at org.apache.spark.internal.Logging.initializeLogging(Logging.scala:127)
at org.apache.spark.internal.Logging.initializeLogIfNecessary(Logging.scala:111)
at org.apache.spark.internal.Logging.initializeLogIfNecessary$(Logging.scala:105)
at org.apache.spark.deploy.SparkSubmit.initializeLogIfNecessary(SparkSubmit.scala:75)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:83)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1007)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1016)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Exception in thread "main" java.lang.ExceptionInInitializerError
at org.fluentd.logger.FluentLoggerFactory.getLogger(FluentLoggerFactory.java:72)
at org.fluentd.logger.FluentLoggerFactory.getLogger(FluentLoggerFactory.java:51)
at org.fluentd.logger.FluentLogger.getLogger(FluentLogger.java:40)
at com.mdsdclient.MdsLogger.(Unknown Source)
at com.log4jappender.common.logger.HdInsightLoggerLinux.(Unknown Source)
at com.log4jappender.common.logger.HdInsightLoggerFactory.getLogger(Unknown Source)
at com.log4jappender.common.appendercore.AnonymizeLogAppenderCommon.getLogger(Unknown Source)
at com.log4jappender.common.appendercore.AnonymizeLogAppenderCommon.(Unknown Source)
at com.log4jappender.AnonymizeLogAppender.activateOptions(Unknown Source)
at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842)
at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
at org.apache.log4j.LogManager.(LogManager.java:127)
at org.slf4j.impl.Log4jLoggerFactory.(Log4jLoggerFactory.java:66)
at org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:72)
at org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:45)
at org.apache.spark.internal.Logging$.org$apache$spark$internal$Logging$$isLog4j12(Logging.scala:222)
at org.apache.spark.internal.Logging.initializeLogging(Logging.scala:127)
at org.apache.spark.internal.Logging.initializeLogIfNecessary(Logging.scala:111)
at org.apache.spark.internal.Logging.initializeLogIfNecessary$(Logging.scala:105)
at org.apache.spark.deploy.SparkSubmit.initializeLogIfNecessary(SparkSubmit.scala:75)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:83)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1007)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1016)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.NullPointerException
at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:418)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:357)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383)
at org.fluentd.logger.sender.RawSocketSender.(RawSocketSender.java:34)
... 31 more
@xerial
Copy link
Contributor

xerial commented Aug 25, 2020

Thanks for reporting. If you are using Spark, airframe-fluentd can be an alternative for a while, which can be used with Scala https://wvlet.org/airframe/docs/airframe-fluentd

I'll try to update slf4j version too

xerial added a commit that referenced this issue Aug 25, 2020
@xerial
#89: Upgrade slf4j-api to 1.7.30
b3f326d
@anusricorp anusricorp changed the title Null Pointer Exception with slf4j-api-1.7.30 (Fluent-logger incompatible with the new version) Null Pointer Exception with slf4j-log4j12-1.7.30 (Fluent-logger incompatible with the new version) Aug 25, 2020
@Niharikadutta
Copy link

Hi @xerial do we have any update on this issue? We are facing a similar issue with NullPointerException errors in slf4j v1.7.30:

	21/08/05 22:21:01 INFO LineBufferedStream: stdout: SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: Failed to instantiate SLF4J LoggerFactory
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: Reported exception:
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: java.lang.NullPointerException
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.slf4j.LoggerFactory.reportActualBinding(LoggerFactory.java:349)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.slf4j.LoggerFactory.bind(LoggerFactory.java:152)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:417)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:362)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:388)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.fluentd.logger.sender.RawSocketSender.<clinit>(RawSocketSender.java:34)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.fluentd.logger.FluentLoggerFactory.getLogger(FluentLoggerFactory.java:72)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.fluentd.logger.FluentLoggerFactory.getLogger(FluentLoggerFactory.java:51)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.slf4j.impl.Log4jLoggerFactory.<init>(Log4jLoggerFactory.java:66)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.slf4j.impl.StaticLoggerBinder.<init>(StaticLoggerBinder.java:72)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:45)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.spark.internal.Logging$.org$apache$spark$internal$Logging$$isLog4j12(Logging.scala:222)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.spark.internal.Logging.initializeLogging(Logging.scala:127)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.spark.internal.Logging.initializeLogIfNecessary(Logging.scala:111)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.spark.internal.Logging.initializeLogIfNecessary$(Logging.scala:105)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.spark.deploy.SparkSubmit.initializeLogIfNecessary(SparkSubmit.scala:75)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:83)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1039)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1048)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: Exception in thread "main" java.lang.ExceptionInInitializerError
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.fluentd.logger.FluentLoggerFactory.getLogger(FluentLoggerFactory.java:72)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.fluentd.logger.FluentLoggerFactory.getLogger(FluentLoggerFactory.java:51)

@KeerthiYandaOS
Copy link

Hi @xerial, is the above NullPointerException is fixed? we are seeing the exact same error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@xerial @Niharikadutta @anusricorp @KeerthiYandaOS