I am experimenting with apache spark 3 in intellij by creating a simple standalone scala application. When I run my program I get lots of INFO logs. Based on various SO answers I tried all of the following:
spark.sparkContext.setLogLevel("ERROR")
SparkSession.builder.getOrCreate().sparkContext.setLogLevel("ERROR")
Logger.getRootLogger().setLevel(Level.ERROR)
Logger.getLogger(classOf[RackResolver]).getLevel
Logger.getLogger("org").setLevel(Level.OFF)
Logger.getLogger("akka").setLevel(Level.OFF)
But none works. I still see tons of INFO logs.
So what's the working way of achieving this?