how to resolve Arithmetic overflow error in Databricks SQL Editor

1.2k Views Asked by At

In Databricks SQL editor , I am trying to get the SUM of few 'Bigint' and 'Double' data type columns of a table. But am getting below error.

Job aborted due to stage failure: Task 0 in stage 29.0 failed 4 times, most recent 
failure: Lost task 0.3 in stage 29.0 (TID 2517) (10.128.2.66 executor 3): 
org.apache.spark.SparkArithmeticException: [ARITHMETIC_OVERFLOW] long overflow. If 
necessary set ansi_mode to "false" to bypass this error.

How to set ansi_mode from Databricks SQL editor?

I tried below in SQL editor

SET spark.sql.ansi.enabled = false

Getting below error

Error running query: org.apache.spark.sql.AnalysisException: Configuration 
spark.sql.ansi.enabled is not available.

How to resolve this Arithmetic overflow error in Databricks?

1

There are 1 best solutions below

2
On

It should be just:

set ansi_mode = false;

See documentation for supported configurations, and examples for set command.