Calculating column value in current row of Spark Dataframe based on the calculated value of a column in previous row using java

46 Views Asked by At

Let's say I have static value as 10000

Now, I have a Dataframe like below

Col A Col B
A 500
S 100
A 1000
S 100

With Transformation I want it like below

Col A Col B Col C
A 500 10500
S 100 10400
A 1000 11400
S 100 11300

The thing works by the details provided below

  • For the first row based on Col A value (A for add and S for subtract) I am adding value Col B value with static value to get Col C value

  • From next row onward based on Col A value I am adding or subtracting Col B value with Col C value of previous row.

I was trying to find out Col A value using spark functions class but haven't found anything. Please help me to form the logic.

0

There are 0 best solutions below