How to merge a Dataframe with new column into Hudi table in Spark

Question

I have a Hudi table origin like :

id     a     b 
1     123    321
2     abc    cba
3     456    654

And a Dataframe modification generated by some other files like:

id     a     b      c
1     xxx    xxx    a
3     xxx    xxx    b

I want to merge the modification into origin, including the new column c in modification. But the MERGE INTO query would raise an exception saying origin and modification have uncompatable columns. How could I do this?

Answer 1

Attaching a link for reference that could help: https://hudi.apache.org/docs/schema_evolution/