Consider a sample file with data as below:
123,Test,data <br>
345,row2,data2,test2 <br>
554,row3,data3 <br>
788,row4,data4,with,all,columnvalues <br>
655,row5,data5,with2,column2
Notice that the rows are having unequal number of columns.
I have tried with databricks spark-csv, with inferSchema option set to 'true', but it infers schema by the first row and therefore the extra cols in the other rows are dropped off.
Is there any way to read such data?