When I try to use the dyF.show() it returns an empty field, even though I checked the schema and count() and I know the table is populated. I transformed it into a spark dataframe and the show() method works fine.
I know that this has happened to others as well, is it a bug? Or it has to do with the file I am reading?
dyF.show()
but when I try it with a normal df:
df.show()
----+-------+-----------------+----------------+-------------------+-------------+-----------+---------------+----------------+---------------+------------+-------+------------------+----------------+
|YEAR|CARRIER| CARRIER_NAME|MANUFACTURE_YEAR|UNIQUE_CARRIER_NAME|SERIAL_NUMBER|TAIL_NUMBER|AIRCRAFT_STATUS|OPERATING_STATUS|NUMBER_OF_SEATS|MANUFACTURER| MODEL|CAPACITY_IN_POUNDS|ACQUISITION_DATE|
+----+-------+-----------------+----------------+-------------------+-------------+-----------+---------------+----------------+---------------+------------+-------+------------------+----------------+
|2006| 16|PSA Airlines Inc.| 2003| PSA Airlines Inc.| 7858| N202PS| B| Y| 50| CANADAIR|CRJ-2/4| 47000| 2003-10-28|
|2006| 16|PSA Airlines Inc.| 2003| PSA Airlines Inc.| 7860| N206PS| B| Y| 50| CANADAIR|CRJ-2/4| 47000| 2003-10-30|
|2006| 16|PSA Airlines Inc.| 2003| PSA Airlines Inc.| 7873| N207PS| B| Y| 50| CANADAIR|CRJ-2/4| 47000| 2003-11-26|
|2006| 16|PSA Airlines Inc.| 2003| PSA Airlines Inc.| 7874| N209PS| B| Y| 50| CANADAIR|CRJ-2/4| 47000| 2003-12-04|
|2006| 16|PSA Airlines Inc.| 2003| PSA Airlines Inc.| 7879| N213PS| B| Y| 50| CANADAIR|CRJ-2/4| 47000| 2003-12-16|
|2006| 16|PSA Airlines Inc.| 2003| PSA Airlines Inc.| 7880| N215PS| ...
Please check the option job bookmarks in job details. Change it to disable. For more information on job bookmark please refer link Job Bookmarks.
It will show data in the dataframe. Also show the data using the below syntax:
dyf.toDF().show()