Web scraping with python and selenium

Question

I am trying to extract data from a website that is structured in this matter using python and selenium:

  <table> 
     <tbody>
         <tr> 
            <td> text </td>
            <td>
               <td> text </td>
            </td> 
            <td> 
               <td> text </td>
            </td>
          <tr>  
            <td> text </td>
            <td>
               <td> text </td>
            </td> 
            <td> 
               <td> text </td>
            </td>

This the code I have for the data extraction using selenium:

data=[]
        for tr in driver.find_elements_by_xpath('//table[@id="pinnedtablepositionsTable"]//tr'):
            tds =tr.find_elements_by_tag_name('td')
        if tds: 
                 data.append([td.text for td in tds])

print(data)

When I print the data I am only the first column of the for each row in the website. The table on the website is structured in a matter where the rows are nested inside of each other. Has any one ever dealt with a website that is structured in this format. I am open to other scrapers besides selenium I just like to use it for automation purposes.

Answer 1

I think the indentation is wrong for your program , It should be -

data=[]
for tr in driver.find_elements_by_xpath('//table[@id="pinnedtablepositionsTable"]//tr'):
    tds =tr.find_elements_by_tag_name('td')
    if tds: 
        data.append([td.text for td in tds])