how to categorize the out put file of the python scraping website with selenium

48 Views Asked by At

this is my code and they are right. But I get the data from tables in blend way which is not in order. May anybody can categorize them?

from selenium import webdriver
import time

url = "https://www150.statcan.gc.ca/n1/pub/71-607-x/2021004/exp-eng.htm?r1=(1)&r2=0&r3=0&r4=12&r5=0&r7=0&r8=2022-02-01&r9=2022-02-01"

driver = webdriver.Chrome("C:\Python\chromedriver.exe")
driver.get(url)
table = driver.find_element_by_id('report_table')
body = table.find_element_by_tag_name('tbody')
cells = body.find_elements_by_tag_name('td')
#---------------------------------------------------------
f = open("Canada.txt", "w")
for cell in cells:
    f.write(str(cell.text))
#--------------------------------------------------------

# run the loop 26 times
i = 26 
for i in range(27):
    
    # your code
    table = driver.find_element_by_id('report_table')
    body = table.find_element_by_tag_name('tbody')
    cells = body.find_elements_by_tag_name('td')
    for cell in cells:
        f.write(str(cell.text))
        
    # click on the Next button    
    driver.find_element_by_xpath('//*[@id="report_results_next"]').click()

f.close()

The out put is really MESSY.

0

There are 0 best solutions below