LXML - parse td content within tr tag

519 Views Asked by At

I want to parse each individual statistic from the yahoo finance tables for formatting purposes - when parsing the entire table the formatting is terrible!! I am currently using the code below and I would have to repeat the 4 lines of contentA code slightly altered to retrieve the stats within each row of the table. This is exemplified in the contentB variables below. I refuse to believe this is the most efficient way to do so. Any suggestions?

from lxml import html   

url = 'http://finance.yahoo.com/q/is?s=MMM+Income+Statement&annual'

tree = html.parse(url)

contentA = tree.xpath("//table[@class='yfnc_tabledata1']/tr[1]/td/table/tr[2]/td[1]")[0].text_content().strip()
contentA1 = tree.xpath("//table[@class='yfnc_tabledata1']/tr[1]/td/table/tr[2]/td[2]")[0].text_content().strip()
contentA2 = tree.xpath("//table[@class='yfnc_tabledata1']/tr[1]/td/table/tr[2]/td[3]")[0].text_content().strip()
contentA3 = tree.xpath("//table[@class='yfnc_tabledata1']/tr[1]/td/table/tr[2]/td[4]")[0].text_content().strip()

contentB = tree.xpath("//table[@class='yfnc_tabledata1']/tr[1]/td/table/tr[3]/td[1]")[0].text_content().strip()
contentB1 = tree.xpath("//table[@class='yfnc_tabledata1']/tr[1]/td/table/tr[3]/td[2]")[0].text_content().strip()
contentB2 = tree.xpath("//table[@class='yfnc_tabledata1']/tr[1]/td/table/tr[3]/td[3]")[0].text_content().strip()
contentG3 = tree.xpath("//table[@class='yfnc_tabledata1']/tr[1]/td/table/tr[3]/td[4]")[0].text_content().strip()
1

There are 1 best solutions below

4
On BEST ANSWER

Use range and format

for i in range(1,5):
    contentA = tree.xpath("//table[@class='yfnc_tabledata1']/tr[1]/td/table/tr[2]/td[{i}]".format(i=i))[0].text_content().strip()
    print(contentA)

Output

Total Revenue
31,821,000
30,871,000
29,904,000

for i in range(1,5):
    contentB = tree.xpath("//table[@class='yfnc_tabledata1']/tr[1]/td/table/tr[3]/td[{i}]".format(i=i))[0].text_content().strip()
    print(contentB)

Output

Cost of Revenue
16,447,000
16,106,000
15,685,000

EDIT

In [22]: d = {}

In [23]: d.setdefault('Revenue', [])
Out[23]: []

In [24]: for i in range(2,5):
   ....:     contentB = tree.xpath("//table[@class='yfnc_tabledata1']/tr[1]/td/table/tr[3]/td[{i}]".format(i=i))[0].text_content().strip()
   ....:     d['Revenue'].append(int(contentB.replace(',', '')))
   ....:     

In [25]: d
Out[25]: {'Revenue': [16447000, 16106000, 15685000]}