SCRAPY not printing all the items in the terminal

Question

I am using this command in the terminal using SCRAPY Shell but it's not printing all the items.

scrapy shell https://access.redhat.com/errata/RHSA-2017:0621

response.xpath('normalize-space((//div[contains(@class, "tab-pane")]/ul)[2]/li/text())').getall()

It just prints the first item.

Answer 1

normalize-space() only works with a single node. If you give it a nodeset, it will return the value produced from the first node.

If you want to apply it to multiple nodes, you can do that as follows (pp is just a pretty-printing function):

>>> products = response.xpath('(//div[contains(@class, "tab-pane")]/ul)[2]/li').xpath('normalize-space()').getall()
>>> pp(products)
[
    'Red Hat Enterprise Linux Server 6 x86_64',
    'Red Hat Enterprise Linux Server 6 i386',
    'Red Hat Enterprise Linux Workstation 6 x86_64',
    'Red Hat Enterprise Linux Workstation 6 i386',
    'Red Hat Enterprise Linux Desktop 6 x86_64',
    'Red Hat Enterprise Linux Desktop 6 i386',
    'Red Hat Enterprise Linux for Power, big endian 6 ppc64',
    'Red Hat Enterprise Linux for Scientific Computing 6 x86_64'
]