How can I scrape the text from this popup window with scrapy python?

40 Views Asked by At

I'm trying to extract the data inside the popup window on a page with scrapy. This is the page: https://todoinmuebles.net/anuncios/inmuebles/450077309169-miraflores-av-jose-pardo-departamento-219-m2-82741aba . But when I click to open the popup the URL is the same.

I tried Splash but I got the same result.

this is what I want to extract.

enter image description here

But when I check the response in the scrapy shell I got 000 000 000 instead the real number.

This is what i got

1

There are 1 best solutions below

0
Leandro Rodrigues de Souza On

In your case, it seems that, when you send the message (per your "Message Sent" confirmation screenshot), the website makes an AJAX request to get the contact information for the ad. This request, which I highlighted in the image, is the one that contains the contact information you need. You can schedule the request directly from your Spider code.

Since it is an async request made after you send the message, if you just grab the popup HTML before this request is issued, you'll get just a placeholder value, which seems to be the case based on your description.

Chrome Dev Tools