In Web_Scraping the list is returning empty list

160 Views Asked by At
import requests
from requests import get
from bs4 import BeautifulSoup
import pandas as pd
import numpy as np

url = "https://www.covid19india.org/"
headers = {"Accept-Language":"en-US, en;q=0.5"}
results = requests.get(url,headers = headers)
soup = BeautifulSoup(results.text,"html.parser")
cases_div = soup.find_all('div', class_="Level1")
print(cases_div)

My expect output is the HTML

However, I am getting an empty list while printing cases_div.

Why is that and how can I fix it?

1

There are 1 best solutions below

3
On

It seems that the specified website uses React, and on the first HTTP request you will not get the whole content. Try to use selenium or try to find the API requests to the server as was suggested in the comment.