I'm working on a project which requires the category which google uses to define that business/store. I'm using beautiful soup library but I am unable to extract that specific part of the result. I'll show you with a help of an example using the image below: click to open the image
when I search "bell canada" for example, I get search results and also you can see a box which google shows on the right side of the page which summaries the search "bell canada". So my task required, extracting the subtitle under BELL CANADA named as Telecommunications company . How can I achieve this for any search on google using web scraping.
user-agent
e.g Chrome version.select_one()
method frombeautifulsoup
which usesCSS
selectors to find an element. Sometimes it's better thanfind()
(check out SelectorGadget to grab CSS selectors by clicking on the desired element in your browser)Code and full example in the online IDE:
Output:
Alternatively, you can use Google Knowledge Graph API from SerpApi, it's a paid API with a free plan.
The difference is that you don't need to create a parser from scratch, maintain it, or figure out how to bypass blocks from Google or other search engines.
Example code to integrate:
Full JSON Knowledge Graph output: