Searching google news articles Python

Question

im currently thinking about writing about a script which is looking out for new articles from google news search. So I want to download the first 10-20 articles and safe it into a folder. Best would be only the main text not the whole html file.

So as an example: Searchterm "FC Barcelona" Folder "01.01.2020" articles in here

Answer 1

Consider using the Google News RSS feed. It is already formatted in a easy to parse format. You can perform a search using the following format.

https://news.google.com/rss/search?{query} where query can be q=keywords so for your example searching for Searchterm "FC Barcelona". The query must be urlencoded. Which can be done with Python

import requests
from urllib.parse import urlencode
query = urlencode({'q': 'Searchterm "FC Barcelona"'})
url = "https://news.google.com/rss/search?" + query

# make requests
resp = requests.get(url)
# parse request

Then you can parse the feed as you liked and place the data into folders.