I'm trying to retrieve the number of search results in multiple search engines and most of them accept my search string such as:
search_engine_url = "https://www.google.com/search?q=" + query + "&num=" + str(number_result)
search_engine_url = "https://www.bing.com/search?q=" + query + "&num=" + str(number_result)
search_engine_url = "https://au.search.yahoo.com/search?q=" + query + "&num=" + str(number_result)
However, when I pass on the same structure to onesearch:
search_engine_url = "https://www.onesearch.com/search?q=" + query
I don't get a proper result.
If I manually do a search in the search engine - for example, "medicine blood sugar", then the search string it produces looks like this:
https://www.onesearch.com/yhs/search;_ylt=AwrCxGGMcZJfaW0AdxzGnIlQ;_ylu=Y29sbwNiZjEEcG9zAzEEdnRpZAMEc2VjA3Fydw--?fr=yhs-ono-df&hsimp=yhs-df&hspart=ono&intl=us&ei=UTF-8&p=bpmCd4P8ioLi2Utwoqr03SXsAFCPzan7PchA19V1xcls5JfyiIr6dcI4epf%2FJJ5NiMNeZpWUeehFb4o680JW6IxaYKmqNhRrLtBltUGzULyKGCPCs3MPOAf0GuLqGRp%2B50Y4dd%2FRKPKYOuJGM5hzZRuACgcuFzq%2BevGv9ySwQz8%3D&fr2=12642&enct=p&encv=1_24&
How do I pass on the proper search string for other queries? If it's not possible, what other search engines show the number of search results on the first page? So far I've only found 'google', 'bing' and 'yahoo' that do that. Other search engines force you to use "next page" to see more results without indicating the total number of research results. I know the numbers are not real, but I'm only using it to decide if a search term belongs in one group or another group, so accuracy is not important - just the relative search result size.
Currently, I am not using search engine APIs for this.
I am using Python, but I think the language is irrelevant as it's about the search string.
elif search_engine == 'onesearch':
search_engine_url = "https://www.onesearch.com/search?q=" + query + "&num=" + str(number_result)
try:
response = requests.get(search_engine_url, headers=rand_browser) # requests.get(url, headers=headers).text
except requests.exceptions.RequestException as e: # NewConnectionError, MaxRetryError, ConnectionError
print(e)
search_engine = 'google'
continue
soup = BeautifulSoup(response.text, "html.parser")
divs = soup.find_all('span') # <span>102,000,000 results</span>
```
You can use a search string like the following:
This search string is insanely long, but it works. Simply replace
your%20query
with the string you want to search for, where%20
replaces spaces between words. For example, the query shown above is searching for "your query".