How to pass search string to Onesearch search engine with Python

372 Views Asked by At

I'm trying to retrieve the number of search results in multiple search engines and most of them accept my search string such as:

search_engine_url = "https://www.google.com/search?q=" + query + "&num=" + str(number_result)
search_engine_url = "https://www.bing.com/search?q=" + query + "&num=" + str(number_result)
search_engine_url = "https://au.search.yahoo.com/search?q=" + query + "&num=" + str(number_result)

However, when I pass on the same structure to onesearch:

search_engine_url = "https://www.onesearch.com/search?q=" + query

I don't get a proper result.

If I manually do a search in the search engine - for example, "medicine blood sugar", then the search string it produces looks like this:

https://www.onesearch.com/yhs/search;_ylt=AwrCxGGMcZJfaW0AdxzGnIlQ;_ylu=Y29sbwNiZjEEcG9zAzEEdnRpZAMEc2VjA3Fydw--?fr=yhs-ono-df&hsimp=yhs-df&hspart=ono&intl=us&ei=UTF-8&p=bpmCd4P8ioLi2Utwoqr03SXsAFCPzan7PchA19V1xcls5JfyiIr6dcI4epf%2FJJ5NiMNeZpWUeehFb4o680JW6IxaYKmqNhRrLtBltUGzULyKGCPCs3MPOAf0GuLqGRp%2B50Y4dd%2FRKPKYOuJGM5hzZRuACgcuFzq%2BevGv9ySwQz8%3D&fr2=12642&enct=p&encv=1_24&

How do I pass on the proper search string for other queries? If it's not possible, what other search engines show the number of search results on the first page? So far I've only found 'google', 'bing' and 'yahoo' that do that. Other search engines force you to use "next page" to see more results without indicating the total number of research results. I know the numbers are not real, but I'm only using it to decide if a search term belongs in one group or another group, so accuracy is not important - just the relative search result size.

Currently, I am not using search engine APIs for this.

I am using Python, but I think the language is irrelevant as it's about the search string.

            elif search_engine == 'onesearch':
                search_engine_url = "https://www.onesearch.com/search?q=" + query + "&num=" + str(number_result)

                try:
                    response = requests.get(search_engine_url, headers=rand_browser)  # requests.get(url, headers=headers).text
                except requests.exceptions.RequestException as e:  # NewConnectionError, MaxRetryError, ConnectionError
                    print(e)
                    search_engine = 'google'
                    continue

                soup = BeautifulSoup(response.text, "html.parser")
                divs = soup.find_all('span')  # <span>102,000,000 results</span>
    ```
2

There are 2 best solutions below

1
On

You can use a search string like the following:

https://www.onesearch.com/yhs/search;_ylt=AwrEeSVn6b1fWhwAmAfJnIlQ;_ylc=X0kDRTBBS3lURXdMaklxUGNJZ3o0QjhXd0F2TUM0d0xnQUFBQUNTdWlfZgRfUwMxMzUxMTk1ODQ5BF9yAzIEYWN0bgNjbGsEY3NyY3B2aWQDRTBBS3lURXdMaklxUGNJZ3o0QjhXd0F2TUM0d0xnQUFBQUNTdWlfZgRmcgN5aHMtb25vLWRmBGZyMgNzYi10b3AEZ3ByaWQDBG5fcnNsdAMwBG5fc3VnZwMwBG9yaWdpbgN3d3cub25lc2VhcmNoLmNvbQRwb3MDMARwcXN0cgMEcHFzdHJsAzAEc2VjA3NlYXJjaARzbGsDYnV0dG9uBHQyA3NlYXJjaAR0NANidXR0b24EdF9zdG1wAzE2MDYyODE1ODUEdnRlc3RpZAM-?hspart=ono&hsimp=yhsm-df&intl=us&ei=UTF-8&pvid=E0AKyTEwLjIqPcIgz4B8WwAvMC4wLgAAAACSui_f&gprid=&fr=yhs-ono-df&q=your%20query&fr2=sb-top

This search string is insanely long, but it works. Simply replace your%20query with the string you want to search for, where %20 replaces spaces between words. For example, the query shown above is searching for "your query".

0
On

This works for me and is a bit more concise ;)

https://www.onesearch.com/yhs/search?q=farside

I checked it on a few different browsers.