I'm using the curl library to go to a site and get the results of a search bar on that site. It's all on the same site with no redirects, so I can't just do .com/search?=... The search bar is defined as:
<input type="search" id="search" name="search" placeholder="Search for a series" wire:model.debounce.600ms="query" class="block w-full rounded border border-transparent bg-neutral-800 py-2 pl-10 pr-3 leading-5 text-neutral-300 placeholder-neutral-400 focus:border-neutral-600 focus:bg-neutral-600 focus:text-white focus:outline-none focus:ring-neutral-900 sm:text-sm">
with two identical listeners that are defined as: imgListeners
Whenever an input is given, these elements appear below it with the search results being further inside them: imgAppear
But no matter what I do, I can't get the search to work (getting the website itself works fine). I always get an error back saying the method is not allowed.
<hr>
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="robots" content="noindex,nofollow,noarchive" />
<title>An Error Occurred: Method Not Allowed</title>
<link rel="icon" href="data:image/svg+xml,<svg xmlns=%22http://www.w3.org/2000/svg%22 viewBox=%220 0 128 128%22><text y=%221.2em%22 font-size=%2296%22>❌</text></svg>">
<style>body { background-color: #fff; color: #222; font: 16px/1.5 -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif; margin: 0; }
.container { margin: 30px; max-width: 600px; }
h1 { color: #dc3545; font-size: 24px; }
h2 { font-size: 18px; }</style>
</head>
<body>
<div class="container">
<h1>Oops! An Error Occurred</h1>
<h2>The server returned a "405 Method Not Allowed".</h2>
<p>
Something is broken. Please let us know what you were doing when this error occurred.
We will fix it as soon as possible. Sorry for any inconvenience caused.
</p>
</div>
<script>(function(){var js = "window['__CF$cv$params']={r:'83bc5ac5bd5b5aec',t:'MTcwMzYyNTQ4OC41NzEwMDA='};_cpo=document.createElement('script');_cpo.nonce='',_cpo.src='/cdn-cgi/challenge-platform/scripts/jsd/main.js',document.getElementsByTagName('head')[0].appendChild(_cpo);";var _0xh = document.createElement('iframe');_0xh.height = 1;_0xh.width = 1;_0xh.style.position = 'absolute';_0xh.style.top = 0;_0xh.style.left = 0;_0xh.style.border = 'none';_0xh.style.visibility = 'hidden';document.body.appendChild(_0xh);function handler() {var _0xi = _0xh.contentDocument || _0xh.contentWindow.document;if (_0xi) {var _0xj = _0xi.createElement('script');_0xj.innerHTML = js;_0xi.getElementsByTagName('head')[0].appendChild(_0xj);}}if (document.readyState !== 'loading') {handler();} else if (window.addEventListener) {document.addEventListener('DOMContentLoaded', handler);} else {var prev = document.onreadystatechange || function () {};document.onreadystatechange = function (e) {prev(e);if (document.readyState !== 'loading') {document.onreadystatechange = prev;handler();}};}})();</script><script defer src="https://static.cloudflareinsights.com/beacon.min.js/v84a3a4012de94ce1a686ba8c167c359c1696973893317" integrity="sha512-euoFGowhlaLqXsPWQ48qSkBSCFs3DPRyiwVu3FjR96cMPx+Fr+gpWRhIafcHwqwCqWS42RZhIudOvEI+Ckf6MA==" data-cf-beacon='{"rayId":"83bc5ac5bd5b5aec","version":"2023.10.0","token":"2b8d18b8cd694f8abcc90dc2b359c7f7"}' crossorigin="anonymous"></script>
</body>
</html>
I just want to get the URLs of the search results and throw them in a vector- I'm open to changing libraries if there's something better.
Here's my current attempt:
size_t WriteCallback(void* contents, size_t size, size_t nmemb, std::string* output) {
size_t totalSize = size * nmemb;
output->append((char*)contents, totalSize);
return totalSize;
}
int main() {
curl_global_init(CURL_GLOBAL_DEFAULT);
CURL* curl = curl_easy_init();
std::string searchQuery = "sss";
std::string postData = "search=" + searchQuery;
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, postData.c_str());
std::string responseData;
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteCallback);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &responseData);
curl_easy_setopt(curl, CURLOPT_USERAGENT, "simple scraper");
curl_easy_setopt(curl, CURLOPT_URL, URL.c_str());
CURLcode res = curl_easy_perform(curl);
curl_easy_cleanup(curl);
curl_global_cleanup();
std::cout << "Response:\n" << responseData << std::endl;
return 0;
}
I also tried a suggested way from the duplicate questions prompt, but that didn't work either:
int main() {
CURL *curl;
curl_global_init(CURL_GLOBAL_ALL);
curl = curl_easy_init();
curl_easy_setopt(curl, CURLOPT_URL, URL.c_str());
curl_easy_setopt(curl, CURLOPT_POST, 1);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, "search=sss");
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteCallback);
curl_easy_setopt(curl, CURLOPT_USERAGENT, "simple scraper");
std::string data = "none";
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &data);
curl_easy_perform(curl);
std::cout << data;
curl_easy_cleanup(curl);
return 0;
}
You haven't show the form and the submit button. From the error, I can only guess it must be a GET request: