Handling remote: true links and searchbots

100 Views Asked by At

I use remote: true links a lot. In most cases, the response can be js or html, so searchbots don't cause a problem

But some links, are purely js response, and there is no HTML response (which causes exceptions and errors, which flood monitoring tools, email, alerts, etc)

How can I setup the link so searchbots (google,bing, other crawlers) won't try it?

Or - Is there a type of response I should return so the crawlers will know not to go back to that url?

1

There are 1 best solutions below

0
On

rel="nofollow" is what you can put on your links. that should prevent bots like google form crawling them https://support.google.com/webmasters/answer/96569?hl=en