TYPO3: Bingbot creates an ext_form error which get cached

Question

We have a problem whit one of our TYPO3 installations. The Bingbot which visits the site calls a controller of the old ext_form extension without parameters and creates an error.

207.46.13.XXX - - [16/Oct/2018:00:18:48 +0200] "GET example.html?tx_form_form%5Baction%5D=process HTTP/1.1" 200 10256 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

The problem for us is not that this happens but that TYPO3 is caching the site with "Oops, an error occurred! Code: 20181016001848e0153dcf" as content.

Is there a way to say TYPO3 to not cache the site if an error occorres or to send the bot to the 404 site if he calls the site with false parameters.

Answer 1

There are several things you can do:

• exclude the page with the parameters in robots.txt (Edit: After consideration this solution is probably unsuitable for the specific problem)
• redirect in .htaccess if the page is called without required parameter, a redirect should be recognized by the bot too.
• check why the bot is even calling the page without required parameter, perhaps you can avoid it.
• bing can be configured for a specific URL, this page can be a starting point for you.

EDIT:
Example for htaccess (not tested):

RewriteCond %{QUERY_STRING} .+
RewriteRule ^example.html?tx_form_form%5Baction%5D=process$ example.html [L,R=301,QSD]

The target example.html can be anything, either a custom 404 page or just the list-view. The code 301 in [L,R=301,QSD] can be adjusted according to HTTP Status Codes (3xx). If it's redirected to a (custom) 4xx page the HTTP status code should be accordingly (404, or perhaps another 4xx-status-message i.e. 400).