I would like to check if a list of URL doesn't have a noindex.
So I check already
< meta name="robots" >
with DOM, but I would like to also check the X-robots-tag. Is there an easy way to do it?
The only way I can imagine is a foreach() on get_headers($url).