I would like to check if a list of URL doesn't have a noindex
.
So I check already
< meta name="robots" >
with DOM, but I would like to also check the X-robots-tag. Is there an easy way to do it?
The only way I can imagine is a foreach()
on get_headers($url)
.