I am creating a spider for one of my websites and I need a way to determine what the index file of a website is. I understand there are a lot of different file names. I am not worried about those who set custom filenames. Below is a list of what I have found so far. Does anyone have a complete list?
index.html
index.htm
index.shtml
index.php
index.php5
index.php4
index.php3
index.cgi
default.html
default.htm
home.html
home.htm
Index.html
Index.htm
Index.shtml
Index.php
Index.cgi
Default.html
Default.htm
Home.html
Home.htm
placeholder.html
default.asp
Default.asp
index.asp
Index.asp