Redirect Default Domain Access and Non-Existent Links to 404 Page with Specific Conditions

Question

I have a website, and by default, it runs in the cgi-bin directory. However, there seems to be an issue in my configuration. When I enter a directory or file that doesn't exist on the system, it redirects to /cgi-bin/index.cgi instead of redirecting to the 404.html page, as it should. Only when I enter a link like example.com/errorlink.pm does it redirect to the 404.html error page. This is the specific configuration for each section, including .htaccess and Apache2:

.htaccess:

AddDefaultCharset utf-8
RewriteEngine on

RewriteCond %{HTTP:Authorization} ^(.*)
RewriteRule ^(.*) - [E=HTTP_CGI_AUTHORIZATION:%1]

RewriteRule ^webdav /cgi-bin/webdav.cgi [L]
RewriteRule ^api/(\w+)/(\w+)$ /cgi-bin/xapi.cgi [L]
RewriteRule \.pm$ /404.html [L]

RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule .* /cgi-bin/index.cgi [L]

ErrorDocument 404 /404.html

<IfModule mod_php5.c>
        php_flag engine off
</IfModule>

Apache2 config:

<VirtualHost IP:8443>

    ServerName localhost
    
    ServerAdmin admin@localhost
    DocumentRoot /home/admin/web/domains/public_html
    ScriptAlias /cgi-bin/ /home/admin/web/domains/cgi-bin/
    Alias /vstats/ /home/admin/web/domains/stats/
    #SuexecUserGroup admin admin
    CustomLog /var/log/apache2/domains/domains.bytes bytes
    CustomLog /var/log/apache2/domains/domains.log combined
    ErrorLog /var/log/apache2/domains/domains.error.log
    <Directory /home/admin/web/domains/stats>
        AllowOverride All
    </Directory>
    <Directory /home/admin/web/domains/public_html>
        AllowOverride All
        Require all granted
        SSLRequireSSL
        Options +Includes -Indexes +ExecCGI
    </Directory>
    <Directory /home/admin/web/domains/cgi-bin>
        Options +ExecCGI
        AddHandler cgi-script .cgi .pl .py .rb
    </Directory>
    SSLEngine on
    SSLVerifyClient none
    SSLCertificateFile /home/admin/conf/web/domains/ssl/domains.crt
    SSLCertificateKeyFile /home/admin/conf/web/domains/ssl/domains.key
    SSLCertificateChainFile /home/admin/conf/web/domains/ssl/domains.ca

    <FilesMatch \.php$>
        SetHandler "proxy:unix:/run/php/php7.4-fpm-domains.sock|fcgi://localhost"
    </FilesMatch>
    SetEnvIf Authorization .+ HTTP_AUTHORIZATION=$0
    
    IncludeOptional /home/admin/conf/web/domains/apache2.ssl.conf_*
    IncludeOptional /etc/apache2/conf.d/*.inc
</VirtualHost>

I want to modify it as follows:

1. By default, when accessing https://example.com, it should point to /cgi-bin/index.cgi.
2. Redirect links that do not exist on the system to the 404 page (e.g., https://example.com/error or https://example.com/error.html). This should apply not only to https://example.com/.pm but also to other non-existent links.

Answer 1

When I enter a directory or file that doesn't exist on the system, it redirects to /cgi-bin/index.cgi instead of redirecting to the 404.html page, as it should.

Yes, that is because you have the following rule in your .htaccess file:

RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule .* /cgi-bin/index.cgi [L]

This is a "front-controller pattern" that internally rewrites all requests for non-files (which includes physical directories) to /cgi-bin/index.cgi. This naturally overrides the ErrorDocument 404 directive, so your /404.html ErrorDocument is not called.

If you only want to rewrite requests for the document root to /cgi-bin/index.cgi then modify the above rule like this (removing the RewriteCond directive entirely):

RewriteRule ^$ /cgi-bin/index.cgi [L]

Now, any request that does not map to a physical file (what you seem to be calling a "link") or a physical directory should be caught by the Apache ErrorDocument 404 directive.

Only when I enter a link like domain/errorlink.pm does it redirect to the 404.html error page.

Because you have a specific rule (RewriteRule \.pm$ /404.html [L]) that rewrites such requests. HOWEVER, this is not triggering a 404 HTTP response status, unless you have some additional server-side processing going on.