lighttpd return 403 for python script that had been working on older installs

Question

I have a github for a kiln controller that runs on raspberry pi. I did not write the server setup. I did change it enough to run python 3 when that started being an issue. Wrote an install script and added a few things. Other than that, I've been running this for years.

I did a clean new install yesterday and for the life of me, I can't figure out why it's returning a 403. I set all the files and folders to 777 in a effort to figure it out. The lighttpd error.log is not returning any errors.

I still have this running on another raspberry pi. Python 3.9.2 and lighttpd 1.4.59

The new install is running Python 3.11.2 and lighttpd/1.4.69 The symlink for python3 is indeed set to python3.11 in /usr/bin

lighttpd.conf

server.modules = (
    "mod_access",
    "mod_alias",
    "mod_compress",
    "mod_redirect",
    "mod_cgi",
)

server.document-root        = "/home/pi/PILN"
server.upload-dirs          = ( "/var/cache/lighttpd/uploads" )
server.errorlog             = "/var/log/lighttpd/error.log"
server.pid-file             = "/var/run/lighttpd.pid"
server.username             = "www-data"
server.groupname            = "www-data"
server.port                 = 80


index-file.names            = ( "index.php", "index.html", "index.lighttpd.html","home.cgi","data.cgi" )
url.access-deny             = ( "~", ".inc" )
static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )

compress.cache-dir          = "/var/cache/lighttpd/compress/"
compress.filetype           = ( "application/javascript", "text/css", "text/html", "text/plain" )

# default listening port for IPv6 falls back to the IPv4 port 
include_shell "/usr/share/lighttpd/use-ipv6.pl " + server.port 
include_shell "/usr/share/lighttpd/create-mime.conf.pl" 
include "/etc/lighttpd/conf-enabled/*.conf"

$HTTP["url"] =~ "/app/" {
 cgi.assign = ( ".py" => "/usr/bin/python3",".cgi"=> "/usr/bin/python3" )
}

I've tried setting all the files and folders to 777 so not being able to execute shouldn't be a problem. Tried to reinstall with a different SD card. Other than that, nothing has changed but the software versions. I vaguely remember having some troubles with mime types at one point but don't remember what I did to fix it, and I'm pretty sure I've reinstalled since then without problems. I've tried googling to see if any if the recommended configurations have changed and tried a few iterations (very similar to what I already have) but nothing worked. The config listed is the same one that is currently working on another machine.

With this version of python I did have to change part of the install script from

sudo pip3 install jinja2
sudo pip3 install psutil
sudo pip3 install adafruit-circuitpython-max31856

to

sudo apt install python3-jinja2
sudo apt install python3-psutil
sudo pip3 install adafruit-circuitpython-max31856 --break-system-packages

Here is a link to the git. https://github.com/fayena/PILN/ The link I'm trying to load is http://IPAddress/app/home.cgi home.cgi calls a jinja2 template.

Any suggestions are appreciated.

EDIT: Installed the legacy version of Raspberry Pi OS Lite (Bullseye) and everything works. Now what? Any tips on how to troubleshoot?

Answer 1

Well, you changed many things with a full upgrade, not just the lighttpd version.

See https://wiki.lighttpd.net/DebugVariables and try debug.log-request-header-on-error = "enable"

You can strace lighttpd and look where a system call fails.

Mode 777 is almost never the right answer or the best answer.

If you print your lighttpd config on the working device running lighttpd 1.4.59, and print your lighttpd config on the new install running lighttpd 1.4.69, are there any differences?

lighttpd -f /etc/lighttpd/lighttpd.conf -p