Assets links inside css are messed up in heroku staging app but not in review apps that use the same config

Question

I have a rails app deployed to heroku that is deployed to a subdirectory in the following way.

In config.ru, we have

map '/diagnostics' do
  run Rails.application
end

And in config/application.rb

config.action_controller.relative_url_root = '/diagnostics'

Now, this setup works just fine with the assets in development mode and in heroku Review apps which uses the staging.rb configuration but things don't work the same way with the same setup for the staging app.

The issue

The assets link inside the css files are messed up.

In the staging application, say we are loading a font, the src is missing the /asset prefix

But in the review apps for heroku with the same configuration as in the above application, the links are fine.

This has me stumped considering I am not sure if its a problem with my configuration or heroku.

Ruby version: 2.7.2 Rails version: 6.1.4