IIS random 500 errors with either rewrite or error passthrough mystery - permissions to isap_redirect.dll issue?

Question

We have a low traffic site that has been through staging and then moved into production and has been live for 30 days. Seemingly randomly 2 days ago the site started to give 500 errors. There are 6 production sites on this server and the site in question is the only site to give 500 errors. A simple restart in IIS of the site resolved the issue. IIS and ColdFusion logging logged nothing more than a 500 response and no errors. The only difference was some URL rewrite rules:

<system.webServer>
  <httpErrors existingResponse="PassThrough" />
  <rewrite>
      <rules>
        <rule name="Redirect .cfm extension" stopProcessing="false">
          <match url="^(.*).cfm$" ignoreCase="true" />
        <conditions logicalGrouping="MatchAny">
          <add input="{URL}" pattern="(.*).cfm$" ignoreCase="false" />
        </conditions>
          <action type="Redirect" url="{R:1}" redirectType="Permanent" />
        </rule>
        <rule name="hide .cfm extension" stopProcessing="true">
          <match url="^(.*)$" ignoreCase="true" />
        <conditions>
          <add input="{REQUEST_FILENAME}" matchType="IsFile" negate="true" />
          <add input="{REQUEST_FILENAME}" matchType="IsDirectory" negate="true" />
          <add input="{REQUEST_FILENAME}.cfm" matchType="IsFile" />
        </conditions>
          <action type="Rewrite" url="{R:0}.cfm" />
        </rule>
    </rules>
</rewrite>

All other rules were the same as on other sites on the same server. We enabled failed request tracing which seems to indicate a permissions error of some sort:

All sites are processed by the same isapi_redirect.dll. We checked uriworkermap.properties and workers.properties and they are the same compared to other production servers.

To eliminate the differences, we pulled out the httpErrors and the redirect and hide rules from web.config - and since then it's run fine with no 500s (knocking on stackwood).

We also spotted yesterday these errors in the isapi_redirect.log:

[Wed Jan 29 13:18:58.021 2020] [9056:9196] [error] uri_worker_map_update::jk_uri_worker_map.c (1299): Unable to stat the D:\ColdFusion2018\config\wsconfig\1\uriworkermap.properties (errno=2)
[Wed Jan 29 13:20:44.970 2020] [9056:9196] [error] uri_worker_map_update::jk_uri_worker_map.c (1299): Unable to stat the D:\ColdFusion2018\config\wsconfig\1\uriworkermap.properties (errno=2)

They also stopped when the web.config was changed. So the question is, after 2 1/2 hours with guru Charlie Arehart helping us with this and ending up with no solid answers, has anyone else seen anything similar? What else should we look at? There have been zero 500 errors since the change to web.config across all sites on the server.

Answer 1

For this kind of issue, the request tracing log information you provided doesn't have quite enough information. If we want to troubleshoot this kind of issue, we need to capture the dump of the application to see what has happened. I suggest you could generate a full dump for the process and use debugdialog analysis to analysis the dump file to find out the reason by using debug

You could download it from this url.

About how to capture dump, you could refer to this article. Notice:You should select the crash instead of performance.

About how to use debugdialog analysis to analysis the dump file ,you could refer to this article.