I recently implemented an autocomplete/autosuggest feature driven by solr. My solr implementation is using the rails sunspot gems. The autocomplete search does not go through rails at all, but goes through an apache reverse-proxy to the solr server directly. So, in this case, the role of sunspot is to re-index when a record is created or updated (or deleted).
My problem is that when a record is created or saved, triggering a delayed_job reindex, it is not reflected in the autocomplete search in production only. When a record is saved in development or staging, the changes are immediately reflected in the autocomplete search.
My code and my solr configs are all managed in git, so they are identical in all my environments (development, staging, and production). I have not done any production specific configuration changes (yet!). I have copied the MySQL database and solr index files from production back to the other environments as well, to make everything the same. I've restarted but rails and solr many times to be sure to be sure.
In production, I see the jobs go through delayed_job, and I see the update in the sunspot solr logs. Although, most of the log output is still greek to me. I've done a line-by-line comparison between production and development and they look more or less the same.
I've tested directly against the solr server using curl to make sure it's just me and solr - no apache, no networks, no browsers.
When that same record is saved, it initiates jobs to reindex a bunch of its child objects, and those do get reindexed as expected.
Also, if I reindex using the rake task for that model, it rebuilds the index correctly. That only takes a few minutes, so that is what I'm reduced to doing now.
So, the problem is this one Type, in this one environment, on create/update only.
The only difference I can identify is the load, although my site has so few users it's hard to think it matters.
Any idea what could be causing this, or what I should look at next to figure it out? My guess would be caching, but since I have not done any kind of production tuning, it should be the same. Unless sunspot does something that I'm not aware of.