Just curious what the recommended best practice is for the 'million follower' twitter-style fanout problem on AppEngine.
Is it still Brett Slatkin's proposed solution (see: http://www.scribd.com/doc/16952419/Building-scalable-complex-apps-on-App-Engine) ?
Or could the new search API be used here? Could you do a search with a large number of author filters e.g. 'author:bob OR author:alice OR author:mike ... ' ad nauseum? Or are there limits on Search API query complexity? Or would it be horrible performance wise? Might it be a reasonable solution if there is a limit on the number of people one can follow?
Thanks for any feedback!
In general, 'or' queries aren't efficient in any database, and that includes the search API - they all require doing multiple independent queries, and gluing the results together.
The fanout problem can be handled much better by the prospective search API.