I want to sort objects after performance in a non-deterministic environment (similar to a chess tournaments though no ties are possible), where the comparisons take quite a lot of effort and thus I need to approximate the order with few comparisons. Comparisons can run in parallel and the amount of objects is variable but generally less than 20.
I've tried all the tournament- and rating systems on Wikipedia, but they all either lack redundancy, take too much time or only determine the best "player". I've also thought about heuristics but there aren't any good ones, since I don't want to introduce biases.
Due to these failures I've taken the more unconventional route of using sorting algorithms as the backbone of the ranking and came to odd-even tables as pretty much ideal for my intentions with them being able to be parallelized and still having few comparisons to make. But in those there is a problem, that is most obvious, when looking at a table of size (2^n)+1, where just a single lucky comparison can land the "+1"-object at the first position - similar things obviously still occur in tables of other sizes.
To introduce redundancy I've thought about making each comparison a best of 3 or running the test twice (in different order), further evaluating every result that changed over both tests, but both of those methods require too many comparisons. From here on I am clueless.