I've implemented a BloomFilter in python 3.3, and got different results every session. Drilling down this weird behavior got me to the internal hash() function - it returns different hash values for the same string every session.
Example:
>>> hash("235")
-310569535015251310
----- opening a new python console -----
>>> hash("235")
-1900164331622581997
Why is this happening? Why is this useful?
Python uses a random hash seed to prevent attackers from tar-pitting your application by sending you keys designed to collide. See the original vulnerability disclosure. By offsetting the hash with a random seed (set once at startup) attackers can no longer predict what keys will collide.
You can set a fixed seed or disable the feature by setting the
PYTHONHASHSEED
environment variable; the default israndom
but you can set it to a fixed positive integer value, with0
disabling the feature altogether.Python versions 2.7 and 3.2 have the feature disabled by default (use the
-R
switch or setPYTHONHASHSEED=random
to enable it); it is enabled by default in Python 3.3 and up.If you were relying on the order of keys in a Python set, then don't. Python uses a hash table to implement these types and their order depends on the insertion and deletion history as well as the random hash seed. Note that in Python 3.5 and older, this applies to dictionaries, too.
Also see the
object.__hash__()
special method documentation:If you need a stable hash implementation, you probably want to look at the
hashlib
module; this implements cryptographic hash functions. The pybloom project uses this approach.Since the offset consists of a prefix and a suffix (start value and final XORed value, respectively) you cannot just store the offset, unfortunately. On the plus side, this does mean that attackers cannot easily determine the offset with timing attacks either.