random.seed question (not reproducing same sequence)

[Nick Mellor]

Hi guys,

(Python 2.7, Windows 7 64-bit)

Here's a bit of code stress-testing a method addUpdate_special_to_cart. Thetest adds and updates random "specials" (multiple products bundled at an advantageous price) of various sizes to thousands of shopping carts, then restocks the whole darn lot. The test passes if the stock level afterwards isthe same as it was before executing the code for all products.

addUpdate_special_to_cart is working perfectly. But the test isn't.

The test iterates over the same code twice, once with special_qty==4, once with special_qty==0, reseeding the Python random module number generator to a fixed seed (a string) between the iterations. special_qty==0removes the special and restocks the products. The test relies on precisely the same random number sequence on both runs.

Can you think of a reason why the random number generator should fall out of sync between the two iterations? Because that's what's happening by the look of it: occasionally products are returned to the wrong stockbin. No "random" module method is used anywhere else while this code is executing.

When I assign something non-random to the stockbin parameter, the test passes.

Best wishes,



Nick

for qty in [4, 0]:
random.seed(seed)
for cart in range(test_size):
for special in range(randrange(3)):
s.addUpdate_special_to_cart(cart=cart, stockbin=randrange(test_size),
special_id=randrange(test_size), special_qty=qty,
products=[(random.choice(PRODUCTS), random.choice(range(10)))
for r in range(randrange(7))])

 

[John Gordon]

No "random" module method is used anywhere else while this code is
executing.

Are you sure? How did you verify this?

 

[Ned Batchelder]

Hi guys,

(Python 2.7, Windows 7 64-bit)

Here's a bit of code stress-testing a method addUpdate_special_to_cart. The test adds and updates random "specials" (multiple products bundled at an advantageous price) of various sizes to thousands of shopping carts, then restocks the whole darn lot. The test passes if the stock level afterwards is the same as it was before executing the code for all products.

addUpdate_special_to_cart is working perfectly. But the test isn't.

The test iterates over the same code twice, once with special_qty==4, once with special_qty==0, reseeding the Python random module number generator to a fixed seed (a string) between the iterations. special_qty==0 removes the special and restocks the products. The test relies on precisely the same random number sequence on both runs.

Can you think of a reason why the random number generator should fall out of sync between the two iterations? Because that's what's happening by the look of it: occasionally products are returned to the wrong stockbin. No "random" module method is used anywhere else while this code is executing.

When I assign something non-random to the stockbin parameter, the test passes.

Best wishes,



Nick

for qty in [4, 0]:
random.seed(seed)
for cart in range(test_size):
for special in range(randrange(3)):
s.addUpdate_special_to_cart(cart=cart, stockbin=randrange(test_size),
special_id=randrange(test_size), special_qty=qty,
products=[(random.choice(PRODUCTS), random.choice(range(10)))
for r in range(randrange(7))])

The best way to ensure repeatability of random numbers is to avoid the
module-level functions, and instead create your own random.Random()
instance to generate numbers. Then you can be certain it isn't being
used by anything else.

 

[Peter Otten]

Hi guys,

(Python 2.7, Windows 7 64-bit)

Here's a bit of code stress-testing a method addUpdate_special_to_cart.
The test adds and updates random "specials" (multiple products bundled at
an advantageous price) of various sizes to thousands of shopping carts,
then restocks the whole darn lot. The test passes if the stock level
afterwards is the same as it was before executing the code for all
products.

addUpdate_special_to_cart is working perfectly. But the test isn't.

The test iterates over the same code twice, once with special_qty==4, once
with special_qty==0, reseeding the Python random module number generator
to a fixed seed (a string) between the iterations. special_qty==0 removes
the special and restocks the products. The test relies on precisely the
same random number sequence on both runs.

Can you think of a reason why the random number generator should fall out
of sync between the two iterations? Because that's what's happening by the
look of it: occasionally products are returned to the wrong stockbin. No
"random" module method is used anywhere else while this code is executing.

When I assign something non-random to the stockbin parameter, the test
passes.

Best wishes,



Nick

for qty in [4, 0]:
random.seed(seed)
for cart in range(test_size):
for special in range(randrange(3)):
s.addUpdate_special_to_cart(cart=cart,
stockbin=randrange(test_size),
special_id=randrange(test_size),

special_qty=qty,

products=[(random.choice(PRODUCTS),random.choice(range(10)))
for r in

range(randrange(7))])

An exotic option: I notice that randrange() is the only random function not
qualified with the module name. If you are using a fancy auto-reloading web
framework things can get out of sync:

import random
from random import randrange
random.seed(42)
[randrange(10) for _ in range(5)] [6, 0, 2, 2, 7]
random.seed(42)
[randrange(10) for _ in range(5)] [6, 0, 2, 2, 7]
reload(random)

random.seed(42)
[randrange(10) for _ in range(5)]

[6, 8, 0, 4, 0]

 

[Dan Stromberg]

You could easily provide your own random number generator, if you
don't need cryptographic-strength random numbers.

EG:
http://stromberg.dnsalias.org/svn/lcgrng/trunk/lcgrng.py

That way you can be certain nothing else is using it.

 

[Ned Batchelder]

You could easily provide your own random number generator, if you
don't need cryptographic-strength random numbers.

EG:
http://stromberg.dnsalias.org/svn/lcgrng/trunk/lcgrng.py

That way you can be certain nothing else is using it.

There's no need to pull in a new implementation. The stdlib module is
perfectly willing to give you your own private random number sequence to
use.

 

[Nick Mellor]

Thanks John and others,

Replies much appreciated. I don't know how it could affect the results, butthe function being tested is using redis. And I am running the test code under PyCharm, so perhaps using the module-level random number generator wasn't such a good idea. Live and learn.

In response to your question, John, all I know is that my own code doesn't use the random module outside of this code fragment.

Ned, thanks for the tip about creating a new instance of Random(). The testfailures are still happening when the stockbins are randomised (as in codebelow.) That is suggesting that my code is somehow at fault.

Peter, I am using PyCharm as I said. But using a new Random() object to generate the sequence doesn't solve the problem apparently. The code now lookslike this:

rnd = random.Random()
...
for qty in [4, 0]:
rnd.seed(seed)
for cart in range(test_size):
for special in range(rnd.randrange(3)):
s.addUpdate_special_to_cart(cart=cart, stockbin=rnd.randrange(test_size),
special_id=rnd.randrange(test_size), special_qty=qty,
products=[(rnd.choice(PRODUCTS), rnd.choice(range(10)))
for r in range(rnd.randrange(7))])


Cheers,

Nick

 

[John Gordon]

In response to your question, John, all I know is that my own code doesn't
use the random module outside of this code fragment.

Does addUpdate_special_to_cart() use any random methods?

In any case, a further test would be to strip out all the business logic
and leave only the calls to random, and see if the results still differ.

Something like this:

rnd = random.Random()
for qty in [4, 0]:
print "qty is %s" % qty
rnd.seed(seed)
print "seed is %s" % seed
for cart in range(test_size):
print "cart is %s " % cart
for special in range(rnd.randrange(3)):
print "special is %s " % special
print "stockbin is %s" % rnd.randrange(test_size)
print "special_id is %s" % rnd.randrange(test_size)
print "products is %s" % [(rnd.choice(PRODUCTS), rnd.choice(range(10))) for r in range(rnd.randrange(7))])

Run that code sample and see if the results differ when qty is 4 vs 0.