Non-deterministic output

[Esben Nielsen]

Hi,

We are making a prototype program in Python. I discovered the output was
non-deterministic, i.e. I rerun the program on the same input files and
get different output files. We do not use any random calls, nor
threading.

One of us thought it could be set and dictionaries not always yielding
the same results. I, however, would think that given the exact same
operations, a set/dictionary would always yield the same results. Am I
correct? Or could different runs of the same program yield different
results due to, say, different memory locations?

Are there any other sources of randomness we ought to look out for?

Esben

 

[Laurent Claessens]

One of us thought it could be set and dictionaries not always yielding
the same results. I, however, would think that given the exact same
operations, a set/dictionary would always yield the same results. Am I
correct? Or could different runs of the same program yield different
results due to, say, different memory locations?

If you have

d={"a":1,"b":2}
print d.keys()

then there are no way to be deterministic.

What you can do is

d={"a":1,"b":2}
k = d.keys()
k.sort()
print k

This is deterministic.

An other source of non-deterministic behaviour is to rely on the 15th
decimal of a difficult computation.

I had the following with Sage[1] :

sage: p=plot(x**2,-1,1)
sage: p.get_minmax_data()
{'xmin': -1.0, 'ymin': 3.0448943892610684e-06, 'ymax': 1.0, 'xmax': 1.0}

and in a new Sage shell :

sage: p=plot(x**2,-1,1)
sage: p.get_minmax_data()
{'xmin': -1.0, 'ymin': 1.2545281288559271e-05, 'ymax': 1.0, 'xmax': 1.0}

Hope it helps
Laurent


[1] www.sagemath.org

 

[Peter Otten]

Hi,

We are making a prototype program in Python. I discovered the output was
non-deterministic, i.e. I rerun the program on the same input files and
get different output files. We do not use any random calls, nor
threading.

One of us thought it could be set and dictionaries not always yielding
the same results. I, however, would think that given the exact same
operations, a set/dictionary would always yield the same results. Am I
correct? Or could different runs of the same program yield different
results due to, say, different memory locations?

If you insert items into a dict/set in the exact same order and the hash
values of the keys are the same across different runs of the program the
order of the dict/set items should be the same.

One way you could inadvertently bring the memory location into play is the
default __hash__() method:

$ cat print_set.py
names = "alpha", "beta", "gamma", "delta", "epsilon"
class A(object):
def __init__(self, name):
self.name = name
def __repr__(self):
return self.name

print set(A(name) for name in names)
$ python print_set.py
set([alpha, beta, delta, epsilon, gamma])
$ python print_set.py
set([alpha, beta, delta, gamma, epsilon])

 

[John Nagle]

Hi,

We are making a prototype program in Python. I discovered the output was
non-deterministic, i.e. I rerun the program on the same input files and
get different output files. We do not use any random calls, nor
threading.

One of us thought it could be set and dictionaries not always yielding
the same results. I, however, would think that given the exact same
operations, a set/dictionary would always yield the same results. Am I
correct? Or could different runs of the same program yield different
results due to, say, different memory locations?

Are there any other sources of randomness we ought to look out for?

Esben

That's worth chasing down. It's hard to accidentally get
nondeterminism in a single-thread batch Python program.
Core Python doesn't have uninitialized variables, the usual
source of trouble, because Python variables are created by
initialization, not declaration.

numpy, though, does have uninitialized variables. Try:

import numpy
a = numpy.empty(10)
print(a)

creates an array of 10 junk floating point numbers. Look for that.
Also look at any other C packages you're using. If you're stil
stuck, give us the list of non-Python modules you're importing,
directly or indirectly.

John Nagle