Are .pyc files portable?

[Roy Smith]

I'm using Python as part of a test fixture for a large (mostly C++)
software project. We build on a lot of different platforms, but
Solaris is a special case -- we build on Solaris 8, and then run our
test suite on Solaris 8, 9, and 10. The way the build system is set
up (driven by Build Forge), the Sol 8, 9, and 10 test passes can
happen in any order. The tests all run in a common directory, shared
by NFS on all three machines.

If (for example) the Sol 10 tests run first, and create .pyc files,
will those .pyc files work correctly when the Sol 8 and Sol 9 test
passes come along and run them in the same directories?

The python binary is a build of 2.5.1 that we did on Solaris 8. The
same binary is used on all Solaris platforms. Although we do separate
32-bit and 64-bit builds of our product, the python binary we use for
the test driver is the same 32-bit build on all of them.

A related question: is there any way to tell Python to put the .pyc
files in some other directory, i.e. NOT the same directory where
the .py sources are. That would make things cleaner, since we could
just tell each system to put its .pyc's in a different place.

We've also got some games we play with symlink farms (lndir), which
may be part of the solution here, but I want to explore all the
possibilities first.

 

[Wubbulous]

Python compiles to bytecode, which means that pyc files can be
interpreted by any Python executable regardless of platform.

As for manual compilation to directories, the py_compile module is the
one you want. Here's an example program to plug in.

#test_compile.py#

import py_compile

print "hello"
py_compile.compile("test_compile.py", "C:\test_compile.pyc")


The manual page for py_compile.compile() can be found on this page:
http://www.python.org/doc/2.5.2/lib/module-pycompile.html

Hope that helps.
Cheers,
Wubbulous

 

[Steven D'Aprano]

Python compiles to bytecode, which means that pyc files can be
interpreted by any Python executable regardless of platform.

No, bytecode isn't compatible from one version number to another.

 

[John Machin]

I'm using Python as part of a test fixture for a large (mostly C++)
software project.  We build on a lot of different platforms, but
Solaris is a special case -- we build on Solaris 8, and then run our
test suite on Solaris 8, 9, and 10.  The way the build system is set
up (driven by Build Forge), the Sol 8, 9, and 10 test passes can
happen in any order.  The tests all run in a common directory, shared
by NFS on all three machines.

If (for example) the Sol 10 tests run first, and create .pyc files,
will those .pyc files work correctly when the Sol 8 and Sol 9 test
passes come along and run them in the same directories?

In general, if the .pyc file is not compatible with the Python binary,
it will be flicked and replaced by compiling the corresponding .py
file if it exists; otherwise you get:
ImportError: Bad magic number in whatever.pyc

However you appear to be running the same Python binary with different
versions of your OS, so you shouldn't have any problems.

 

[Wubbulous]

Yes, apologies, I overlooked that detail. If using a different version
of the binary, (i.e. 3.0 vs 2.6) you will have to re-compile the
source code.

 

[Lie Ryan]

I'm using Python as part of a test fixture for a large (mostly C++)
software project. We build on a lot of different platforms, but Solaris
is a special case -- we build on Solaris 8, and then run our test suite
on Solaris 8, 9, and 10. The way the build system is set up (driven by
Build Forge), the Sol 8, 9, and 10 test passes can happen in any order.
The tests all run in a common directory, shared by NFS on all three
machines.

If (for example) the Sol 10 tests run first, and create .pyc files, will
those .pyc files work correctly when the Sol 8 and Sol 9 test passes
come along and run them in the same directories?

The python binary is a build of 2.5.1 that we did on Solaris 8. The
same binary is used on all Solaris platforms. Although we do separate
32-bit and 64-bit builds of our product, the python binary we use for
the test driver is the same 32-bit build on all of them.

A related question: is there any way to tell Python to put the .pyc
files in some other directory, i.e. NOT the same directory where the .py
sources are. That would make things cleaner, since we could just tell
each system to put its .pyc's in a different place.

We've also got some games we play with symlink farms (lndir), which may
be part of the solution here, but I want to explore all the
possibilities first.

If all that responses confuse you, the simple answer is:
1. Python bytecode is cross-platform
2. but is not cross-version

i.e. as long as the .pyc can be run accross different platforms as long
as the version number of the compiling python and executing python is
equal.

A simple solution if you've got different version of python across many
platform is to provide a .py file that redirects the program to each
version-specific .pyc (cross-platform and cross-version if written with
some care)