How can I package a python script and modules into a single script?

[Noah]

I would like to package my main script and all the
modules it imports into a single script that will
run just as the collection would. It should not
need to package standard Python lib modules -- just
my local modules. This is not the same question
that would be answered with py2exe or py2app. I
don't need to package the script into a binary
along with Python. I expect Python to be installed.
I also don't want to use distutils to install the
script. The script will always run on Unix.

I thought that there might be some way to run a
script package from a tar file. One way would be to
have the package script untar itself into a temp
directory; run the main script; then delete the
temporary package directory when done. That sounds
clunky and prone to leave around trash if someone
does a 'kill -9'. However, this might be an
acceptable compromise. I'm sure that I've seen a
tool around like this, but I can't find it anymore.
I checked the usual places (google and sf.net).

I also considered simply cutting and pasting all
the modules I need into one single, giant script,
but that's less appealing than the tarball idea
(unless it could be done automatically).

Yours,
Noah

 

[Thomas Heller]

I would like to package my main script and all the
modules it imports into a single script that will
run just as the collection would. It should not
need to package standard Python lib modules -- just
my local modules. This is not the same question
that would be answered with py2exe or py2app. I
don't need to package the script into a binary
along with Python. I expect Python to be installed.
I also don't want to use distutils to install the
script. The script will always run on Unix.

I thought that there might be some way to run a
script package from a tar file. One way would be to
have the package script untar itself into a temp
directory; run the main script; then delete the
temporary package directory when done. That sounds
clunky and prone to leave around trash if someone
does a 'kill -9'. However, this might be an
acceptable compromise. I'm sure that I've seen a
tool around like this, but I can't find it anymore.

Wasn't this named 'squeeze'?

Thomas

 

[Alex Martelli]

I would like to package my main script and all the
modules it imports into a single script that will
run just as the collection would. It should not
need to package standard Python lib modules -- just
my local modules. This is not the same question
that would be answered with py2exe or py2app. I

Make a zipfile with the .pyc of all the modules you want; put a small
text header right before it to import the mainscript and run its main
function; that's it, thanks to Python's ability to import from zipfiles.

There's a recipe for that in the Python Cookbook (2nd edition).

(I believe py2app doesn't need to package the standard library, either,
since it comes with MacOSX 10.3 and later, but I haven't checked).


Alex

 

[utabintarbo]

I believe you are thinking of "freeze", which is part of the normal
python distro.

There is also cx_Freeze (
http://starship.python.net/crew/atuining/cx_Freeze/ )....

Bob

 

[Noah]

Freeze also packages the python interpreter into a binary.
I need a cross platform solution that just packages the scripts.
I expect the user to already have python installed.

Yours,
Noah

 

[Noah]

Bingo! That is what I was thinking of.
http://effbot.org/zone/squeeze.htm
It turns out that it doesn't quite do what I want
because the resulting script is tightly coupled
to the version of Python used to build the package.
It compiles the PYC byte-code into the package.
It's neat and I may be able to use the ideas to do
what I want.

Yours,
Noah

 

[Brett Hoerner]

Would PythonEggs be close?

It's basically a zip format with all the Python files inside, it would
maintain the structure and not be a single script file, but would be a
single .egg file... the end user would just run the script and all
would be installed, essentially.

http://peak.telecommunity.com/DevCenter/PythonEggs

 

[Noah]

This is interesting, but requires two separate files -- the ZIP file
and the boot script.
This is because zipimport can only import from file paths.
It can't import from a string or an IOString (as far as I can tell).
If only they would let you import from an already open file. Damn!

Squeeze is 99% of what I want except for the fact that it precompiles
the py files to bytecode
which locks you to the version of python you compiled on. Python byte
code is not guaranteed
to be portable across different versions of Python. Squeeze relies on
ihooks, which is
a standard module, but it is undocumented.

Anders Hammarquist has a Python Cookbook example that seems to do what
I want.
"Importing a Dynamically Generated Module". So far so good. I have to
check it out a bit more.
It relies on the "imp" standard library module.

With a bit of inspiration from "Squeeze" and help from the "imp" module
I might have what I want.

Yours,
Noah

 

[Alex Martelli]

This is interesting, but requires two separate files -- the ZIP file
and the boot script.

No, it doesn't.

This is because zipimport can only import from file paths.

It can import from a file, and the file (like all zipfiles) can have a
prefix. That prefix is where you put the "boot" script.

See http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/215301
(though I believe I did some minor enhancements, as well as adding
useful discussion, in the 2nd edition of the O'Reilly printed version of
the cookbook, Raedler's basic idea is presented here; you can get all
the sources as they appear in the printed version as an archive, a
zipfile I believe, on the O'Reilly site for the book).

Anders Hammarquist has a Python Cookbook example that seems to do what
I want.
"Importing a Dynamically Generated Module". So far so good. I have to
check it out a bit more.

I know Anders (known to his friends as "Iko") quite well, worked at his
side in Sweden in past years, and I selected his recipe for the first
edition of the O'Reilly printed version of the Cookbook -- but it does
not address the specific problem you desire. Raedler's recipe does.


Alex

 

[Noah]

It can import from a file, and the file (like all zipfiles) can have a prefix
That prefix is where you put the "boot" script.
See http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/215301

Unfortunately, Raedler's boot script closes stdin, which is a fairly
big limitation.
The herefile in the shell script (END_OF_PYTHON_CODE) redirects stdin
before
python starts; stdin is closed at the end of the herefile. This
disables raw_input()
and anything else that reads sys.stdin. Enter the following at a shell
prompt
to demonstrate the problem:
python - << HEREFILE

> print raw_input("Enter something")
> HEREFILE

You will get the following error:
Enter somethingTraceback (most recent call last):
File "<stdin>", line 1, in ?
EOFError: EOF when reading a line

But all is not lost. The "zipheader.unix" script can be rewritten to
use the '-c' option of python
instead of stdin and a herefile. This also handles the case where
Python may be even less
than version 2.0. This works the same. Just "cat zipheader.unix
main.zip > main" then run "main".

---- 8< ---- 8< ---- save as zipheader.unix ---- 8< ---- 8<
--------------
#!/bin/sh
# This is a self-extracting executable.
# Execute this like any normal executable.
# You may need to "chmod a+x" this file.
# This is a binary ZIP file with a Python loader header.
#
# Bourne shell loader:
PYTHON=$(which python 2>/dev/null)
if [ ! -x "$PYTHON" ] ; then
echo "Python not found!"
exit 1
fi
exec $PYTHON -c "
# Python loader:
import sys, os
if int(sys.version[0])<2:
print 'Python version 2.3 final or greater is required.'
print 'Your version is', sys.version
os._exit(1)
major = sys.version_info[0]
minor = sys.version_info[1]
releaselevel = sys.version_info[3]
if (major==2 and minor<3) or (major==2 and minor==3 and
releaselevel!='final'):
print 'Python version 2.3 final or greater is required.'
print 'Your version is', sys.version
os._exit(1)
sys.path.insert(0, sys.argv[1])
del sys.argv[0:1]
print sys.argv[1]
import main
main.main()
" $0 $@

# Zip file:
---- end of zipheader.unix

 

[Fuzzyman]

You could use my includer script.

http://www.voidspace.org.uk/python/recipebook.shtml#includer

It effectively adds an include direct to python scripts.

##include module.py
from module import *

You then run ``includer.py infilename outfilename``

This replaces the ``##include ..`` with the source of the included
module and *removes* the import statement.
This makes it possible to maintain modules separately, but distribute
as a single script.

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml