Python 3.3 and .pyo files

[Marco]

I was trying to import a pyo module in Python 3.3, but Python does not
find it:

$ echo "print(__file__)" > foo.py
$ python3.3 -O -m foo
/home/marco/temp/foo.py
$ ls
foo.py __pycache__
$ rm foo.py
$ mv __pycache__/foo.cpython-33.pyo foo.pyo
$ rm __pycache__ -r
$ ls
foo.pyo
# The following works in Python3.2, but not in 3.3
$ python3.3 -O -m foo
/usr/local/bin/python3.3: No module named foo

How come? Thanks in advance, Marco

 

[Steven D'Aprano]

I was trying to import a pyo module in Python 3.3, but Python does not
find it:

$ echo "print(__file__)" > foo.py
$ python3.3 -O -m foo
/home/marco/temp/foo.py
$ ls
foo.py __pycache__
$ rm foo.py
$ mv __pycache__/foo.cpython-33.pyo foo.pyo

I cannot duplicate the creation of the foo.cpython-33.pyo file using just
the -m option. I believe that you created the foo*.pyo file some other
way. Nevertheless, moving along:

$ rm __pycache__ -r
$ ls
foo.pyo
# The following works in Python3.2, but not in 3.3
$ python3.3 -O -m foo
/usr/local/bin/python3.3: No module named foo

I can confirm that (1) it works using Python 3.2; (2) it doesn't work
using Python 3.3; and (3) it does work in Python 3.3 if you don't use the
-O option.

I believe that is a bug.

(Tested using Python 3.2.2 and Python 3.3.0a1)

 

[Marco]

I can confirm that (1) it works using Python 3.2; (2) it doesn't work
using Python 3.3; and (3) it does work in Python 3.3 if you don't use the
-O option.

It doesn't work with Python 3.3.0rc2 too.

 

[Terry Reedy]

I was trying to import a pyo module in Python 3.3, but Python does not
find it:

You appear to be trying to *run*, not *import* a .pyo module.

$ echo "print(__file__)" > foo.py
$ python3.3 -O -m foo

Since foo.py is in the current directory, I am not sure why you use '-m
foo' instead of 'foo.py'. -m is for running a module somewhere on sys.path.

/home/marco/temp/foo.py
$ ls
foo.py __pycache__
$ rm foo.py
$ mv __pycache__/foo.cpython-33.pyo foo.pyo
$ rm __pycache__ -r
$ ls
foo.pyo
# The following works in Python3.2, but not in 3.3
$ python3.3 -O -m foo

I would try just 'foo.pyo' in case the -m part is the problem.
Also, the -O is sort of redundant, or perhaps interfering, since its
usual effect to to say 'get and put, from and to the cache, .pyo instead
of .pyc'.

/usr/local/bin/python3.3: No module named foo

How come? Thanks in advance, Marco

You might read some of http://bugs.python.org/issue12982

in particular, from http://bugs.python.org/issue12982#msg162814

"Indeed, since I posted last night, the pydev discussion has moved to
the question of whether -O, __debug__, and .pyo as now defined are worth
the nuisance they cause or whether some or all should be deprecated.
(Docstring stripping for saving space could then be a separate tool.)
---
Python interpreters exist to run Python code. The existence,
persistence, and other details of compilation caches are
version-dependent implementation details. Being able to execute from
such caches without source present is also an implementation detail, and
for CPython, it gets secondary support at best. (This is a compromise
between full support and no support.)"

 

[Steven D'Aprano]

You appear to be trying to *run*, not *import* a .pyo module.

Marco is using the standard mechanism for finding, importing, and running
a module. I don't believe his use of -m should be a problem. It works in
3.2, and it works with .pyc files in 3.3, I see nothing to suggest it
shouldn't work with .pyo files in 3.3.

Since foo.py is in the current directory, I am not sure why you use '-m
foo' instead of 'foo.py'. -m is for running a module somewhere on
sys.path.

Yes, and the current directory is on sys.path.

I would be astonished if python -m could not find a module that happened
to be in the current directory.


[...]

Also, the
-O is sort of redundant, or perhaps interfering, since its usual effect
to to say 'get and put, from and to the cache, .pyo instead of .pyc'.

No it is not redundant. You link specifically to an bug tracker issue
below where is is clearly decided that if you want to run a .pyo file you
*must* use the -O switch. (I approve of this decision.)

You might read some of http://bugs.python.org/issue12982

in particular, from http://bugs.python.org/issue12982#msg162814

Whose words are these following?

Python interpreters exist to run Python code. The existence,
persistence, and other details of compilation caches are
version-dependent implementation details. Being able to execute from
such caches without source present is also an implementation detail, and
for CPython, it gets secondary support at best. (This is a compromise
between full support and no support.)"

I'm not sure if these are your words, or if you are quoting some random
commenter on the pydev list, or one of the lead developers who might
actually know what he is talking about.

As I recall, some time in the recent past Guido came down *hard* against
the suggestion that support for running sourceless files (.pyc and .pyo)
should be dropped. Even if I'm misremembering, it is the case that the
Python 3.3 will find and run a .pyc file, but not a .pyo file. There's a
new candidate release of 3.3 due out over the next couple of days. If it
shows the same behaviour, it should be reported as a bug.

 

[Ramchandra Apte]

On 9/21/2012 5:10 AM, Marco wrote:

You appear to be trying to *run*, not *import* a .pyo module.

Marco is using the standard mechanism for finding, importing, and running

a module. I don't believe his use of -m should be a problem. It works in

3.2, and it works with .pyc files in 3.3, I see nothing to suggest it

shouldn't work with .pyo files in 3.3.

Since foo.py is in the current directory, I am not sure why you use '-m

foo' instead of 'foo.py'. -m is for running a module somewhere on

sys.path.

Yes, and the current directory is on sys.path.



I would be astonished if python -m could not find a module that happened

to be in the current directory.





[...]

Also, the

-O is sort of redundant, or perhaps interfering, since its usual effect

to to say 'get and put, from and to the cache, .pyo instead of .pyc'.

No it is not redundant. You link specifically to an bug tracker issue

below where is is clearly decided that if you want to run a .pyo file you

*must* use the -O switch. (I approve of this decision.)

You might read some of http://bugs.python.org/issue12982

in particular, from http://bugs.python.org/issue12982#msg162814

Whose words are these following?

Python interpreters exist to run Python code. The existence,

persistence, and other details of compilation caches are

version-dependent implementation details. Being able to execute from

such caches without source present is also an implementation detail, and

for CPython, it gets secondary support at best. (This is a compromise

between full support and no support.)"

I'm not sure if these are your words, or if you are quoting some random

commenter on the pydev list, or one of the lead developers who might

actually know what he is talking about.

It is Terry J. Reedy's words - see end of http://bugs.python.org/issue12982#msg162814

 

[Terry Reedy]

You appear to be trying to *run*, not *import* a .pyo module.

Marco is using the standard mechanism for finding, importing, and running
a module. I don't believe his use of -m should be a problem. It works in
3.2, and it works with .pyc files in 3.3, I see nothing to suggest it
shouldn't work with .pyo files in 3.3.

Since foo.py is in the current directory, I am not sure why you use '-m
foo' instead of 'foo.py'. -m is for running a module somewhere on
sys.path.

Yes, and the current directory is on sys.path.

I would be astonished if python -m could not find a module that happened
to be in the current directory.


[...]

Also, the
-O is sort of redundant, or perhaps interfering, since its usual effect
to to say 'get and put, from and to the cache, .pyo instead of .pyc'.

No it is not redundant. You link specifically to an bug tracker issue
below where is is clearly decided that if you want to run a .pyo file you
*must* use the -O switch. (I approve of this decision.)

You might read some of http://bugs.python.org/issue12982

in particular, from http://bugs.python.org/issue12982#msg162814

Whose words are these following?

Python interpreters exist to run Python code. The existence,
persistence, and other details of compilation caches are
version-dependent implementation details. Being able to execute from
such caches without source present is also an implementation detail, and
for CPython, it gets secondary support at best. (This is a compromise
between full support and no support.)"

I'm not sure if these are your words, or if you are quoting some random
commenter on the pydev list, or one of the lead developers who might
actually know what he is talking about.

My words summarizing the discussion on pydev which included at least a
few lead developers. My initial post was probably 6/12/2012

 

[Steven D'Aprano]

I was trying to import a pyo module in Python 3.3, but Python does not
find it:

[...]

Marco, this bug is apparently now fixed:

http://bugs.python.org/issue16046

Please download the latest version of Python 3.3 and try again.

(Folks, this is why testing and reporting bugs to beta and rc versions is
so useful. You can often get them fixed quite quickly.)