how do you prevent distutils from downloading and building packageswithout consent?

[lkcl]

folks, hi,

a number of people using pyjamas are not only encountering
difficulties with setup.py endeavouring to download and install
"setuptools" but also they are ... the best word to use is
unfortunately "offended" - by the fact that distutils, in its default
configuration, downloads and even _compiles_ its dependencies -
*without consent*.

a copy of the setup.py can be found here:
http://pyjamas.svn.sourceforge.net/viewvc/pyjamas/trunk/setup.py

this alarming situation has shown to occur on windows, ubuntu and
debian. personally, i used setup.py to install pyjamas on a debian
system: as i happened not to have python-setuptools installed, i was
alarmed to find that setup.py, thanks to distutils, was not only
downloading but also attempting to _compile_ setuptools. fortunately,
the compile actually failed.

windows users have also reported similar failures (presumably because
the do not have a suitable build environment, involving downloading
and installing a proprietary c compiler) and have also expressed their
displeasure at not being consulted as to which bits of software are
going to get thrown onto their system without their permission.

so it should be fairly clear as to why it is unacceptable for this to
occur.

the question is: how can it be stopped? what option, in distutils
"terminology", tells distutils "please simply tells users that a
package is missing, providing them with sufficient information such
that they can rectify the missing dependency in a manner which is
satisfactory _to them_?

answers and guidance greatly appreciated, including telling me that
i've made a simple mistake in the setup.py, linked above.

l.

 

[John Machin]

folks, hi,

a number of people using pyjamas are not only encountering
difficulties with setup.py endeavouring to download and install
"setuptools" but also they are ... the best word to use is
unfortunately "offended" - by the fact that distutils, in its default
configuration, downloads and even _compiles_ its dependencies -
*without consent*.

distutils is *not* involved.

Here are all of the imports in your setup.py:

import glob
from setuptools import setup, find_packages
from setuptools.command import test
from setuptools.command.install import install
import sys
import os
from pprint import pprint

A distutils setup.py will be an elaboration on this example from the
manual:

from distutils.core import setup
setup(name='foo',
version='1.0',
py_modules=['foo'],
)

See that "from distutils.core import setup"? See anything like that in
your setup.py?

Whatever is happening, don't blame distutils -- it's absolutely
nothing to do with distutils. *You* are using a setup.py that calls
setuptools, which is a quite different distribution method. Have you
read the setuptools documentation? Didn't you *test* your setup.py
before making it available to the world?

a copy of the setup.py can be found here:http://pyjamas.svn.sourceforge.net/viewvc/pyjamas/trunk/setup.py

this alarming situation has shown to occur on windows, ubuntu and
debian.  personally, i used setup.py to install pyjamas on a debian
system: as i happened not to have python-setuptools installed, i was
alarmed to find that setup.py, thanks to distutils, was not only
downloading but also attempting to _compile_ setuptools.  fortunately,
the compile actually failed.

What is so gut-wrenchingly awful about "compile" that you must
underline its every occurrence?

windows users have also reported similar failures (presumably because
the do not have a suitable build environment, involving downloading
and installing a proprietary c compiler) and have also expressed their
displeasure at not being consulted as to which bits of software are
going to get thrown onto their system without their permission.

so it should be fairly clear as to why it is unacceptable for this to
occur.

the question is: how can it be stopped?  what option, in distutils
"terminology", tells distutils "please simply tells users that a
package is missing, providing them with sufficient information such
that they can rectify the missing dependency in a manner which is
satisfactory _to them_?

Try reading the *setuptools* documentation.

HTH,
John

 

[Gabriel Genellina]

a number of people using pyjamas are not only encountering
difficulties with setup.py endeavouring to download and install
"setuptools" but also they are ... the best word to use is
unfortunately "offended" - by the fact that distutils, in its default
configuration, downloads and even _compiles_ its dependencies -
*without consent*.

a copy of the setup.py can be found here:
http://pyjamas.svn.sourceforge.net/viewvc/pyjamas/trunk/setup.py

distutils doesn't attempt to download *anything*. You're confusing it with
setuptools and easy_install, which is a different beast.
Your setup.py seems overly complicated to me. For a pure Python package,
setup.py looks like this:

from distutils.core import setup
setup(name='Foo package',
description='My fine foo package',
version='1.0',
packages=['foo'])

Two statements, nothing more. To include an extension module bar.c:

setup(...
ext_modules=[Extension('bar', ['bar.c', 'other.c'])],
...)

See http://docs.python.org/distutils/index.html

 

[Gabriel Genellina]

Whatever is happening, don't blame distutils -- it's absolutely
nothing to do with distutils. *You* are using a setup.py that calls
setuptools, which is a quite different distribution method. Have you
read the setuptools documentation? Didn't you *test* your setup.py
before making it available to the world?

(Ouch, I didn't notice your earlier reply when I posted mostly the same
thing, sorry!)

 

[lkcl]

read the setuptools documentation? Didn't you *test* your setup.py

before making it available to the world?

yes - and it worked on my debian linux box. so, off it went. turns
out that it worked because i had python-setuptools preinstalled.

What is so gut-wrenchingly awful about "compile" that you must
underline its every occurrence?

the disparity between the expectations and the actual behaviour.
but, given that both you and gabriel spotted that the setup.py uses
setuptools (which i'm not familiar with) that would make sense. duh.

that's what i was missing - thank you both, john and gabriel. i tend
to cut/paste quite horrendous amounts of bits of code together. most
of the time, it works. some of the time, i miss things. much
appreciated that you pointed the mistake out.

l.

 

[lkcl]

ok - john, gabriel, i've now removed setuptools, which is the area
that's problematic for many people. however.... there's a feature of
setuptools which _is_ useful:

- entry_points = {'console_scripts':[
- 'pyjsbuild=pyjs.build:main',
- 'pyjscompile=pyjs:main',
- ]},

whoops - those are now gone.

i have a command, bootstrap.py, which replicates that exact
functionality: it creates a customised pyjsbuild which can go in /usr/
bin or /usr/local/bin and it will add the library locations to
sys.path and also as a -D option to the pyjs main() function - it's a
little obscure, but works out fine.

e.g. if you specify no arguments, this gets auto-generated, in ./bin/
pyjsbuild :

#!/usr/bin/python

pth = '/home/lkcl/src/sf.pyjamas/svn-pyjamas/pyjamas'

import os
import sys
sys.path[0:0] = [
pth,
]

import pyjs, sys
pyjs.path += [os.path.join(pth, 'library'),
os.path.join(pth, 'library', 'builtins'),
os.path.join(pth, 'addons'),
]
sys.argv.extend(['-D', pth])

import pyjs.build

if __name__ == '__main__':
pyjs.build.main()

if you specify "python bootstrap.py /usr/share/pyjamas /tmp" you get,
in /tmp/bin/pyjsbuild:


#!/usr/bin/python

pth = '/usr/share/python'

import os
import sys
sys.path[0:0] = [
pth,
]

import pyjs, sys
pyjs.path += [os.path.join(pth, 'library'),
os.path.join(pth, 'library', 'builtins'),
os.path.join(pth, 'addons'),
]
sys.argv.extend(['-D', pth])

import pyjs.build

if __name__ == '__main__':
pyjs.build.main()


so, would you (or anyone else) happen to know how i can get setup.py
to run bootstrap.py to create these two scripts, in a way that's
compatible with the setuptools "entry_points" thing? i.e. taking the
option of "--install-data" as the first argument to bootstrap.py and
the option of "--prefix" as the second argument.

i envisage it to be something like:


from bootstrap import create_commands
if distutils.XXXXX.cmd == "build":
create_commands(distutils.XXXXXX.install_data_path,
distutils.XXXXXX.prefix)

but it all looks a bit hair-raising, if you ask me. i'm not often on
wobbly ground when it comes to python, but this is pretty specific
stuff that has to be got right across multiple platforms.

tia,

l.

 

[lkcl]

http://mail.python.org/pipermail/distutils-sig/2008-March/008925.html

dang.