Improving interpreter startup speed

[Pedro Borges]

Hi guys,


Is there a way to improve the interpreter startup speed?

In my machine (cold startup) python takes 0.330 ms and ruby takes
0.047 ms, after cold boot python takes 0.019 ms and ruby 0.005 ms to
start.


TIA

 

[Lie Ryan]

Hi guys,


Is there a way to improve the interpreter startup speed?

In my machine (cold startup) python takes 0.330 ms and ruby takes 0.047
ms, after cold boot python takes 0.019 ms and ruby 0.005 ms to start.


TIA

um... does it really matter? It's less than a second and only once on the
program startup...

if you've found yourself destroying small python processes thousands of
times, try creating the controller program in python, so this controller
program would import the "small modules" and doesn't restart python
interpreters that many times.

 

[Steven D'Aprano]

Hi guys,


Is there a way to improve the interpreter startup speed?

Get a faster computer?

In my machine (cold startup) python takes 0.330 ms and ruby takes 0.047
ms, after cold boot python takes 0.019 ms and ruby 0.005 ms to start.

How are you measuring this?

 

[BJörn Lindqvist]

Is there a way to improve the interpreter startup speed?

In my machine (cold startup) python takes 0.330 ms and ruby takes
0.047 ms, after cold boot python takes 0.019 ms and ruby 0.005 ms to
start.

How are you getting those numbers? 330 μs is still pretty fast, isn't
it? Most disks have a seek time of 10-20 ms so it seem implausible
to me that Ruby would be able to cold start in 47 ms.

 

[Terry Reedy]

Hi guys,


Is there a way to improve the interpreter startup speed?

In my machine (cold startup) python takes 0.330 ms and ruby takes
0.047 ms, after cold boot python takes 0.019 ms and ruby 0.005 ms to
start.

You of course mean CPython, but Version, version, what Version?
3.0 starts much quicker than 2.5. Don't have 2.6.

 

[James Mills]

How are you getting those numbers? 330 μs is still pretty fast, isn't
it? Most disks have a seek time of 10-20 ms so it seem implausible
to me that Ruby would be able to cold start in 47 ms.

$ time python -c "pass"

real 0m0.051s
user 0m0.036s
sys 0m0.008s

$ time python3.0 -c "pass"

real 0m0.063s
user 0m0.048s
sys 0m0.004s

And yes I agree. the CPython interpreter startup times is
a stupid thing to be worrying about, especially since that
is never the bottleneck.

Python loads plenty fast enough!

--JamesMills

 

[Pedro Borges]

The scripts i need to run but be executed with no apparent delay
specially when the text transforms are simple.

 

[Paul Rubin]

The scripts i need to run but be executed with no apparent delay
specially when the text transforms are simple.

Basically you should keep the interpreter running and the script in
memory in that case.

 

[Terry Reedy]

I disagree. The extra time Python takes to start makes it unsuitable
for many uses. For example, if you write a simple text editor then
Pythons longer startup time might be to much.


You must be in a real big hurry if half a second matters that much to
you. Maybe if it took 5 seconds for the interpreter to start up, I could
understand having a problem with the start up time.

The secret is to start Python at least a second before one is actually
ready to start.

 

[James Mills]

The scripts i need to run but be executed with no apparent delay specially
when the text transforms are simple.

That makes no sense whatsoever!

If you are performing data conversion with
Python, interpreter startup times are going
to be so insignificant.

--JamesMills

 

[James Mills]

You must be in a real big hurry if half a second matters that much to you.
Maybe if it took 5 seconds for the interpreter to start up, I could
understand having a problem with the start up time.

+1 This thread is stupid and pointless.
Even for a so-called cold startup 0.5s is fast enough!

--JamesMills

 

[James Mills]

Pedro was talking about cold startup time:

$ sudo sh -c "echo 3 > /proc/sys/vm/drop_caches"
$ time python -c "pass"

real 0m0.627s
user 0m0.016s
sys 0m0.008s

$ sudo sh -c "echo 3 > /proc/sys/vm/drop_caches"
$ time python -S -c "pass"

real 0m0.244s
user 0m0.004s
sys 0m0.008s

With -S (don't imply 'import site' on initialization)

I suspect that's to do with importing modules,
the site specific modules, etc. Disk access will
tend to chew into this "startup time". Use -S
if you're that worried about startup times (heaven
knows what affect it'll have on your app though).

--JamesMils

 

[David Cournapeau]

On Mon, Oct 27, 2008 at 4:12 AM, Benjamin Kaplan
+1 This thread is stupid and pointless.
Even for a so-called cold startup 0.5s is fast enough!

Not if the startup is the main cost for a command you need to repeat many times.

David

 

[James Mills]

Not if the startup is the main cost for a command you need to repeat many times.

Seriously if you have to spawn and kill python
processes that many times for an initial cold
startup and subsequent warm startups to be
any significance, you're doing something wrong!

--JamesMills

 

[Terry Reedy]

Not if the startup is the main cost for a command you need to repeat many times.

It this a theoretical problem or an actual one, that we might have other
suggestions for?

 

[David Cournapeau]

It this a theoretical problem or an actual one, that we might have other
suggestions for?

Any command line based on python is a real example of that problem.
There are plenty of them.

David

 

[James Mills]

It this a theoretical problem or an actual one, that we might have other
suggestions for?

Heaven knows! I hardly think invoking hundreds
and possibly thousands of short-lived python
interpreters to be an optimal solution that may
have spawned this particular thread.

--JamesMills

 

[James Mills]

Any command line based on python is a real example of that problem.
There are plenty of them.

Yes, but in most cases you are not invoking your
command-line app x times per y units of time.

--JamesMills

 

[Paul Rubin]

Heaven knows! I hardly think invoking hundreds
and possibly thousands of short-lived python
interpreters to be an optimal solution that may
have spawned this particular thread.

It's not optimal but it is very common (CGI for example).

 

[David Cournapeau]

Yes, but in most cases you are not invoking your
command-line app x times per y units of time.

Depends on the tool: build tool and source control tools are example
it matters (specially when you start interfaciing them with IDE or
editors). Having fast command line tools is an important feature of
UNIX, and if you want to insert a python-based tool in a given
pipeline, it can hurt it the pipeline is regularly updated.

cheers,

David