Speed Comparison Perl Python & C

[beliavsky]

It actually compares the speed of integers large enough to need 64 bits
of precision where Fortran can use a 64 bit integer and Python uses an
unbounded integer. The test can be sped up in two ways, first by using
floating point (which is fixed length in Python) and by using Psyco. On my
machine:

Original test: 245 seconds
Using floats: 148 seconds
Using psyco with integers: 69 seconds
Using psyco with floats: 18.4 seconds

Therfore gaining a speed up of 13 times. This leads to Fortran likely
remaining 22 times faster.

from psyco.classes import *
import psyco

def xx():
i = 1.0
j = 0.0
while i < 100000000.0:
j = j + i
i = i + 1
print int(j)

psyco.profile()

xx()

Neil

Thanks. I am going to learn about Psyco. In this case, I assume that
doing the computations with floating point numbers and finally
converting the result to int gives the same values as the original
integer calculation. In other cases, integer arithmetic will need to
be done with integers to ensure correct results.

Python is supposed to be easy (and in general I agree that it is), but
your solution requires some knowledge of

(1) how integer and floating point calculations are done (which many
novices do not have)
(2) when Psycho can speed things up

and the final result is still much slower than Fortran. For the
Fortran program, the only "trick" is the use of integer*8.

 

[Paul Rubin]

Thanks. I am going to learn about Psyco. In this case, I assume that
doing the computations with floating point numbers and finally
converting the result to int gives the same values as the original
integer calculation.

This isn't obvious since the answer around 1.15*2**52 and Python
floats have 53 bits of precision.

 

[Neil Hodgson]

(e-mail address removed):

Thanks. I am going to learn about Psyco. In this case, I assume that
doing the computations with floating point numbers and finally
converting the result to int gives the same values as the original
integer calculation. In other cases, integer arithmetic will need to
be done with integers to ensure correct results.

Yes, in this case float has enough range. That is something you get to
determine with any fixed length numeric representation.

Python is supposed to be easy (and in general I agree that it is), but
your solution requires some knowledge of

(1) how integer and floating point calculations are done (which many
novices do not have)
(2) when Psycho can speed things up

and the final result is still much slower than Fortran. For the
Fortran program, the only "trick" is the use of integer*8.

With Fortran, you need to know how large your values are going to become.
If you increased the number of iterations in your example sufficiently,
Fortran's 64 bit integers would overflow requiring understanding of the
concept of fixed range integers. Python avoids this by using unbounded
integers. Python is oriented towards ease of use and correctness at the
expense of speed. If the speed of the Python program is inadequate then you
can take a working Python program and work on its speed. Or decide that the
speed problem needs another language.

Python does optimize integers that can be represented in 32 bits but
larger than that and unbounded integers are used. For some applications, it
would be better if Python also optimized integers that require between 32
and 64 bits.

Neil

 

[Bob Ippolito]

(e-mail address removed):


Yes, in this case float has enough range. That is something you get to
determine with any fixed length numeric representation.


With Fortran, you need to know how large your values are going to become.
If you increased the number of iterations in your example sufficiently,
Fortran's 64 bit integers would overflow requiring understanding of the
concept of fixed range integers. Python avoids this by using unbounded
integers. Python is oriented towards ease of use and correctness at the
expense of speed. If the speed of the Python program is inadequate then you
can take a working Python program and work on its speed. Or decide that the
speed problem needs another language.

Python does optimize integers that can be represented in 32 bits but
larger than that and unbounded integers are used. For some applications, it
would be better if Python also optimized integers that require between 32
and 64 bits.

Especially on architectures that have 64 bit integer registers, but are
running on an operating system/compiler combination that uses 32 bits
for int and long (such as the PowerPC 970, on OS X). I would imagine a
similar situation exists for other processor/enivronment combinations.

-bob

 

[Thomas Heller]

Especially on architectures that have 64 bit integer registers, but
are running on an operating system/compiler combination that uses 32
bits for int and long (such as the PowerPC 970, on OS X). I would
imagine a similar situation exists for other processor/enivronment
combinations.

Instead of making this is responsibility of the Python code, wouldn't
this better be a job for psyco?

BTW, does psyco run on non-x86 architectures?

Thomas

 

[ciw42]

Even the most basic optimising compiler would turn your example Fortran
increment variable loop into a few basic machine code instructions operating
on hardware registers, which combined with the internal cache would keep the
entire loop inside the processor avoiding memory bandwidth issues etc. It's
not surprising you're getting such dramatic ratios, but it's hardly a real
world example.

Compiled optimised code will (should) always be significantly faster than
interpreted code, but you lose all of the benefits of using a language such
as Python in the process, so it's all rather pointless comparing languages
in this way.

I've spent over 20 years coding all manner of languages - assembly, BASIC,
COBOL, C/C++, VB, Delphi etc. and speed of execution is of little or no
concern in 99% of the projects I work on, especially these days. If it were
the be-all and end-all of development I'd still be coding in assembly.

 

[Bob Ippolito]

Instead of making this is responsibility of the Python code, wouldn't
this better be a job for psyco?

Perhaps psyco and/or Numeric/Numarray, I guess... I would imagine that
the people who would need such speed don't care where it is, so long as
it's available to them.

BTW, does psyco run on non-x86 architectures?

Yes, barely.. it uses a written-in-C "microVM" that can do some things
more efficiently than Python's "monsterVM"

-bob

 

[Paul Rubin]

Compiled optimised code will (should) always be significantly faster than
interpreted code, but you lose all of the benefits of using a language such
as Python in the process, so it's all rather pointless comparing languages
in this way.

Of course you won't lose those benefits. The result of compiling
Python to optimized native code is that the Python code will run
faster that way. In other regards, it will be the same. You keep the
benefits and gain more speed.

 

[David M. Cooke]

Python is supposed to be easy (and in general I agree that it is), but
your solution requires some knowledge of

(1) how integer and floating point calculations are done (which many
novices do not have)
(2) when Psycho can speed things up

and the final result is still much slower than Fortran. For the
Fortran program, the only "trick" is the use of integer*8.

It's quite easy (using F2PY) to make Python wrapper around a Fortran
routine. So if you're in a situation where Fortran is much faster,
then use it. Wrap it in Python and slap a webserver and a GUI on
that puppy, and run rings around pure Fortran code.

[Heck, I'll cry if I ever have to write any serious user interface
code in Fortran. More than my sanity is worth.]

 

[Dave Cole]

.
.
.


.
.
.
Yup; I'm now sufficiently arrogant/provocative/confident/experienced/...
to tell people to expect that their Python codings will be *faster* than
what they achieve with C.

I can cite an example of this from my own experience.

My first implementation of the Sybase DB-API implemented most of the
interface in C for speed. I had only been using Python for a couple of
weeks when I developed that version. After a while I started to think
about supporting array binding to fetch results from the server more
than one row at a time. The amount of work necessary to make the C code
support array binding was more than I was prepared to undertake.

I wanted to support array binding so I decided to simply wrap the Sybase
CT API and implement the DB-API in Python on top of the wrapping. This
was a huge win because I was able to implement a feature that would
probably have been too hard in plain C (at least for my puny brain).

So I agree, in some (maybe even many) cases the Python implementation
will be faster than the C implementation purely because of the increased
sophistication of the solutions you are able to implement.

- Dave