where are the program that are written in python?

[Deep_Feelings]

thank you very much ,your reply guys are very nice and informative.

hope you best luck in your life

 

[Tim Chase]

That only applies to CPU bound program code (most program code is I/O
bound), and only to computational bottlenecks (usually less than 5% of
the code) in the CPU bound programs. Today, most programs are I/O
bound: You don't get a faster network connection or harddrive by using
C. In this case, performance depends on other factors than choice of
language. That is why Mercurial (written in Python) can be much faster
than SVN (written in C).

For computational bottlenecks we might want to try high-performance
numerical libraries first. If that does not help, we can try to
replace some Python with C.

Just as an aside, last I checked, mercurial had some core code in
C for speed. But that doesn't negate your line of reasoning,
rather it cements it -- they found it was most productive to work
in Python, but needed the core bits to improve in speed so
rewrote them in C.

I'd also include that a change in algorithm can be a big help for
speeding up CPU-bound code. It doesn't matter much if you're
using Python or hand-coding that inner loop in C/ASM, if you're
using a O(2^N) algorithm. I find it easier to write good/speedy
algorithms in Python because I have a toolkit of built-in
data-types (sets, dicts, lists, etc) that I can reach for,
without making sure I've added-on certain C libraries.

-tkc

 

[Aahz]

I'd also include that a change in algorithm can be a big help for
speeding up CPU-bound code. It doesn't matter much if you're using
Python or hand-coding that inner loop in C/ASM, if you're using a
O(2^N) algorithm.

Rewriting an algorithm also helps I/O-bound code when you're doing
something stupid like querying a DB multiple times for each record
instead of caching the result. Also, rewriting your algorithm to just
pull the entire DB into RAM helps, too. (If you know your dataset must
fit into RAM, anyway, in order to process your algorithm.)

 

[Patrick Maupin]

That only applies to CPU bound program code (most program code is I/O
bound), and only to computational bottlenecks (usually less than 5% of
the code) in the CPU bound programs. Today, most programs are I/O
bound: You don't get a faster network connection or harddrive by using
C. In this case, performance depends on other factors than choice of
language. That is why Mercurial (written in Python) can be much faster
than SVN (written in C).

For computational bottlenecks we might want to try high-performance
numerical libraries first. If that does not help, we can try to
replace some Python with C. Python as "glue" does not mean Python is
inferior to C. It just means it is a PITA to write C or Fortran all
the time. I value my own time a lot more than a few extra CPU cycles.
Who cares about speed where it is not needed?

I think we're in violent agreement here -- you neglected to quote the
part where I said "(But the up-front choice of another language simply
for speed, rather than prototyping with Python and then recoding the
slow bits, would probably be a decision borne of ignorance.)"

Regards,
Pat

 

[Terry Reedy]

doubting python's speed?

The is a somewhat bizarre response to me. I have been promoting Python
for about 13 years, since I dubbed it 'executable pseudocode', which is
to say, easy to write, read, understand, and improve. I am also a
realist. Any fixed (C)Python program can be sped up, at least a bit, and
possibly more, by recoding in C. At minimum, the bytecodes can be
replaced by the C code and C-API calls that they get normally get
translated into. Ints can be unboxed. Etcetera. This tend to freeze a
program, which is fine when development is finished.

I believe Raymond wrote each itertool (or at least most) in Python
first, then rewrote each in C for speed, since they are intented to be
repeatedly used components of other Python programs. He is not 'doubting
Python's speed', just being realistic.

> Look at Mercurial vs. SVN;

Neither are being sold, as far as I know.

Mercurial is written in Python while SVN in C.
> Mercurial beats SVN in speed by several orders
of magnitude.

Because, as I said, and as you explain further, Python favors programmer
speed, including speed of testing new algorithms, over raw execution
speed of current algorithms. (Current) speed is (also) easier to test
than improvability and hence possible speed improvements.

One of Mercurial's design goal was to be faster than SVN, if the
programmers have naively believed that choice of language would matter
to program's speed, they'd choose to write Mercurial in assembly instead
(the same argument applies to Git, written in shell scripts).

Now, you may think this is an unfair comparison, since Mercurial is hype
and new, SVN is antiquated and old. But it shows that in real-life, the
language being inherently slow often dosn't matter. What matters more
are the choice of data structure and algorithm, I/O speed, network
latency, and development speed.

If and when mercurial deveopment slows down, some of its developers
might consider whether any of the utility functions written in Python
might usefully be rewritten in C. One of the intentional virtues of
Python is that one can transparently convert part or all of a module
*without changing* the import and use of the module.

Terry Jan Reedy

 

[Patrick Maupin]

Because, as I said, and as you explain further, Python favors programmer
speed, including speed of testing new algorithms, over raw execution
speed of current algorithms. (Current) speed is (also) easier to test
than improvability and hence possible speed improvements.

First of all, I don't think you and Lie have any basic disagreements.
The key realization is that the quantitative difference in programmer
speed you mention is so large (orders of magnitude) that, for many
classes of problems, it is not just *possible*, but actually
*probable*, that a Python implementation *will be faster* than a C
implementation. Yes, you are absolutely correct that most Python
programs can be made faster by adding a bit of C, but that doesn't
negate the fact that if I can throw 'x' man-hours at a problem, for
lots of real-world values of 'x' and of 'the problem', a pure Python
implementation will run rings around a pure C implementation, assuming
the C implementation even works by the time I've burned through 'x'
hours. I discussed this a bit on this newsgroup over five years ago,
and the points are still pertinent:

http://groups.google.com/group/comp.lang.python/msg/910a54ddec946567

If and when mercurial deveopment slows down, some of its developers
might consider whether any of the utility functions written in Python
might usefully be rewritten in C. One of the intentional virtues of
Python is that one can transparently convert part or all of a module
*without changing* the import and use of the module.

I don't even think that Mercurial development has to slow down to
decide to recode a few things in C. A tiny bit of C at the right
place can often provide more than enough leverage to be worthwhile,
and be small enough to throw away if need be.

Regards,
Pat

 

[Patrick Maupin]

Erm, in fairness, I recall hearing that some speed-critical bits of hg
are written in C. It does lend credence to the "Python as glue
language" argument though; I doubt hg's extensibility and friendly
interface would have been as easy to implement it C (particularly the
slick instant-server feature).

Is C viewed as a "glue language" in those environments where it is the
primary tool and sometimes some small bits are recoded into assembly
language for speed?

Regards,
Pat

 

[Adam Tauno Williams]

There are a lot of commercial programs written in Python. But any
company which thinks it has a lock on some kind of super secret sauce
isn't going to use Python,

Is it [only] the aspect of being "sold" that makes software
"commercial"?

A better question would be is how many Python applications, in house or
not, are used to facilitate commerce. Answer: a lot.

I have an Open Source project with >100,000 lines of Python code [which
I think qualifies as a 'real' application]
<https://www.ohloh.net/p/coils/analyses/latest>. But that it is Open
Source makes it non-commercial? I doubt anyone would use it outside of
a commercial environment, and one of its principle goals is to serve as
the backend for CRM systems [essentially commercial] and facilitate
automation of business processes [essentially commercial]. The 'secret
sauce' isn't the code [which is MIT licenses] but what you do with it.
But since the framework is essentially general purpose - why not publish
the code?

I think of my Open Source code as "commercial".

 

[Gregory Ewing]

i will be interested more in COMMERCIAL programs written in python

I came across a game on Big Fish Games recently (it was
"The Moonstone" IIRC) that appeared to have been built using
Python and py2app.

 

[Lie Ryan]

The is a somewhat bizarre response to me. I have been promoting Python
for about 13 years, since I dubbed it 'executable pseudocode', which is
to say, easy to write, read, understand, and improve. I am also a
realist. Any fixed (C)Python program can be sped up, at least a bit, and
possibly more, by recoding in C. At minimum, the bytecodes can be
replaced by the C code and C-API calls that they get normally get
translated into. Ints can be unboxed. Etcetera. This tend to freeze a
program, which is fine when development is finished.

I'm not claiming Python is faster than C, but I'm just being a realists,
when I say that in real life 9 out of 10 writing a program in a slow
language doesn't really matter to actual program speed. I used Mercurial
as an example where the developers choose an initially irrational
decision of using a slow language (python) to beat the speed of a fast
language (C).

Of course, you can always point out the 1 case out of 10. In this cases,
python can still cope with C extension, Psyco, Numpy-and-friends,
Cython, or even dumping python and using full C all the way.

But the point still hold, that in real life, often the language's raw
speed doesn't really limit the program's speed.

 

[David Cournapeau]

But the point still hold, that in real life, often the language's raw
speed doesn't really limit the program's speed.

I would rather say that Python vs C does not matter until it does, and
it generally does when constants factor matter (which is one way to
draw the line between system programming and "general" programming).
It totally depends on the applications, I don't think you can draw
general arguments.

David

 

[alex23]

I came across a game on Big Fish Games recently (it was
"The Moonstone" IIRC) that appeared to have been built using
Python and py2app.

Python tends to be used more for scripting internal game logic than
for every aspect of a game (which is, IMO, the right way to go about
it). It's not a huge list of commercial games that does this[1], but
it's a fairly classy one

1: http://en.wikipedia.org/wiki/Category:Python-scripted_video_games

 

[Adam Tauno Williams]

Agree, reveres engineering is crucial issuer for programming
language
but every executable file can be cracked, for example by using
disassembler!!!
For each weapon there is antiweapon, so
is it possible to prevent reveres engineering when customer have
access to executable made from Python code???

No. But you can make it hard.

Store a GPG encrypted blob in your program that contains you secret
sauce, is decrypted to memory, executed, and then discarded. Setup
some kind of license manager like dongle or application to perform the
key management.

 

[Terry Reedy]

is it possible to prevent reveres engineering when customer have access
to executable made from Python code???

If there is, it is a trade secrets that you will not be able to reverse
engineer ;-).

 

[Lie Ryan]

No. But you can make it hard.

Store a GPG encrypted blob in your program that contains you secret
sauce, is decrypted to memory, executed, and then discarded. Setup
some kind of license manager like dongle or application to perform the
key management.

That merely gives a false sense of security. If the program is decrypted
in memory, you can easily make a memory dump to get the unencrypted
program. If I am a competitor that can make economic advantage by
cracking your secret sauce, it wouldn't be difficult for me to do that.

 

[D'Arcy J.M. Cain]

That merely gives a false sense of security. If the program is decrypted
in memory, you can easily make a memory dump to get the unencrypted
program. If I am a competitor that can make economic advantage by
cracking your secret sauce, it wouldn't be difficult for me to do that.

Yes, in fact the only people inconvenienced are your paying clients.

 

[Grant Edwards]

Yes, in fact the only people inconvenienced are your paying clients.

After several bad experiences over the years, I'm now willing go to
quite a bit of effort to avoid using products that require dongles,
node-locked licenses or license servers.

 

[Adam Tauno Williams]

That merely gives a false sense of security.

There is no "true" sense of security. There is only degrees of
obfuscation, hence the first sentence: "No. But you can make it hard"

If the program is decrypted
in memory, you can easily make a memory dump

Easily? Really? You vastly over estimate the majority of computer
users. If someone who knows how to read the memory of a running process
wants your secret sauce - they are going to get it.

to get the unencrypted
program. If I am a competitor that can make economic advantage by
cracking your secret sauce, it wouldn't be difficult for me to do that.

True. That is pretty much always true. The only effective solution is
to have your app call a web service [or some kind of RPC] to a server
where you keep the secret sauce hidden away.

 

[Daniel Fetchinson]

is it possible to prevent reveres engineering when customer have access
The only secure way of protecting your code is if you expose it as a
web service or some such. (Yes, people can still crack your web
server, but that's nitpicking )

Cheers,
Daniel