python vs perl lines of code

[akameswaran]

Yes, like the shorter version might be overlooking many real world
situations and is naive code. As for generalization, if you bet that the
shorter one is later written, that's to me a generalization. I agree that
there is a change that after reexamining the code, and algorithm can be
written shorter, but I have also seen algorithms refactored for better
readability.

All very good points - I need to be more specific. I've been working
on some data analysis stuff etc, lately - so I'm often time just
reimplementing a specific algo or library I've written. The actual
program as a whole generaly does get larger. But I was really
thinking about a handful of data manipulation or aggregation algo's
that were functionaly fine - but I realized could be done better.

Enough to bet on it ;-)

I'm a gambling man, what can I say?

Aren't those closely related?

Yep, but not the same thing. Maintainability is a subset of quality.

Because it depends a lot on the skill level of the maintainer. By just
counting lines and characters you can't measure quality IMO. It's a naive
way of measuring and it reminds me of the early days of search engines.

And if you mistake understanding that it's not a good way to measure
things as having no intellectual curiosity, you're again mistaken.

All I would ask is what objective evidence does either of actually
have? How can you know? What is a fair way to even count line
numbers? From there how do we begin to objectively measure software
quality? That's why this discussion interests me, and why I don't
understand why you are so adamant it doesn't work. I'll agree that I
have never seen line count/char count type data used for anything other
than marketing swill and kitty litter. Doesn't mean it can't be used.
But first things first... and this one I think is solvable - their has
got to be an equitable way to count how much code was written - maybe
it isn't lines maybe it is. In truth, since you are so opposed to the
idea, I'd be curious if you can think of a way to measure the quantity
of code objectively? ANd that's it - not can we make a qualitative
statement beyond that. But simply can we quanitfy the amount of code
in some fashion that allows a reasonable comparison?

 

[Edward Elliott]

But first things first... and this one I think is solvable - their has
got to be an equitable way to count how much code was written - maybe
it isn't lines maybe it is....
ANd that's it - not can we make a qualitative
statement beyond that. But simply can we quanitfy the amount of code
in some fashion that allows a reasonable comparison?

Maybe a count of identifiers and keywords. Symbols like ()[];,. don't add
much beyond where to parse the code. And even though operators like
+-*/^&| et al convey semantics, they need expressions to operate on, in
which the identifiers are already counted. You could make a case either
way but I'd leave them out for simplicity.

The question is how to count explicit names like module.class.func; should
that be 1 identifier or 3? Counting as 3 would reward things like "from X
import *" which are generally frowned on in python while 'use MOD qw(id)'
is encouraged in perl. I'm inclined to just count explicit names as 1.
It's still a single object, and I'm not sure a.b.c is more work to process
than just c anyway. If you don't care where c is, they're equivalent. If
you do care, you've got to remember where the naked c lives, so it just
shifts the work from the code to your brain.

Then you've got problems with implicit variables/operations. Should perl's
$_ be counted? What about python's copy-slice [:]? You can drive yourself
crazy worrying about such things, so I'd just start with the simple case by
ignoring them.

This id-keyword count would appear to be related to how much information you
must process to read through the code. Unfortunately you need a parser for
each language to measure it. If anyone can provide such a beast, I'll
gladly run it against my code. In the meantime, line/char counts are the
best I've got.

 

[John Bokma]

The question is how to count explicit names like module.class.func;
should that be 1 identifier or 3? Counting as 3 would reward things
like "from X import *" which are generally frowned on in python while
'use MOD qw(id)' is encouraged in perl.

Not by me, and I doubt it is in general.

 

[Edward Elliott]

Not by me, and I doubt it is in general.

Well it's all over the Perl Cookbook.

 

[John Bokma]

Well it's all over the Perl Cookbook.

Yeah, sure, all over. Maybe check the book again. It is used in some
examples, sure. And it even explains how it works. Don't forget that most
of the book was written around 1998. Yes, 8 years ago.

You can even find examples on my site that use imported functions (which I
will fix, because I frown upon it ). But I doubt you can find a
majority in the perl community that *encourages* the use of imported
functionality.

 

[Edward Elliott]

Yeah, sure, all over.

125 occurences in 78 recipes. Sure looks like all over to me.

Maybe check the book again. It is used in some
examples, sure. And it even explains how it works.

Yep, 125 times. In 78 recipes. Out of 105 total recipes with 'use'. I'd
say a 3:1 ratio is pretty strong encouragement.

Don't forget that most
of the book was written around 1998. Yes, 8 years ago.

Doesn't matter. It's still the standard example reference. People use it
heavily. They don't magically know what parts are now deprecated.

You can even find examples on my site that use imported functions (which I
will fix, because I frown upon it ). But I doubt you can find a
majority in the perl community that *encourages* the use of imported
functionality.

I can readily believe that the "community" frequenting the newsgroups,
mailing lists, and blogs don't encourage it anymore. But that's a tiny
fraction of all perl programmers, and most of them have no exposure to this
little clique. For many people, whatever the cookbook says goes. If it's
wrong, update it.

 

[John Bokma]

125 occurences in 78 recipes. Sure looks like all over to me.


Yep, 125 times. In 78 recipes. Out of 105 total recipes with 'use'.
I'd say a 3:1 ratio is pretty strong encouragement.


Doesn't matter.

Yes it matters. 8 years is a lot of time in IT. In 1998 Perl5.005 was
announced (22 July). [1] Which should tell you a lot if you indeed have
3 years of Perl skills.

It's still the standard example reference. People
use it heavily. They don't magically know what parts are now
deprecated.

If they are serious with their study they know that a 8 year old book is
hardly up to date. I tend to take most IT related books I use that are
older then 3-4 years with quite a grain of salt. I am not going to take
an 8 year old Java CookBook very seriously for example.

I can readily believe that the "community" frequenting the newsgroups,
mailing lists, and blogs don't encourage it anymore. But that's a
tiny fraction of all perl programmers, and most of them have no
exposure to this little clique.

Pfft, you are just guessing around now.

For many people, whatever the
cookbook says goes. If it's wrong, update it.

Well, contact the authors or O'Reilly. Seriously, are you using 8 year
old Python recipes without thinking?


[1] http://history.perl.org/PerlTimeline.html

 

[Edward Elliott]

A little out-of-order execution seems useful here.

Pfft, you are just guessing around now.

How many organizations have you worked at? How much exposure to coders
whose daily job description doesn't include the word programming? I've
been at Fortune 100 companies with thousands of programmers and support
staff, and at startups with a half dozen employees. I've been employed at
3 universities from small to huge. I know full-time software developers,
software testers, sys admins, network admins, managers, web developers,
graphic artists, physics researchers, bioinformatics researchers,
instructors, librarians, consultants, contractors, hardware engineers,
forensic analysts, and even a law professor. They all code perl to some
degree. Many of them don't even know what a newsgroup is, and you can have
their cookbook when you pry it from their cold dead hands. Guessing?
Hardly. Just not trapped in your insular world of who makes up the perl
"community".

Doesn't matter.

Yes it matters. 8 years is a lot of time in IT. In 1998 Perl5.005 was
announced (22 July). [1] Which should tell you a lot if you indeed have
3 years of Perl skills.

Tell that to everyone who relies on the cookbook. It gets their job done.
They don't care if it was written in the dark ages. Until a replacement
comes along, that's what they'll use. Of course Perl itself has moved on,
that's not the point.

If they are serious with their study they know that a 8 year old book is
hardly up to date. I tend to take most IT related books I use that are
older then 3-4 years with quite a grain of salt. I am not going to take
an 8 year old Java CookBook very seriously for example.

Many/most people aren't "serious with their study" of perl. They just want
to get things done. Perl 5 works for them now, it will work for them 10
years from now. Unless something significantly better comes along to
justify the cost of switching, they'll stick with what they know. And that
includes perl 6. One would hope an updated cookbook is out before then.
Or maybe it's not needed because the current one works just fine.

Time isn't a great measure of language change. A C++ book from 8-10 years
ago is just as good as one today in most respects (maybe exceptions and
templates are a bit underused). A C book from 20 years ago is perfectly
fine for most purposes (how well does K&R stand the test of time?). C99
and C++0x aren't revolutionary changes (ISO committees frown on such
things). God only knows how far back useful LISP resources go.


I'm not arguing what best practices the hardcore community recommends.
Many/most perl programmers aren't part of that community, their only
exposure is the perl books (especially the cookbook), and they'll do
whatever it says in there. Call perl a victim of its own success (a nice
position to be in).

Well, contact the authors or O'Reilly.

Sorry, I've got a bad case of "not my problem".

Seriously, are you using 8 year
old Python recipes without thinking?

Sure, if they do the job and an updated reference isn't handy.

 

[John Bokma]

A little out-of-order execution seems useful here.

No, not interested in a pissing contest. Your statement that the Perl
community encourages importing is *encouraged* (over using OO without
importing) is false.

 

[Edward Elliott]

No, not interested in a pissing contest. Your statement that the Perl
community encourages importing is *encouraged* (over using OO without
importing) is false.

The cookbook says otherwise. So it depends how you define community.

You lecturing people on pissing contests, that's rich. Nice way to duck the
issue and sound like a winner. Wake me when you decide to address the
substance of my arguments.

 

[George Sakkis]

The cookbook says otherwise. So it depends how you define community.

You lecturing people on pissing contests, that's rich. Nice way to duck the
issue and sound like a winner. Wake me when you decide to address the
substance of my arguments.

Not trying to be as ass, but can you take this offline or at least in a
perl newsgroup ? Arguing on a particular fact or speculation about the
perl community is rather unproductive and offtopic for a python
newsgroup.

Thanks,
George

 

[Larry Bates]

This is just anecdotal, but I still find it interesting. Take it for what
it's worth. I'm interested in hearing others' perspectives, just please
don't turn this into a pissing contest.

I'm in the process of converting some old perl programs to python. These
programs use some network code and do a lot of list/dict data processing.
The old ones work fine but are a pain to extend. After two conversions,
the python versions are noticeably shorter.

The first program does some http retrieval, sort of a poor-man's wget with
some extra features. In fact it could be written as a bash script with
wget, but the extra processing would make it very messy. Here are the
numbers on the two versions:

Raw -Blanks -Comments
lines chars lines chars lines chars
mirror.py 167 4632 132 4597 118 4009
mirror.pl 309 5836 211 5647 184 4790

I've listed line and character counts for three forms. Raw is the source
file as-is. -Blanks is the source with blank lines removed, including
lines with just a brace. -Comments removes both blanks and comment lines.
I think -Blanks is the better measure because comments are a function of
code complexity, but either works.

By the numbers, the python code appears roughly 60% as long by line and 80%
as long by characters. The chars percentage being (higher relative to line
count) doesn't surprise me since things like list comprehensions and
explicit module calling produce lengthy but readable lines.

I should point out this wasn't a straight line-for-line conversion, but the
basic code structure is extremely similar. I did make a number of
improvements in the Python version with stricter arg checks and better
error handling, plus added a couple minor new features.

The second program is an smtp outbound filtering proxy. Same categories as
before:

Raw -Blanks -Comments
lines chars lines chars lines chars
smtp-proxy.py 261 7788 222 7749 205 6964
smtp-proxy.pl 966 24110 660 23469 452 14869

The numbers here look much more impressive but it's not a fair comparison.
I wasn't happy with any of the cpan libraries for smtp sending at the time
so I rolled my own. That accounts for 150 raw lines of difference. Another
70 raw lines are logging functions that the python version does with the
standard library. The new version performs the same algorithms and data
manipulations as the original. I did do some major refactoring along the
way, but it wasn't the sort that greatly reduces line count by eliminating
redundancy; there is very little redundancy in either version. In any
case, these factors alone don't account for the entire difference, even if
you take 220 raw lines directly off the latter columns.

The two versions were written about 5 years apart, all by me. At the time
of each, I had about 3 years experience in the given language and would
classify my skill level in it as midway between intermediate and advanced.
IOW I'm very comfortable with the language and library reference docs (minus
a few odd corners), but generally draw the line at mucking with interpreter
internals like symbol tables.

I'd like to here from others what their experience converting between perl
and python is (either direction). I don't have the sense that either
language is particularly better suited for my problem domain than the
other, as they both handle network io and list/dict processing very well.
What are the differences like in other domains? Do you attribute those
differences to the language, the library, the programmer, or other
factors? What are the consistent differences across space and time, if
any? I'm interested in properties of the code itself, not performance.

And just what is the question to the ultimate answer to life, the universe,
and everything anyway?

Sorry, I don't buy this. I can write REALLY short programs that don't handle
exceptions, don't provide for logging for debugging purposes, don't allow
for future growth, etc. I find that 60% of my code has nothing to do with
the actual algorithm or function I'm trying to accomplish. It has more to
do with handling user's bad input, exceptions, recovering from hardware or
communications failures, etc. Inexperienced programmers can write some
pretty short programs that get the job done, but can't handle the real world.

Also, many years ago I wrote a number of applications in APL. We often
referred to programs written in APL as "write only code" because going back
to read what you had written after-the-fact was very hard. You could write
in one line of APL what takes 1000's of lines of C or even Python and it was
pretty efficient also (for applications that needed to manipulate vectors or
matrices).

I understand what you are trying to say, but I can't support your conclusions
as presented.

-Larry Bates

 

[John Bokma]

You lecturing people on pissing contests, that's rich. Nice way to
duck the issue and sound like a winner.

Then you've missed what a discussion really is. It's not about winning,
it's about learning. Sadly you missed that point.

Wake me when you decide to address the substance of my arguments.

I have done so. If you don't consider it enough and think that code in a
book based on 8+ year old modules is the current state of Perl
programming, fine with me.

As a final word, I quote from chapter 12.10 of Learning Perl Objects,
References, & Modules [1]:

"As seen earlier, *the normal means* of using an object-oriented module
is to call class methods and then methods against instances
resulting from constructors of that class. This means that an OO
module *typically exports nothing*, ..."


Which is in general recommended as the book one has to read after Learning
Perl (IIRC it will be renamed in a next edition to reflect this).

 

[John Bokma]

Not trying to be as ass, but can you take this offline or at least in
a perl newsgroup ? Arguing on a particular fact or speculation about
the perl community is rather unproductive and offtopic for a python
newsgroup.

Use a real Usenet client, and you can make it skip a thread.

Like I said, I am not interested in a(n OT) pissing contest, but when
people make silly statements like Edward did, I think its a good thing to
point out that import is *not* encouraged by the Perl community. It has
been used (abused IMO) a lot in the past, and yes, those modules are still
around.

But in general OO is perferred, except in those cases that import extends
the language with for example constructs like try / catch or case etc.

 

[Edward Elliott]

Not trying to be as ass, but can you take this offline or at least in a
perl newsgroup ? Arguing on a particular fact or speculation about the
perl community is rather unproductive and offtopic for a python
newsgroup.

No offense taken. It's definitely OT. I left it here because 1)
comp.lang.python seems pretty lax about going OT when it's related to the
thread, which in this case it was, and 2) the general discussion about what
constitutes a community seemed kinda useful.

That said, I definitely think the discussion has run its course. It's
getting dangerously close to flaming at this point, which indicates it's
time to go offline.

 

[Edward Elliott]

Sorry, I don't buy this. I can write REALLY short programs that don't
handle exceptions, don't provide for logging for debugging purposes, don't
allow
for future growth, etc. I find that 60% of my code has nothing to do with
the actual algorithm or function I'm trying to accomplish. It has more to
do with handling user's bad input, exceptions, recovering from hardware or
communications failures, etc.

Wow, only 60%, I'm surprised it's that low . When I say the algorithms
are roughly equivalent, I'm including the amount of input verification and
error checking that they do. To me, that's part of the algorithm.

Inexperienced programmers can write some
pretty short programs that get the job done, but can't handle the real
world.

Tell me about it. I've taught intro comp sci, things can get real ugly real
quick.

Also, many years ago I wrote a number of applications in APL. We often
referred to programs written in APL as "write only code" because going
back
to read what you had written after-the-fact was very hard. You could
write in one line of APL what takes 1000's of lines of C or even Python
and it was pretty efficient also (for applications that needed to
manipulate vectors or matrices).

Of course. Comparing line counts between assembly and Prolog is pretty
useless given the vast discrepancy in their expressive power. Perl and
Python are roughly comparable in expressiveness, so it doesn't seem
unreasonable to compare their line counts. It might not tell you much,
there are undoubtedly better comparisons to make, but I don't think it's
grossly unfair the way you suggest. I'm all ears if you have another
metric I can test as easily.

I understand what you are trying to say, but I can't support your
conclusions as presented.

What would those be? I tried hard not draw any conclusions. I just want to
see how other people's data compares to mine.

 

[George Sakkis]

Use a real Usenet client, and you can make it skip a thread.

That's funny, coming from the same guy that's been harassing Xah for OT
posts. Oh, the irony.

George

 

[John Bokma]

That's funny, coming from the same guy that's been harassing Xah for OT
posts. Oh, the irony.

Xah was harassing 5 groups with a cross post without setting a follow-up
to for the sole purpose of spamvertizing his website [1] repeatedly.

If you are that clueless, don't post.


[1] He is looking for another hoster btw.

 

[George Sakkis]

That's funny, coming from the same guy that's been harassing Xah for OT
posts. Oh, the irony.

Xah was harassing 5 groups with a cross post without setting a follow-up
to for the sole purpose of spamvertizing his website [1] repeatedly.

And somehow this made all real usenet clients unable to skip his
thread.

If you are that clueless, don't post.

I will seriously ponder upon this, rest assured.

[1] He is looking for another hoster btw.

This must feel really empowering huh ?

George

 

[John Bokma]

[ Xah Lee ]

[1] He is looking for another hoster btw.

This must feel really empowering huh ?

I am sure I've had quite some help. Also, you made quite a mistake. I have
0 power, I just reported what I saw: repeatedly cross posting to 5 groups
for the sole purpose of trolling and spamvertizing a website. And I am
afraid that 5 is a limit set by Google Groups, not by your kook buddy.

Funny though, how you have a problem with a thread that side steps to Perl
only for 4 or 5 postings, but have no problem with a hit & run post in 5
groups to spamvertize a site.

Have fun with the pondering btw.