application and web app technologies

[cartercc]

January, 2006.

I do not intend to start any kind of flame war, but only to seek advice
about different technologies concerning which I am mostly ignorant.

I am the database manager for a unit of a major state university. A
part of my job includes building database access and web enabled
applications for student services, faculty, and staff. These include
both information gathering applications and dynamic updates from our
big database to other applications. About 90 percent of our current
programming is in Perl. Of the rest, some is in ColdFusion, some is in
VB6, and we may have snippets in other languages. I might add that our
WWW pages are all in ColdFusion, which we like very much and have no
interest in changing.

I have been tasked by me IT department with investigating different
technologies for what will be a total rewrite and major update of our
applications. The problem with Perl is that it seems dowdy and old
fashioned, and that we never really investigate alternatives. We just
fell into Perl because that's what people knew. Also, we have had some
staff changes, and updating Perl code, some of which is years old, has
proved to be a real nightmare. Perl works great! ... but trying to read
and modify someone else's code, or even your own, is pretty darn tough.

A real important part of this is database connectivity. We use a number
of different databases, Access, SQL Server, Datatel (the big University
DB), PostgreSQL (my favorite), MySQL, and a couple of others.

Here is where we currently are:

* Java/JSP -- We have already made the decision to go with Java, but
we haven't started with it, and have not committed to Java.
* Python -- Some of us have had limited experience with Python.
* Ruby -- We have a Ruby advocate here, but no one knows anything
about it.
* .NET/ASP -- We have a MS shop, I think that we have two UNIX servers
out of several dozen, and we are pretty firmly committed to Windows,
but no one is real excited about .NET, and the chances that we will
choose .NET seem real remote. Are there any advantages to using .NET?
* OO Perl/Perl6 -- Perl has worked real well for us, but we have
doubts that it is the best technology, and we want to make a serious
attempt to look at other things.
* C/C++ -- I mention this only because this is what IT uses. We have
no interest in C/C++, unless it really is the best.

We want something that we can use across the board, from web apps to
sys admin (which is why ColdFusion is not a candidate). I'm not
interested in advocacy, but if anyone has experience in and compare
these technologies, we would be grateful for your experiences.

My apologies for cross posting, I don't do it often, but I'd like to
reach as wide an audience as possible.

CC

 

[Reinder Verlinde]

January, 2006.

I do not intend to start any kind of flame war, but only to seek advice
about different technologies concerning which I am mostly ignorant.
[...]
I have been tasked by me IT department with investigating different
technologies for what will be a total rewrite and major update of our
applications.

From your post, it is not clear what these applications do. That may
hugely influence any advice you can get.

For instance: if you are running weather simulations on your back end,
Fortran and C would probably be good choices for part of your system. If
you are mostly running static web pages, your choice of CMS will
probably have more impact than your choice of scripting language.

Moreover, you do not tell who will do the maintenance on these systems.
If it is "IT", the choice might move towards C/C++ or a C/C++-like
scripting language.

[...]
The problem with Perl is that it seems dowdy and oldfashioned,

Never dismiss anything because it 'seems' bad. Your 'seems
old-fashioned' is someone else's 'proven technology'. You should work on
objectifying this statement (because I am not a Perl fan, I expect that
this will be possible).

A real important part of this is database connectivity. We use a number
of different databases, Access, SQL Server, Datatel (the big University
DB), PostgreSQL (my favorite), MySQL, and a couple of others.

Any decent scripting language will be able to connect to most, if not
all, of these.

* Java/JSP -- We have already made the decision to go with Java, but
we haven't started with it, and have not committed to Java.

If your 'deciding to' does not imply commitment, you have larger
problems then choosing a technology.

* Ruby -- We have a Ruby advocate here, but no one knows anything
about it.

Never trust an advocate who knows nothing about the thing (s)he
advocates.

However, you should really take a look at Ruby on Rails
(www.rubyonrails.org). It is both good under the hood, and well-marketed
(check out some of the videos)

* C/C++ -- I mention this only because this is what IT uses. We have
no interest in C/C++, unless it really is the best.

If your web apps contain stuff that needs high performance and is CPU
bound, you should keep these in as a language. Calling C/C++ code from
any decent scripting language is easy. Also, if you need commitment from
IT, choosing a less than best language may be a good idea.

Having said that, as far as I know, there are good reasons C and C++ are
not really popular for the development of web applications.

We want something that we can use across the board, from web apps to
sys admin

Why would you? programming languages all have their strengths and
weaknesses. Good programmers will be able to choose a midway path
between standardisation on a single language and using the best language
for every task.

Reinder

 

[Matt Garrish]

January, 2006.

* OO Perl/Perl6 -- Perl has worked real well for us, but we have
doubts that it is the best technology, and we want to make a serious
attempt to look at other things.

It might help if you elaborated on what these "doubts" are. It doesn't sound
like you know any of the languages you've listed and are hoping that somehow
you'll find one magical beast by cross-posting to a bunch of groups. I don't
expect you're going to have much luck.

The fact that you list Perl 6 shows you aren't following Perl's development
very closely. Perl 6 is not on the near horizon, and even as an avid Perl
enthusiast I'd say you'd have to be insane to jump on it for production use
as soon as it is.

That said, Perl is still one of the best choices for both Web and admin
scripting, and I don't see that you'd gain anything by rewriting all of your
existing code to Ruby or Python just for the sake of saying you now use Ruby
or Python (not that there's anything wrong with either, but why rewrite code
for the sake of rewriting it?). If you wrote terrible and unreadable Perl
code, what's really going to stop you from writing terrible and unreadable
Ruby and Python code? That's more a statement on your programmers and lack
of in-house style than the language.

C# isn't too bad for Web scripting and quick GUIs, but I've never used it
for admin scripting and the downside is that it takes a lot of effort to do
tasks in .Net that are simple in Perl/Python/Ruby (particularly database
work). I wouldn't use C/C++ for the web, but there's nothing stopping you.

Matt

 

[Roedy Green]

* Java/JSP -- We have already made the decision to go with Java, but
we haven't started with it, and have not committed to Java.
* Python -- Some of us have had limited experience with Python.
* Ruby -- We have a Ruby advocate here, but no one knows anything
about it.

In many shops, perhaps yours too, you are relying on volunteers. The
problem is they suddenly disappear on you leaving no notes. In that
situation, you want to push for as vanilla technology as possible so
that you have maximal chance the next guy will already be familiar
with the tools.

From that point of view Java is a nice choice, as is MySQL and JSP.

 

[Van]

Try matching technologies when building a decision matrix...
For example, if you need the PHP open source pile then MySQL works
well.
If you like SQL Server then ASP.NET or Mono with C# is attractive.
Eliminate contenders that have performance obstacles...
PERL does not multithread, PHP.com holds back the cache goods from
PHP.org.
Find a theme and follow it a logical conclusion, here is my
experience....I was looking for a good JavaScript Library to Ajax my
offering.I found Mochikit to be well documented, cross browser with an
active forum.The author described Mochikit as Python like which caused
me to investigate Python.Terse, compact, easy to embed C/C++ with
multithreading are impressive.ZOPE a python written web server with
ZODB (in-memory db/cache mechanism) sealed the deal for me. Most of
the Zope/Python/SQL code uses PostgreSQL examples.
I am concerned about bandwidth associated with the myriad dev tasks.
A good JavaScript library eliminates major hassles, you may have to
combine some.
SQL is SQL, it comes down to documentation, interface, licensing or
preference.
Server code that does not render standards compliant web pages is a
pain to test.
ZPT from ZOPE solves that problem nicely, with a fast XML page parser.
So prioritize you needs: Labor, Standards, Functionality, Training,
etc.
Then evaluate you stacks based on your priorities.
The last time I was involved with university operations we had unit
record gear.
Good Luck,
Van

 

[ccc]

Thanks for your post. I see from the tone of the replies that I may
have made a 'huge' mistake in using the ngs for research.

From your post, it is not clear what these applications do. That may
hugely influence any advice you can get.

Just about everything. For two cases in point: (1) We have a web
enabled application for faculty members to do things like posts
attencence, print rosters, etc., which is updated four times a day from
the big DB. The big DB uses a product named Datatel, and our webapp
runs on SQL Server. Our Perl script queries Datatel, runs a number of
queries, downloads the data, shakes it a bakes it, and then shoves it
into SQL Server with DTS. This is one of our most important
applications and it is an absolute monster to do manually. (2) We also
run a little app to generate email accounts for new employees, which is
not a big deal unless you happen to be a new employee who doesn't have
an email address. Our applications run the gamut from the large,
critical apps to the convenience ones.

Never dismiss anything because it 'seems' bad. Your 'seems
old-fashioned' is someone else's 'proven technology'. You should work on
objectifying this statement (because I am not a Perl fan, I expect that
this will be possible).

I *am* a Perl fan, but after having looked at scripts someone else
wrote (who is no longer with us) with a view toward updating them, I
have concluded that a quick and dirty scripting language in someone
else's idiom isn't a very good choice institutionally. Which is why I'm
looking at OO Perl.

Moreover, you do not tell who will do the maintenance on these systems.

The database people will maintain the programs, and as you can imagine
these skills are quite varied, which is one reason we want to settle on
one technology we can all use.

If your 'deciding to' does not imply commitment, you have larger
problems then choosing a technology.

Our 'deciding to' was like this: 'Java seems to be hot, so let's all
use it.' Yes, we may have bigger problems, but we still want to commit
to one technology that will do the job.

Never trust an advocate who knows nothing about the thing (s)he
advocates.

I mispoke ... the advocate know a little about Ruby, but none of the
others do, and we don't want to take his word without satisfying
ourselves that what he says is true.

Why would you? programming languages all have their strengths and
weaknesses. Good programmers will be able to choose a midway path
between standardisation on a single language and using the best language
for every task.

We really have a hodgepodge. We have snippets written in ColdFusion,
Perl, VB6, VB7, a little C, a couple of Java apps, and some other
miscellanous stuff. We don't have any 'programmers' on staff. People
have tended to focus on the problem at hand and have used whatever
language they were familier with. We want to move beyond this, and do
some real SW engineering.

From what I have gathered, our first impulse to use Java was probably

the right one, even though it wasn't really based on any kind of
investigation. I just hate to commit to something that I have doubts
about.

CC

 

[cartercc]

It might help if you elaborated on what these "doubts" are. It doesn't sound

like you know any of the languages you've listed and are hoping that somehow
you'll find one magical beast by cross-posting to a bunch of groups. I don't
expect you're going to have much luck.

No, we don't know any of these languages. I'm reasonably competent in
Perl, and I have used some Java and Python (and taught C++ a loooong
time ago but have never actually written any C++). The problem is that
none of us can compare apples to apples, even though we more or less
can do what needs to be done with the tools we know.

I don't expect the 'magical beast.' What I do expect is several posts
along the following lines: 'We faced a similar situation, and used X,
Y, and Z. X proved the best choice because of reasons A, B, and C. The
problem with Y was D and the problem with Z was E.'

That said, Perl is still one of the best choices for both Web and admin
scripting, and I don't see that you'd gain anything by rewriting all of your
existing code to Ruby or Python just for the sake of saying you now use Ruby
or Python (not that there's anything wrong with either, but why rewrite code
for the sake of rewriting it?).

I agree with you about Perl, and CPAN is a fab resource, but the reason
we need to rewrite the code is because (1) it doesn't work (due to
external changes) and (2) it takes us less time to write new routines
that it does to decypher the old ones and modify them. Besides, I work
in an academic setting, and when people ask you what you use, I have
learned to cringe when I reply, 'Perl.'

CC

 

[Matt Garrish]

No, we don't know any of these languages. I'm reasonably competent in
Perl, and I have used some Java and Python (and taught C++ a loooong
time ago but have never actually written any C++). The problem is that
none of us can compare apples to apples, even though we more or less
can do what needs to be done with the tools we know.

I don't expect the 'magical beast.' What I do expect is several posts
along the following lines: 'We faced a similar situation, and used X,
Y, and Z. X proved the best choice because of reasons A, B, and C. The
problem with Y was D and the problem with Z was E.'

But that's what makes it impossible to give you any meaningful advice: every
situation is different. Without being intimate with your architecture and
what exactly web scripting and admin scripting means to you, it's nearly
impossible to give vanilla advice about what language to use. You also need
to bear in mind the skill set at your disposal. If no one knows the language
you want to use, do you have time to account for the learning curve? Do you
really want to try and replace all your programmers?

In Perl's defence, bad programmers write bad Perl code. There is nothing
about the language that makes for unreadable code, only how you choose to
write it. Going OO should be a no-brainer if you stay with Perl (just for
the maintenance savings alone), but regardless of which language you choose
you should have your programmers develop an accepted style (everything from
how to name functions to how to structure your code library). If you let
programmers build their own little universes they will!

I agree with you about Perl, and CPAN is a fab resource, but the reason
we need to rewrite the code is because (1) it doesn't work (due to
external changes) and (2) it takes us less time to write new routines
that it does to decypher the old ones and modify them.

That's again a sign of poor documentation coupled with bad style. It will
always take a while to get back into your code, no matter what language you
use. If you maintain consistency as you go, however, it eases a lot of the
curve when you have to go back. You should probably look at other measures
that involve your programmers more in making the coding a collective
practice (peer review, for example). So long as the focus is constructive,
it will help the group better understand what they should all be striving
for and what they should all be doing. That more than anything will help
prevent you from winding up in the same mess in a few years when you
discover each person has their own coding ideas for whatever language you
opt for.

Matt

 

[ccc]

If no one knows the language

you want to use, do you have time to account for the learning curve? Do you
really want to try and replace all your programmers?

We don't have any 'programmers' on staff. At most, we have several
people writing, maybe, two hours of code a week, with maybe once a
year building an application. We are just your basic IT shop, system
and network administration, hardware, help desk, the web site, and
database administration. This is also the reason for the 'bad code' (
which we have in abundance.) People who are not programmers and whose
job it isn't to program will not write good code. I'm not being
perjorative, just factual.

If you let
programmers build their own little universes they will!

Yeah, well, if you have a database admin writing his scripts, a network
admin writing his scripts, and a couple of floaters trying to fix
things that break, with no one holding the reins, you get little
universes. Which is what happened and which we want to be proactive and
prevent in the future.

You should probably look at other measures
that involve your programmers more in making the coding a collective
practice (peer review, for example). So long as the focus is constructive,
it will help the group better understand what they should all be striving
for and what they should all be doing. That more than anything will help
prevent you from winding up in the same mess in a few years when you
discover each person has their own coding ideas for whatever language you
opt for.

Exactly! And a major decision is deciding on a technology so that we'll
all be using the same thing.

Actually, Java was a pretty easy decision to make, since we already had
a problem with Perl, no one wanted to do C or C++, and no one knows VB,
and most of us had used Java at some point along the way. HOWEVER, we
know we face a learning curve, and we want to get the most bang for the
buck, and we have a good enough handle on our Perl scripts so this
isn't a time critical desicion, so we are just looking.

CC

 

[Harpo]

We don't have any 'programmers' on staff. At most, we have several
people writing, maybe, two hours of code a week,

Fine !

with maybe once a
year building an application. We are just your basic IT shop, system
and network administration, hardware, help desk, the web site, and
database administration. This is also the reason for the 'bad code' (
which we have in abundance.) People who are not programmers and whose
job it isn't to program will not write good code. I'm not being
perjorative, just factual.

Let's be positive : 2 hours of bad code a week is better than 40 hours
of bad code a week.

And what is bad code and what is good code ? Your problem doesn't seem
to be a programming issue, Often, the problem is not at this level,
trying to find 'the good language' is just spending time, there is no
'good language', it is just a thing that doesn't exist in the real
life,

Gathering code to make an heteroclitic system is never a good solution,
threwing heterodoxic code (but maybe good code) to the trashcan is not
a good solution, rewriting in another language is painful and bug
prone, therefore not a good solution if not the worst.

In the real life, there is no good solution but many false problems.
Your problem in not a programming problem but a liability problem, not
seeing this problem will give more problems.

What do you expect ? You crosspost to perl, python, java and ruby NGs,
Do you want me to say 'Ruby is better' ? This would be stupid.

You have Perl code that you seem to find non understandable ? Does it
work ? If it works, it's ok but it would have been better if you
understood why.

My advice : Just don't touch anything.

FU2

 

[l v]

[snip] We are just your basic IT shop, system
and network administration, hardware, help desk, the web site, and
database administration. This is also the reason for the 'bad code' (
which we have in abundance.) People who are not programmers and whose
job it isn't to program will not write good code.

Technology is not your problem just as the above are not the reasons
for the bad code. In my 15 years in IT, I've see bad COBOL, Quickjob,
JCL, Java, shell scripts, DB modeling, Perl, SQL, ABAP, etc. Bad code
moves from one language to another. Procedures such as good coding
standards, peer reviews, adherence to coding standards, etc are the
key.

Yeah, well, if you have a database admin writing his scripts, a network
admin writing his scripts, and a couple of floaters trying to fix
things that break, with no one holding the reins, you get little
universes. Which is what happened and which we want to be proactive and
prevent in the future.

Exactly! And a major decision is deciding on a technology so that we'll
all be using the same thing.

Ah, one language fits all. Perhaps OO COBOL should be considered.

Actually, Java was a pretty easy decision to make, since we already had
a problem with Perl, [snip]

IMO, your problem was not with Perl, it was not having standards,
reviews and quite possibility the staff's unfamiliarity with the Perl
language. Is it unreadable code or code that uses more advanced
techniques than the programmers knowledge?

Len

 

[Juha Laiho]

I *am* a Perl fan, but after having looked at scripts someone else
wrote (who is no longer with us) with a view toward updating them, I
have concluded that a quick and dirty scripting language in someone
else's idiom isn't a very good choice institutionally. Which is why I'm
looking at OO Perl.

This tells a lot more about the original author of the scripts than the
language used. I'm seeing unmaintainable code written in pretty much any
language. Java (and in some ways, especially JSP in the web side) is no
cure for that.

Having said that, it may be easier to write unmaintainable Perl than
unmaintainable Java. The key is to educate (and to a certain limit
restrict) the programmers on what kind of code is acceptable to write.
And this work I think is not that dissimilar between different
languages.

Also, OO is not a magic bullet in making code readable. At worst it
gives the programmer a couple of extra levels of indirection to use
just to complicate the otherwise streamlined program. OO is good
where the data used naturally forms nice object relationships, but
there are a good number of cases where this does not happen (or at
least the relationships do not help that much in solving the actual
problem).

If you really, as you mentioned somewhere, want an all-around-language,
then Perl might be your best bet. Java is nice for web applications,
however I wouldn't use it for sysadmin stuff (or, it could be used
for part, but part is that you need an one-off tool; something you
cook up fast and use once or twice -- or a quick modification for
the existing tools). Also, much of sysadmin (esp. on Unix side) is
processing various text formats, which is one of the great strengths
of Perl. It's not just regexes (which finally are available in Java
as well), but overall facilities of the language.

Then, regarding the performance/efficiency; yes; there are cases where
use of C is justified for performance reasons. This performance
comes at a cost of programmer productivity (and possible higher
rate of certain kinds of errors). Also, in higher-level languages
the libraries are getting pretty-well tuned at least for the general
case, so it may well be that much of the general routines you find
in the library of a given language are of better performance than
something you yourself cook in C.

Another area where C would be useful is for making the necessary
glue to access some backend system (for which only C interface
library is provided) in the higher-level language of your choice.
A good example of this are the various database connectivity modules
in Perl (part Perl, part C; linking to the database client library
code).

So, overall:
- you're a Perl shop - however have lost some of your assets
(knowledgeable personnel)
- one problem may have been too great independence of the developers,
and lacking maintainability guidelines for the created code
- making the hop to another language would be a major hurdle
(possibly giving major gains, though)
- could the same potential gains be realised without doing the
language hurdle? Potentially yes; but would require more-or-less
redesign/rewrite of the whole current environment, so a major
hurdle anyway
- does the current language limit what you can do - doesn't seem so
- what would be gained by another language? For some other languages
it could be easier to find developers -- however, finding competent
developers appears to be hard for any language (competent defined
as "able to produce code that is maintainable in the long run")

.... so, all in all, I wouldn't be that eager in shifting from the
current. Java/JSP (and higher-level frameworks built on top of these)
could be good for web side - but these require quite a lot of knowledge
to do properly (just plugging things from a full Java/J2EE toolbox to
solve your problem could well give you nothing else but horrible
performance problems -- whereas proper use of J2EE technologies could
give you a somewhat elegant solution).

Apologies for omitting completely Python, Ruby, and MS technologies;
I don't know enough of them to do any comparision.