Motivation of software professionals

[Seebs]

No, the engineering mindset is that a thing is what it's been validated by
testing to be. If it works but you haven't proven it works, then it
doesn't work.

Proof and testing are not the same thing. Ideally, you'd both prove and test
that something works.

You could see qualifications as being the HR equivalent of
testing.

I'd see them as the HR equivalent of abstract proof without any testing.
You test an engineer by having the engineer build stuff and then see whether
it works (both proof and test). Merely knowing that the engineer has
"qualifications" is like knowing that, on paper, a design "works".

The minor problem here is that no *software* engineering qualifications
are worth shit, because there isn't really such a thing as software
engineering, but that's a different debate.

I don't buy it. Software engineering isn't the exact same kind of thing
as structural engineering is, or the exact same kind of thing as electrical
engineering is, but then, those aren't quite exactly the same kind of thing
either. There is an overall pattern to the things that get labeled
engineering, and it doesn't require that every instance of it have precisely
the same traits in every last respect.

-s

 

[MarkusSchaber]

Hi,

I don't buy it.  Software engineering isn't the exact same kind of thing
as structural engineering is, or the exact same kind of thing as electrical
engineering is, but then, those aren't quite exactly the same kind of thing
either.

I personally think that the main difference between "Software
Development" and classical "engineering" is the level of abstraction.
Software development tends to have far more levels of abstraction, and
even their lowest levels of abstraction build on top of those levels
provided by the electrotechnical engineers.

http://www.artima.com/weblogs/viewpost.jsp?thread=255898 is also a
good reading on why software development is neither engineering nor
mathematics.

Markus

 

[Seebs]

I personally think that the main difference between "Software
Development" and classical "engineering" is the level of abstraction.
Software development tends to have far more levels of abstraction, and
even their lowest levels of abstraction build on top of those levels
provided by the electrotechnical engineers.

This is true. However, I don't think that makes it "not really engineering",
any more than using a shovel means you're not *really* digging, just using
a tool provided by someone who makes digging implements.

Of course... It would make sense for me to think that way. Since I do
software development, and that means I spend a lot of time working with
many levels of abstraction, I tend to think that abstractions built on
other things are not all that different from the things they're built on.

-s

 

[Malcolm McLean]

Arved Sandstrom wrote:

As long as we disclaim all liability and give no warranties for the
solutions/products we build, SD cannot be an engineering field and the
term "software engineer" remains as an oxymoron.

Basically no-one knows how to built bug-free software, so the lability
exclusions are necessary. They would be commercial suicide in any
other field. That doesn't mean there is no such thing as software
engineering, only that it is new and undeveloped. Boilers used to
regularly explode at the beginning of the industrial revolution, now
such accidents are almost unheard of.

 

[Arved Sandstrom]

Basically no-one knows how to built bug-free software, so the lability
exclusions are necessary. They would be commercial suicide in any
other field. That doesn't mean there is no such thing as software
engineering, only that it is new and undeveloped. Boilers used to
regularly explode at the beginning of the industrial revolution, now
such accidents are almost unheard of.

It's not a question of bug _free_ software. There aren't any other
fields I can think of where it's possible to get away with no liability
simply by claiming that it's impossible to achieve perfection.

It's also not entirely an issue of software "engineering" being an
infant field. The fact is that there exist adequate processes that apply
to every stage of the software lifecycle, and most software shops only
pay lip service to some or all of them. Doing proper requirements
analysis is not "undeveloped". Doing proper design is not "undeveloped".
Doing proper implementation in your languages of choice is not
"undeveloped". And doing proper testing and QA/GC is not "undeveloped".

In other words, we have adequate processes available but tend not to
adopt them. And _then_ because the products are seriously flawed we
disclaim liability because the products are in poor shape. That whole
mindset is not a problem that's going to be fixed by pushing for a
software engineering profession, because the desire for such a
professional status follows from a mindset that already follows the
basic principles in the first place.

We need to get pushed from the outside, by purchasers of our software.
Unfortunately that hasn't happened.

AHS

 

[Michael Foukarakis]

Basically no-one knows how to built bug-free software, so the lability
exclusions are necessary.

Nobody knows how to build earthquake-immune buildings, yet engineers
give certain guarantees. When those are failed to be met, (s)he is
held liable. Maybe it's about time some "software engineers" were held
liable for their unreliable code in the same way.

They would be commercial suicide in any
other field.

I don't see structural engineering or computer hardware engineering
going anywhere anytime soon..

That doesn't mean there is no such thing as software
engineering, only that it is new and undeveloped. Boilers used to
regularly explode at the beginning of the industrial revolution, now
such accidents are almost unheard of.

That's absolutely right. But we can't sit and wait for software
development to be refined on its own, we should probably do something
about it. Collectively or whatnot.

 

[Malcolm McLean]

In other words, we have adequate processes available but tend not to
adopt them. And _then_ because the products are seriously flawed we
disclaim liability because the products are in poor shape.

We need to get pushed from the outside, by purchasers of our software.
Unfortunately that hasn't happened.

Software management is not so stupid. If adequate procedures were
available that could ensure bug-free software, at reasonable cost and
time, they they would have been adopted. Except in a few areas
customers would soon shy away from 'no warrantry including the implied
warrantry of suitability for any particular purpose' products.

The fact is that many many formal methods are in existence. Some of
them might work, to some extent, and in some circumstances. But none
have really proved themselves when it comes to the acid test of
developing real software for non-trivial projects.

 

[John Koy]

It's not a question of bug _free_ software. There aren't any other
fields I can think of where it's possible to get away with no liability
simply by claiming that it's impossible to achieve perfection.

Exactly. Engineering is about measurable outcomes, quantification.
What's the equivalent of "this building can withstand a quake of
magnitude 7.5 for 30 seconds" in software? Can any of us state "this
software will stand all virus attacks for 12 months" or "this software
will not crash for 2 years, and if it does your loss won't exceed 20% of
all digital assets managed by it" ?

We can't even guarantee that it won't crash tomorrow, why? Well, for me,
because the underlying platform (OS/JRE/CLR/VM/etc) does not give me any
guarantees. I cannot build any engineering product on top of that, no
matter what process I employ.

Engineering is not about "during", it's about "after": accountability,
liability, warranties, hence insurability. And these shape how the
process of "during" must be. Without them, it's just some monkey
business, hence SD.

 

[Registered User]

It's also not entirely an issue of software "engineering" being an
infant field. The fact is that there exist adequate processes that apply
to every stage of the software lifecycle, and most software shops only
pay lip service to some or all of them. Doing proper requirements
analysis is not "undeveloped". Doing proper design is not "undeveloped".
Doing proper implementation in your languages of choice is not
"undeveloped". And doing proper testing and QA/GC is not "undeveloped".

In other words, we have adequate processes available but tend not to
adopt them. And _then_ because the products are seriously flawed we
disclaim liability because the products are in poor shape. That whole
mindset is not a problem that's going to be fixed by pushing for a
software engineering profession, because the desire for such a
professional status follows from a mindset that already follows the
basic principles in the first place.

We need to get pushed from the outside, by purchasers of our software.
Unfortunately that hasn't happened.

All good points but the 'we' gives me trouble. Weak project management
is the real 'we'.

<rant>
The SDLC should not be determined by the calendar, don't ask how long
'it' will take after promising a due date plucked out of thin air.
Images of what a web app should look like doesn't qualify as a design.
A design document shouldn't contain contradictory requirements and
business rules shouldn't describe how the UI is to function.
Re-usability doesn't end at the UI. Security shouldn't be an
afterthought. Requirements shouldn't disappear solely because the
completion date is nearing, maybe they shouldn't have been
requirements initially. Success shouldn't be based upon meeting an
arbitrary date. Define "forward compatibility" before making it a
requirement. Understand the differences between requirements, features
and chrome. ... KISS
</rant>

The are many reasons why over-promised and under-delivered appears to
be the de facto standard. Rather than continue to rant, let me point
to this article.
http://www.infoworld.com/d/adventur...thats-train-wreck-waiting-happen-477?page=0,0
or
http://tinyurl.com/it-train-wreck

regards
A.G.

 

[Wojtek]

Arved Sandstrom wrote :

And doing proper testing and QA/GC is not "undeveloped".

This is the equivalent of building a bridge, then driving over it. If
it does not collapse, then proper engineering was used.

If it does collapses, well then we just re-build it and try again.

 

[MarkusSchaber]

Hi,

We can't even guarantee that it won't crash tomorrow, why? Well, for me,
because the underlying platform (OS/JRE/CLR/VM/etc) does not give me any
guarantees. I cannot build any engineering product on top of that, no
matter what process I employ.

And that is the problem with the abstraction levels I mentioned. Even
the OS cannot guarantee anything because our mainstream hardware is
extremely underspecified (e. G. try to get any time guarantees for
hard disk access) and does not guarantee for anything (see the long
errata list of current CPUs, or the statistics about RAM error rates
due to cosmic rays.

 

[Seebs]

And that is the problem with the abstraction levels I mentioned. Even
the OS cannot guarantee anything because our mainstream hardware is
extremely underspecified (e. G. try to get any time guarantees for
hard disk access) and does not guarantee for anything (see the long
errata list of current CPUs, or the statistics about RAM error rates
due to cosmic rays.

Yeah, and if all you have is improvised materials, you can't be sure the
things you build will work. That doesn't mean you're not an engineer. The
sign of being not-an-engineer would be promising that they'll work anyway.

The job of the engineer is to know enough to build something as reliably
as possible with the available tools, and tell you how reliable that is.
Usually, with software, the answer is "not all that reliable". But since
software engineers know that, and tell people that, and give clear evidence
that they've carefully analyzed the components to be able to tell you that,
it sounds to me like they're doing exactly what engineers ought to do.

I think the obsession with whether or not there is liability misses the point;
that's a social convention adopted for fields where the tools are much
simpler, it's nothing to do with the substance of what engineers are doing.

-s

 

[Seebs]

Nobody knows how to build earthquake-immune buildings, yet engineers
give certain guarantees. When those are failed to be met, (s)he is
held liable. Maybe it's about time some "software engineers" were held
liable for their unreliable code in the same way.

Why?

Do you have any evidence to suggest that this kind of liability is actually
reasonable, justified, and/or necessary?

That's absolutely right. But we can't sit and wait for software
development to be refined on its own, we should probably do something
about it. Collectively or whatnot.

I agree. I just don't think that rants about liability or certification
are going to do anything. Neither of those has a thing to do with learning
to write more reliable software.

-s

 

[Lew]

Software management is not so stupid. If adequate procedures were
available that could ensure bug-free software, at reasonable cost and
time, they they would have been adopted. Except in a few areas

Bullshit.

Management doesn't adopt effective practices for a variety of reasons, but the
fact is that far too many projects are managed in fashions contrary to best
practices. It's a combination of application of an inappropriate management
paradigm (factory work vs. talent work), ignorance or disbelief of the
fundamentals, mistrust of the commitment and understanding of the developers,
and a desire to keep the process inefficient in order to collect more money.

The observable fact is that software projects are managed as though management
were stupid. That's why half to two-thirds (depending on whose figures you go
by) of all major software projects never go into production, and so many that
do have major, preventable bugs.

customers would soon shy away from 'no warrantry including the implied
warrantry of suitability for any particular purpose' products.

Adequate procedures are available, or haven't you been studying the subject?

The fact is that many many formal methods are in existence. Some of
them might work, to some extent, and in some circumstances. But none
have really proved themselves when it comes to the acid test of
developing real software for non-trivial projects.

Again, not true. Iterative development alone greatly increases productivity
and quality. Add in other elements of agile programming, or whatever the
rubric is for effective practices these days, and you asymptotically approach
perfection.

The field of software project management is well studied and well documented.
Effective techniques are known and published. And have been for decades.
They simply are not followed.

 

[Volker Borchert]

And within the available budget.

and tell you how reliable that is.

Anyone can build a bridge that stands up. It takes an engineer to build
on that barely stands up.

What about Galloping Gertie?

 

[James Kanze]

Arved Sandstrom wrote:

[...]
Engineering "certification" processes are considerably
better and more comprehensive than anything that most
software developers are ever exposed to. Starting with
education - there's no requirement at all that software
developers have a relevant degree or associate degree, or
indeed any real SD training at all. Try that with
prospective professional engineeers.
It's not just entry-level certification that software
developers lack. It's code of conduct, professional
education, duty to the client, professional discipline and
so forth. These are all standards. In order for software
"engineering" to really be engineering it has to adopt
similar standards.

As long as we disclaim all liability and give no warranties
for the solutions/products we build, SD cannot be an
engineering field and the term "software engineer" remains as
an oxymoron.

But that's really only the case for shrink-wrapped software (and
presumably, it doesn't exclude your legal guarantees). Most of
the projects I've worked on did have guarantees, and contractual
penalties for downtime.

 

[James Kanze]

Arved Sandstrom wrote:

Basically no-one knows how to built bug-free software, so the
lability exclusions are necessary.

Basically no one knows how to build 100% bug-free anything.
Witness Toyota. Globally, in fact, you can probably do better
with software than with most other things. And I've never
worked on a project where there have been liability exclusions
(which probably aren't legal anyway).

They would be commercial suicide in any other field. That
doesn't mean there is no such thing as software engineering,
only that it is new and undeveloped. Boilers used to regularly
explode at the beginning of the industrial revolution, now
such accidents are almost unheard of.

Well written software fails a lot less frequently than
automobile brakes.

 

[James Kanze]

Software management is not so stupid. If adequate procedures
were available that could ensure bug-free software, at
reasonable cost and time, they they would have been adopted.

Guaranteed error-free doesn't exist in any domain. It's not too
difficult to achieve error rates of less than one error per 100
KLoc, however, and at least one shop does less than one error
per million lines of code. Curiously enough, achieving one
error per 100 KLoc typical reduces your development costs. But
for whatever reasons, in many companies, this goes
unnoticed---in the end, if you're selling shrink wrapped
software, development costs are more or less negligible,
compared to marketing costs, so it doesn't matter if you spend a
bit more.

Except in a few areas customers would soon shy away from 'no
warrantry including the implied warrantry of suitability for
any particular purpose' products.

Legally, of course, you have certain rights, regardless of what
the "warrenty" says. It's just that as far as I know, no one
has attempted to enforce them for software.

The fact is that many many formal methods are in existence.
Some of them might work, to some extent, and in some
circumstances. But none have really proved themselves when it
comes to the acid test of developing real software for
non-trivial projects.

Less that one error per million lines of code is achievable.
And I've worked on projects where we had less than one error per
100 KLoc (going into integration).

 

[James Kanze]

Management doesn't adopt effective practices for a variety of
reasons, but the fact is that far too many projects are
managed in fashions contrary to best practices. It's a
combination of application of an inappropriate management
paradigm (factory work vs. talent work), ignorance or
disbelief of the fundamentals, mistrust of the commitment and
understanding of the developers, and a desire to keep the
process inefficient in order to collect more money.

I suspect that it's often a case of management not being able to
be everywhere, and the fact that marketing issues are far more
important than saving a bit of money (while simultaneously
improving quality) in development.

The observable fact is that software projects are managed as
though management were stupid. That's why half to two-thirds
(depending on whose figures you go by) of all major software
projects never go into production, and so many that do have
major, preventable bugs.

I've often wondered where such statistics come from. Over the
last twenty years, only one project I've worked on failed to go
into production, and none of the others had major, preventable
bugs---in at least one, the most serious bug that was found was
a spelling error in a log message, and in one other case, no
errors were found after delivery (but that was a very small
project).

Adequate procedures are available, or haven't you been
studying the subject?

Again, not true. Iterative development alone greatly
increases productivity and quality. Add in other elements of
agile programming, or whatever the rubric is for effective
practices these days, and you asymptotically approach
perfection.

From what I've seen, "agile" programming hasn't really changed
much. In fact, it's often been just a relabeling of the same
old procedures. (That's a common problem with "in"
labels---before agile programming, a lot of projects became "OO"
overnight. With no change in procedures.)

The field of software project management is well studied and
well documented. Effective techniques are known and
published. And have been for decades. They simply are not
followed.

Except where they are. I was a contractor for the last 25 or 30
years, and most of the places I worked did you more or less
effective techniques.

 

[James Kanze]

Exactly. Engineering is about measurable outcomes,
quantification. What's the equivalent of "this building can
withstand a quake of magnitude 7.5 for 30 seconds" in
software?

A contractual indemnity for each second of downtime? (Most of
the projects I've worked on have had such clauses in their
contracts.)

Can any of us state "this software will stand all virus
attacks for 12 months" or "this software will not crash for 2
years, and if it does your loss won't exceed 20% of all
digital assets managed by it" ?

Most of the projects I've worked on have had clauses to the
effect that "we will pay you x Euros per minute downtime". It
seems to be a more or less standard clause.

We can't even guarantee that it won't crash tomorrow, why?
Well, for me, because the underlying platform
(OS/JRE/CLR/VM/etc) does not give me any guarantees. I cannot
build any engineering product on top of that, no matter what
process I employ.

True, the underlying platform is always a risk. But then, you
don't have to push it, and you're normally not the first user of
it, and you've validated it, so you have some confidence that it
won't crash for what you're using it for.