multi-core software

[Lew]

I agree entirely but my statements were about parallelism and not
concurrency. Parallel and concurrent programming have wildly different
characteristics and solutions. I don't believe shared mutable state is
overly problematic in the context of parallelism. Indeed, I think it is
usually the best solution in that context.

Interesting distinction. Would it be fair to compare concurrent programming
to the bricks used to build the parallel program's edifice?

 

[Arved Sandstrom]

Interesting distinction. Would it be fair to compare concurrent
programming to the bricks used to build the parallel program's edifice?

Way too much of a fine distinction. While they are in fact different,
the point of concurrent programming is to structure programs as a group
of computations, which can be executed in parallel (however that might
actually be done depending on how many processors there are). Parallel
computing means to carry out many computations simultaneously. These are
interleaved definitions. And they are *not* wildly different.

If you talk about shared mutable state, it is not as easy to use as Dr
Harrop seems to think it is. Maybe in his experience it has been, but in
general it's no trivial thing to manage. Lew, you probably summarized it
best a few posts upstream.

AHS

 

[Dave Angel]

Those good old days never existed. Those manuals never accounted for
things that affected timing even then, like memory latency or refresh
time. SRAM cache made things worse, since the published timings never
mentioned cache-miss delays. Though memory cache might seem a recent
innovation, it's been around a while. It would be challenging to find
any published timing since the commercialization of computers that
would actually tell the cost of any loop.

Things got worse when chips like the '86 family acquired multiple
instructions for doing loops, still worse when pre-fetch pipelines
became deeper and wider, absolutely Dark Art due to multi-level memory
caches becoming universal, and
throw-your-hands-up-and-leave-for-the-corner-bar with multiprocessor
NUMA systems. OSes and high-level languages complicate the matter -
you never know how much time slice you'll get or how your source got
compiled or optimized by run-time.

So the good old days are a matter of degree and self-deception - it
was easier to fool ourselves then that we could at least guess timings
proportionately if not absolutely, but things definitely get more
unpredictable over evolution.

Nonsense. The 6502 with static memory was precisely predictable, and
many programmers (working in machine language, naturally) counted on
it. Similarly the Novix 4000, when programmed in its native Forth.

And previous to that, I worked on several machines (in fact, I wrote the
assembler and debugger for two of them) where the only variable was the
delay every two milliseconds for dynamic memory refresh. Separate
control memory and data memory, and every instruction precisely
clocked. No instruction prefetch, no cache memory. What you see is
what you get.

Would I want to go back there? No. Sub-megaherz clocks with much less
happening on each clock means we were operating at way under .01% of
present day.

 

[Arved Sandstrom]

No. Concurrent programming is about interleaving computations in order to
reduce latency. Nothing to do with parallelism.

Jon, I do concurrent programming all the time, as do most of my peers.
Way down on the list of why we do it is the reduction of latency.

AHS

 

[rossberg]

That actually didn't require predictable timing.  You could tell the
video chip to send you an interrupt when it got to a given scan line.  I
used this myself.  

I don't know what Elite did, but I know for sure that it was a common
trick on the Atari ST to switch color palettes or graphics mode at a
fixed point *in each single scan line* to get more colors, or display
graphics on the screen borders. That required "synchronous
programming", i.e. counting clock cycles of machine instructions such
that for every point in the program you knew exactly where the
electron ray would be.

The Atari ST had an M68000 with exactly 8 MHz, which made this
possible. There were no caches in those times, and clock cycles were
entirely predictable.

 

[Piet van Oostrum]

By the way, there is a series of articles about concurrency on ACM Queue
which may be interesting for those participating in or just following
this discussion:

http://queue.acm.org/listing.cfm?it...ist&filter=Concurrency&page_title=Concurrency

Here is one introductory paragraph from one of the articles:

Parallel programming poses many new challenges to the developer, one of
which is synchronizing concurrent access to shared memory by multiple
threads. Programmers have traditionally used locks for synchronization,
but lock-based synchronization has well-known pitfalls. Simplistic
coarse-grained locking does not scale well, while more sophisticated
fine-grained locking risks introducing deadlocks and data races.
Furthermore, scalable libraries written using fine-grained locks cannot
be easily composed in a way that retains scalability and avoids deadlock
and data races.

 

[Paul Wallich]

I don't know what Elite did, but I know for sure that it was a common
trick on the Atari ST to switch color palettes or graphics mode at a
fixed point *in each single scan line* to get more colors, or display
graphics on the screen borders. That required "synchronous
programming", i.e. counting clock cycles of machine instructions such
that for every point in the program you knew exactly where the
electron ray would be.

The Atari ST had an M68000 with exactly 8 MHz, which made this
possible. There were no caches in those times, and clock cycles were
entirely predictable.

The usual trick for these machines was an exact multiple of the NTSC
"color clock", which was approx 3.58 MHz. The 8-bit atari video games
and home computers all used this technique, as did the C-64/128.
68000-based machines (such as the ST and the Amiga) could not only
exploit that synchrony, they could also (this was the days before memory
wall) exploit the fact that a 680x0 typically accessed memory only once
every 4 clock cycles to do DMA from the same memory when the CPU wasn't
using it. High display resolutions would lock the processor out of RAM
except during blanking intervals. (Talk about contention and hot spots.)

Figuring out how to reuse resources most effectively was pretty much the
same as the register-allocation problem for compilers, and was sometimes
solved using the same kinds of graph-coloring algorithms...

 

[Seamus MacRae]

By the way, there is a series of articles about concurrency on ACM Queue
which may be interesting for those participating in or just following
this discussion:

http://queue.acm.org/listing.cfm?it...ist&filter=Concurrency&page_title=Concurrency

Here is one introductory paragraph from one of the articles:

Parallel programming poses many new challenges to the developer, one of
which is synchronizing concurrent access to shared memory by multiple
threads. Programmers have traditionally used locks for synchronization,
but lock-based synchronization has well-known pitfalls. Simplistic
coarse-grained locking does not scale well, while more sophisticated
fine-grained locking risks introducing deadlocks and data races.
Furthermore, scalable libraries written using fine-grained locks cannot
be easily composed in a way that retains scalability and avoids deadlock
and data races.

Is that the one about transactional memory?

 

[Piet van Oostrum]

SM> Is that the one about transactional memory?

Yes

 

[toby]

No. Most programmers still care about performance

Frequently when they shouldn't.

and performance means
mutable state.

Hm, not sure Erlangers would wholly agree.

 

[George Neuner]

Hm, not sure Erlangers would wholly agree.

Erlang uses quite a bit of mutable state behind the scenes ... the
programmers just don't see it.

George

 

[Dimiter \malkia\ Stanev]

Erlang uses quite a bit of mutable state behind the scenes ... the

programmers just don't see it.

George

Heh... "The CPUs use quite a bit of mutable state behind the scenes ...
the programmers just don't see it."

Actually with CPU they see it more, than... say Erlang (that's why you
need to use fences/barriers/locked access here and there).

 

[Emile van Sebille]

On Jun 7, 2:41 pm, Jon Harrop <[email protected]> wrote:
Frequently when they shouldn't.

I disagree. A lot of software is still far too slow because the programmers
failed to pay attention to performance. [/QUOTE]

For a properly written spec, performance is spec'd and paid for.

Blogspot in Firefox being one
example: it can barely keep up with my typing!

There are so many ways that's not the program.

Performance,accuracy,cost -- pick two sacrifice one.

Emile

 

[Arved Sandstrom]

What is higher on the list?

Correctness.

I'm not being facetious. I write applications that run on application
servers, and from time to time I have had to write various special
purpose servers. This kind of programming is all about managing
concurrent execution of computations. The overarching concern is
reliability and correct function. For many corporate situations, even
with hundreds of users, the actual load at any instant is low enough
that the various servers involved are nowhere close to being stressed
out - performance is a secondary issue.

AHS

 

[Jeff M.]

Correctness.

IMO, that response is a bit of a cop-out. Correctness is _always_ most
important, no matter what application you are creating; without it,
you don't have a job and the company you work for goes out of
business.

But, assuming that your program works and does what it's supposed to,
I agree with Jon that performance needs to be right near the top of
the list of concerns. Why? Performance isn't about looking good as a
programmer, or having fun making a function run in 15 cycles instead
of 24, or coming up with some neat bit packing scheme so that your app
now only uses 20K instead of 200K. Performance is - pure and simple -
about one thing only: money.

Programs that use more memory require more money for the hardware of
every user. Programs that run slower eat more time per day. If you
have 100,000 users, all doing an operation once per day that takes 20
seconds, being able to shave 5 seconds off that saves 5.78 man-days of
work. Hell, for some applications, that 20 seconds is just startup
time spent at a splash screen. Just imagine if every Google search
took even 5 seconds to resolve, how much time would be wasted every
day around the world - ignoring the fact that Google wouldn't exist if
that were the case ;-). Obviously Google engineers work incredibly
hard every day to ensure correct results, but performance better be
right up there at the top of the list as well.

Jeff M.

 

[Matthias Blume]

IMO, that response is a bit of a cop-out. Correctness is _always_ most
important, no matter what application you are creating; without it,
you don't have a job and the company you work for goes out of
business.

But, assuming that your program works and does what it's supposed to,
I agree with Jon that performance needs to be right near the top of
the list of concerns. Why? Performance isn't about looking good as a
programmer, or having fun making a function run in 15 cycles instead
of 24, or coming up with some neat bit packing scheme so that your app
now only uses 20K instead of 200K. Performance is - pure and simple -
about one thing only: money.

Programmer time is vastly more expensive than CPU time, so the
money argument often leads to slow ("low performance") solutions as long
as they are "good enough" because developing a faster solution would
mean spending more valuable programmer time at a cost that cannot
be recovered over the life cycle of the product in question.

That being said, there are plenty of situations where performance
obviously does matter a great deal -- as you correctly pointed out.
(It all depends on the above mentioned "product in question" and the
nature of its life cycle.)

Matthias

 

[Paul Rubin]

In other words, without concurrency the latency would be so high
that you would consider the program to be wrong. However you cut it,
the real reason is latency.

I don't think that follows, if there is two-way communication and
dependency between the servers, combined with lack of control over
when any particular server decides to initiate an outgoing request.
Stuff may have to happen concurrently to avoid complete deadlock.

 

[Seamus MacRae]

IMO, that response is a bit of a cop-out. Correctness is _always_ most
important, no matter what application you are creating; without it,
you don't have a job and the company you work for goes out of
business.

And when, exactly, did Microsoft go out of business? I hadn't heard it
mentioned in the news.

 

[Jeff M.]

And when, exactly, did Microsoft go out of business? I hadn't heard it
mentioned in the news.

Touche.

Jeff M.

 

[Dimiter \malkia\ Stanev]

IMO, that response is a bit of a cop-out. Correctness is _always_ most
important, no matter what application you are creating; without it,
you don't have a job and the company you work for goes out of
business.

PC / Video Games definitely fall out of the correctness. As long as the
game does not crash your XBOX/PS3/Whatever for certain amount of time,
and behaves well then, it's fine.

Bugs are already part of the "genre".

In reality you can't ship on time, there are always BUGS

Most important thing in games is (at least for large percent of them)
speed of graphics - fluid 60fps, or stable 30fps.