Beyond threads? Better concurrency methods?

A

anmus

Hi All,

The recent IEEE Computer society magazine 'Computer' May 2006 has a
thought-provoking article on threading and concurrency (p. 33).

The author has three points:
1. Threading is an error prone method of parallelizing a program.
Basically, his thesis is that thread packages support non-deterministic
coding and then adds methods of constraining the non-determinism, and
that such methods are brain-twistingly difficult to write and read,
difficult to test, and have latent bugs that don't appear for years
which are correspondingly impossible to duplicate.

2. Better methods of expressing concurrency exist, have been implemented
in many obscure languages, and still don't have mainstream acceptance.

3. The author advocates use of 'coordination' or 'composition' languages
on top of existing general purpose languages to express concurrency.
These coordination languages still have a lot of work yet to be done. My
thought is perhaps Ruby can express concurrency cleanly _without_
needing another language.

I thought this idea might appeal to Matz, language-aficionado that he
is, and that Ruby has demonstrated with Rake and Rails that not having
multiple languages in a development environment has benefits.

My point in posting this message is to ask the Ruby community if it is
worth thinking about laying some foundations in Ruby 2.0 and YARV to
elegantly support other methods of expressing concurrency. Perhaps this
work won't show results until Ruby 3.0, but reserving some keywords in
the grammar and some hooks in the VM may yield dividends in the future.

It is clear to me that single processor machines are becoming quaint,
and that the new norm in desktop machines will be multi-core, multi-chip
SMP and NUMA machines along with clusters for servers.

In this new environment, if Ruby can seamlessly and cleanly take
advantage of available concurrent resources, it will be a huge win for
Ruby over other popular languages. My hope is that the Ruby VM will take
care of each architecture's concurrency ugliness behind the scenes,
leaving the fun stuff in front.

Yes, I'm posting this essentially anonymously. I'm new to Ruby, and
rusty at coding and threading. I'm intrigued by the idea of having fun
again and I want Ruby to be the best language it can be. I did search
the archives for discussions of concurrency and parallelism - I didn't
find very much. I also want to be able to attend Ruby events without
needing a paper bag on my head if it turns out that this is a pointless
post. I trust the community won't flame me too badly.

-AA
 
Y

Yukihiro Matsumoto

Hi,

In message "Re: Beyond threads? Better concurrency methods?"

|The recent IEEE Computer society magazine 'Computer' May 2006 has a
|thought-provoking article on threading and concurrency (p. 33).

|I thought this idea might appeal to Matz, language-aficionado that he
|is, and that Ruby has demonstrated with Rake and Rails that not having
|multiple languages in a development environment has benefits.

Interesting. But I don't have IEEE Computer magazine at hand. Could
anyone point me further information about this 'coordination'?

matz.
 
M

M. Edward (Ed) Borasky

anmus said:
Hi All,

The recent IEEE Computer society magazine 'Computer' May 2006 has a
thought-provoking article on threading and concurrency (p. 33).

The author has three points:
1. Threading is an error prone method of parallelizing a program.
Basically, his thesis is that thread packages support
non-deterministic coding and then adds methods of constraining the
non-determinism, and that such methods are brain-twistingly difficult
to write and read, difficult to test, and have latent bugs that don't
appear for years which are correspondingly impossible to duplicate.

2. Better methods of expressing concurrency exist, have been
implemented in many obscure languages, and still don't have mainstream
acceptance.

3. The author advocates use of 'coordination' or 'composition'
languages on top of existing general purpose languages to express
concurrency. These coordination languages still have a lot of work yet
to be done. My thought is perhaps Ruby can express concurrency cleanly
_without_ needing another language.
Click to expand...
I read the article and somewhat vaguely remember what the author
recommended. I think "still have a lot of work yet to be done" is a
gross understatement. :)
I thought this idea might appeal to Matz, language-aficionado that he
is, and that Ruby has demonstrated with Rake and Rails that not having
multiple languages in a development environment has benefits.

My point in posting this message is to ask the Ruby community if it is
worth thinking about laying some foundations in Ruby 2.0 and YARV to
elegantly support other methods of expressing concurrency. Perhaps
this work won't show results until Ruby 3.0, but reserving some
keywords in the grammar and some hooks in the VM may yield dividends
in the future.
Click to expand...
Uh ... the primitives need to be in the OS for most
"concurrency/parallelism" implementations. Keywords and virtual machines
come after that. And for the primitives to be in the OS, they need to be
in the hardware. The paradigms supported by today's hardware and
operating systems are the paradigms that have a track record for the
most part.
It is clear to me that single processor machines are becoming quaint,
and that the new norm in desktop machines will be multi-core,
multi-chip SMP and NUMA machines along with clusters for servers.
Click to expand...
And the stories I'm seeing in the trade press are that "parallel
programming" is no easier today than it was when Gene Amdahl first
published his law. There aren't any silver bullets.
In this new environment, if Ruby can seamlessly and cleanly take
advantage of available concurrent resources, it will be a huge win for
Ruby over other popular languages. My hope is that the Ruby VM will
take care of each architecture's concurrency ugliness behind the
scenes, leaving the fun stuff in front.
Click to expand...
Strangely enough, I don't recall ever seeing a *real* programming
language, to be distinguished from academic ones, that ever handled
parallelism in a manner other than as calls to run-time libraries. Ruby
already has that.

Well, actually, there was *one* ... Occam for the Transputer. Some
companies actually built products around this, although they were not
economically viable. Ruby seems to be too well established for it to
suffer this unhappy fate.
Yes, I'm posting this essentially anonymously. I'm new to Ruby, and
rusty at coding and threading. I'm intrigued by the idea of having fun
again and I want Ruby to be the best language it can be. I did search
the archives for discussions of concurrency and parallelism - I didn't
find very much. I also want to be able to attend Ruby events without
needing a paper bag on my head if it turns out that this is a
pointless post. I trust the community won't flame me too badly.
Click to expand...
No, it's not a pointless post by any stretch of the imagination. I think
most of us over a certain level of experience in programming have these
dreams. In my own career, so far I've had the dreams of automatically
proving programs correct, widespread adoption of formal semantics in
programming languages, functional languages and programming styles
dominating the practice, literate programming, the ability to write
programs faster, etc. All of these dreams have fallen to the tyranny of
"good enough", and I suspect "seamless supercomputing" is another one.
So I write my code the way I know how, hope that others can read it, try
to keep it simple enough that I can convince myself it's correct, and
try to reserve the time to refactor.
 
M

M. Edward (Ed) Borasky

Yukihiro said:
Hi,

In message "Re: Beyond threads? Better concurrency methods?"

|The recent IEEE Computer society magazine 'Computer' May 2006 has a
|thought-provoking article on threading and concurrency (p. 33).

|I thought this idea might appeal to Matz, language-aficionado that he
|is, and that Ruby has demonstrated with Rake and Rails that not having
|multiple languages in a development environment has benefits.

Interesting. But I don't have IEEE Computer magazine at hand. Could
anyone point me further information about this 'coordination'?

matz.
Click to expand...
It's on line ... try the IEEE "Computer" web site. I saw a link go by on
this list back when it was published.
 
D

Daniel DeLorme

P

Phil Tomson

I read the article and somewhat vaguely remember what the author
recommended. I think "still have a lot of work yet to be done" is a
gross understatement. :)

Uh ... the primitives need to be in the OS for most
"concurrency/parallelism" implementations.
Click to expand...

I've got an SMP kernel.
Keywords and virtual machines
come after that. And for the primitives to be in the OS, they need to be
in the hardware.
Click to expand...

I've got a dual core processor.
The paradigms supported by today's hardware and
operating systems are the paradigms that have a track record for the
most part.
Click to expand...

Some support for parallelism seems to be already in place at that OS
and hardware levels. I think the point of the article (which I only
read a summary of) was that we need better ways of describing
parallelism (better ways than threads).
And the stories I'm seeing in the trade press are that "parallel
programming" is no easier today than it was when Gene Amdahl first
published his law. There aren't any silver bullets.
Strangely enough, I don't recall ever seeing a *real* programming
language, to be distinguished from academic ones, that ever handled
parallelism in a manner other than as calls to run-time libraries. Ruby
already has that.
Click to expand...

In the hardware world there are HDLs (hardware description languages)
which model parallelism using an RTL/dataflow model. Oddly enough,
the hardware folks are trying to figure out how to use C/C++ to model
hardware. I'm wondering if they're going the wrong direction; C/C++
don't seem to be a good fit for hardware design from what I've seen so
far. Maybe we need to inroduce dataflow concepts into general purpose
programming languages. (project plug: See RHDL:
http://rhdl.rubyforge.org/ ).

The basic idea is that in an HDL everything is happening at once; all
statements outside of a process block execute concurrently. Inside a
process they execute as they would in a normal programming language,
but all of the processes are considered to be executing in parallel.
processes get triggered by changes in signals. Of course HDL
simulators often make use of threads or continuations (RHDL uses
continuations which in turn are implemented as threads in Ruby).

Hardware in inherently parallel. You can think of logic gates as
being simple little processors. Outputs change when inputs change.
dataflow. That's why HDLs were developed in the mid 80's to model
hardware.

Phil
 
P

Pit Capitain

Yukihiro said:
In message "Re: Beyond threads? Better concurrency methods?"

|The recent IEEE Computer society magazine 'Computer' May 2006 has a
|thought-provoking article on threading and concurrency (p. 33).

Interesting. But I don't have IEEE Computer magazine at hand. Could
anyone point me further information about this 'coordination'?
Click to expand...

Matz, you can find it online from

http://www.computer.org/portal/site/computer/

Regards,
Pit
 
A

Alex.Dalitz

Strangely enough, I don't recall ever seeing a *real* programming
language, to be distinguished from academic ones, that ever handled
parallelism in a manner other than as calls to run-time libraries. Ruby
already has that.

Well, actually, there was *one* ... Occam for the Transputer. Some
companies actually built products around this, although they were not
economically viable. Ruby seems to be too well established for it to
suffer this unhappy fate.
Click to expand...

I spent some great years programming the Transputer in OCCAM commercially.
It would be good to see some ideas from CSP in Ruby...


Alex.
 
B

Brad Phelan

Perhapps have a look at Fortress from Sun Micro

http://research.sun.com/projects/plrg/faq/index.html

"Fortress is designed to make parallel programming as painless as
possible. Many of the core language constructs in Fortress, including
function argument evaluation and ``for" loops, are parallel by default.
Threads can synchronize using atomic code blocks. Fortress supplements
this implicit fork-join threading model with explicit futures, and
provides programmers with ways to control where particular threads are
run and how large objects are laid out in memory."
 
D

Daniel DeLorme

Pit said:
Wierd. I didn't have to pay for it. I've seen that Srinivas has already
sent you the URL of the article.
Click to expand...

Yeah it's really weird. If you go to the main page, the article is there
available for free. But if you go to past issues > May 2006 you can only
read the summary with an option to buy the article as a pdf. Talk about
consistency...

Daniel
 
M

M. Edward (Ed) Borasky

I spent some great years programming the Transputer in OCCAM commercially.
It would be good to see some ideas from CSP in Ruby...
Click to expand...
And I spent four terrible years at a company called Floating Point
Systems that bet the farm on an Occam/Transputer hypercube and ended up
becoming one of the Portland area's larger disemployers. To be fair, I
liked the Transputer and Occam, but the guys with the big bags of
nickels bought other stuff from other companies.

Anyhow, CSP is "old hat" -- this year's "silver bullet" is the
PI-Calculus, a close relative of Hoare's CSP and a direct descendant of
Milner's CCS. I like the PI-Calculus just as much as I liked CSP, Occam,
Concurrent Pascal, CCS, Linda/Rinda and all the other theoretical
computer science approaches. In my own field, performance engineering, I
like the CCS derivative, Jane Hillston's PEPA. But what are the guys
with the big bags of nickels buying?
 
D

Daniel Martin

M. Edward (Ed) Borasky said:
Uh ... the primitives need to be in the OS for most
"concurrency/parallelism" implementations. Keywords and virtual
machines come after that. And for the primitives to be in the OS, they
need to be in the hardware. The paradigms supported by today's
hardware and operating systems are the paradigms that have a track
record for the most part.
Click to expand...

Well, the Scala language is hosted on the JVM, so has as its
underlying "OS" support only traditional threads. However, because of
the abstrction capabilities in the language, they've got a concurrency
model similar to Erlang's. I definitely think that the Ruby community
should look to see what additional concurrency abstractions can be
swiped from elsewhere.
And the stories I'm seeing in the trade press are that "parallel
programming" is no easier today than it was when Gene Amdahl first
published his law. There aren't any silver bullets.
Click to expand...

"no silver bullets" is different from "this is the best we can do."
Structured programming wasn't the silver bullet it was promoted as at
first, but I doubt anyone thinks it wasn't an improvement.
Strangely enough, I don't recall ever seeing a *real* programming
language, to be distinguished from academic ones, that ever handled
parallelism in a manner other than as calls to run-time
libraries. Ruby already has that.
Click to expand...

Erlang doesn't count? True, I don't know of anyone using it besides
Erikson, but they use it in real commercial products that other
companies pay large amounts of money for.

And what about Ada? I seem to remember that someone somewhere was
building production systems for some rather extreme operating
environments in Ada...

Also, I'd object to the idea that java handles parallelism only with
calls to the run-time library: the concept of different threads,
synchronized blocks, and object monitors is built into the language
syntax even if there is some code in a run-time library to implement
some aspects of java.lang.Thread.
 
R

rmeenaks

M. Edward (Ed) Borasky said:
Anyhow, CSP is "old hat" -- this year's "silver bullet" is the
PI-Calculus, a close relative of Hoare's CSP and a direct descendant of
Milner's CCS. I like the PI-Calculus just as much as I liked CSP, Occam,
Concurrent Pascal, CCS, Linda/Rinda and all the other theoretical
computer science approaches. In my own field, performance engineering, I
like the CCS derivative, Jane Hillston's PEPA. But what are the guys
with the big bags of nickels buying?
Click to expand...


You should take a look at Occam-PI
(http://www.cs.kent.ac.uk/projects/ofa/kroc/) which is a marriage of
PI-Calculus and OCCAM...


Ram
 
T

tsuraan

Erlang doesn't count? True, I don't know of anyone using it besides
Erikson, but they use it in real commercial products that other
companies pay large amounts of money for.
Click to expand...

Erlang's sort of a strange one. Obviously, it can't be called
academic, and the programming model is entirely based on concurrency,
but at the same time, it doesn't support native threading. I've always
thought that is a strange choice. They have really fast user-space
threading, but it will never use multiple processors unless you're
running multiple Erlang VMs, and then you have to explicitly start your
thread on the VM of your choice, from what I understand. Still, I love
the language. I wish ruby allowed overloading of the ! operator, just
so I could implement a ruby thread wrapper that could accept messages
with Erlang syntax :)
 
B

Bob Gustafson

I'm glad you mentioned Occam. I was going to suggest a look at the link;

http://groups.google.com/group/comp...4b7f1cf2425?lnk=st&q=&rnum=4#cd5214b7f1cf2425

This is a little old. I'm glad that the hot bed of Occam programming
(Kent) is still working with it.

Besides it's explicit parallelism, Occam's other claim to fame is its
basis in Formal Methods. Many years ago, silicon chip designs were
'derived' from Occam program descriptions. When INMOS was bought by EMI
and then absorbed into STM, the idea of deriving silicon went silent. My
hypothesis was that STM was using Occam ++ internally as its silver
bullet design methodology. Complex chips need to work with a minimum of
tweeks in the manufacturing process.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

Merge sort and threads 0
Ruby 1.9.x Concurrency 19
Java Tutorial, Concurrency trail 3
Problem with concurrency 5
{Detached} Mysql Gem and beyond: Fixing, Supporting and Celebrating 2
Threads 1
Followup to "Serious concurrency problems on fast systems" 8
Composability and Concurrency and Functional programming 1

Members online

Total: 64 (members: 1, guests: 63)
Robots: 342

Forum statistics

Threads
473,982
Messages
2,570,186
Members
46,739
Latest member
Clint8040

Latest Threads

Top