Recent Criticism about Ruby (Scalability, etc.)

[Gary Wright]

It definitively is. One aspect of Ruby that hinders scaling is the
absence of native threads IMHO. On the other hand, mechanisms are
provided for IPC (DRb for example) which are easy to use and thus
may be counted as compensating at least partially for the lack of
native threading.

I admit to being puzzled by the general fascination
with threads (native or not) for solving scaling
problems.

Good old processes have always seemed like a reasonable
way to partition problems and take advantage of
concurrency opportunities due to waiting on I/O
(single processor) or the parallel nature of a
CPU-bound calculation (multi-processor).

Processes also prevent the kind of problems associated
with concurrent access to shared memory that are
inherent in a multi-thread/single-process model.
A multi-process solution can more easily be
retargeted to a multi-machine solution across a network
than can a multi-thread solution.

I suspect that language and OS features have a great
affect on the concurrency model a programmer might
select or prefer.

I'm not suggesting that processes are in all cases
preferred to threads just that I would tend to explore
a multi-process solution *first* before a multi-thread
solution.

 

[Daniel DeLorme]

I'm not suggesting that processes are in all cases
preferred to threads just that I would tend to explore
a multi-process solution *first* before a multi-thread
solution.

+1
threads are a misfeature

 

[Chad Perrin]

"Most"? If you define "large data center" as the very top echelon,
then maybe, but even then I'd like to see some data. I expect the vast
majority (all?) of readers of this ng will be involved in scenarios in
which the cost of development time far exceeds electricity or server
costs for their deployed applications.

Part of what kept me from getting involved in Ruby sooner than I did
was my erroneous view that I wanted to be using technology that would
be sufficient to run Amazon, Ebay, etc. Little did it matter that I
wasn't pursuing that type of project - analogous to the fact that
most, if not all, Hummer drivers will never encounter serious off road
or combat situations

That's okay, though, because most civilian Hummers are unsuited to
offroading and combat situations.

I'm all for increasing the performance and scalability of Ruby, but I
think the productivity gains still outweigh the extra runtime costs
for most projects.

Agreed -- for the vast majority of projects in the range for which Ruby
is typically used, as compared with languages with execution performance
that is actually sufficiently better than Ruby's to bother measuring the
difference. I mean, sure, it's fun to compare benchmarks between Python
and Ruby (for instance), but if you actually need a performance boost you
should be comparing Ruby with C instead (or comparing Python with C, as
your tastes in productivity-enhancing languages may lead you).

 

[Chad Perrin]

Amdahl's law is relevant because most software _can't_ be written to
scale entirely linearly with the hardware, because most computational
problems are limited in the amount of parallelism they admit. You may
have been fortunate enough to have been presented with a lot of
embarrassingly parallel problems to solve, but that isn't the norm.

Maybe not "entirely", but certainly close enough for government (or
corporate) work. I was under the impression we were talking about
massive-traffic server-based systems here, where throwing more hardware
at the problem (in the sense of extra blades, or whatever) is an option.
I did not think we were talking about something like a desktop app where
opportunities for parallelism are strictly limited -- in which case I'd
agree that throwing more hardware at the problem is a non-starter. Of
course, I don't know anyone who thinks endlessly adding processors to
your desktop system is the correct answer to a slow word processor.

No argument there, as long as it's understood that there are limits to
what can be achieved. I don't want to discourage anyone from seeking
linear scalability as an ideal, but it's not a realistic thing to
promise or assume.

It's close enough (again), for many purposes, to "realistic". When you
can get roughly linear scaling up to 100 times as much scaling needs, as
opposed to trying to get similar scaling capabilities out of throwing
programmers (or programmer time) at the problem, that's certainly
"realistic" in my estimation.

Obviously I'm not saying that you should write crap code and throw
hardware at it. On the other hand, there's a sweet spot for effort spent
in developing good, performant code -- and beyond that point, you should
consider throwing hardware at the problem. In such circumstances, one of
the primary measures of quality code is "Does it scale in a roughly
linear manner when you add compute resources?"

 

[Chad Perrin]

I admit to being puzzled by the general fascination
with threads (native or not) for solving scaling
problems.

Good old processes have always seemed like a reasonable
way to partition problems and take advantage of
concurrency opportunities due to waiting on I/O
(single processor) or the parallel nature of a
CPU-bound calculation (multi-processor).

There's entirely too much focus on threads these days. In many cases,
separate processes are superior to multithreading concurrency. On the
other hand, there are cases where multithreading concurrency is superior
to multiprocessing concurrency.

Processes also prevent the kind of problems associated
with concurrent access to shared memory that are
inherent in a multi-thread/single-process model.
A multi-process solution can more easily be
retargeted to a multi-machine solution across a network
than can a multi-thread solution.

On the other hand, sometimes it's nice to keep process overhead down.
Most of the time, however, I think multithreaded concurrency is a case of
premature optimization.

I suspect that language and OS features have a great
affect on the concurrency model a programmer might
select or prefer.

I'm not suggesting that processes are in all cases
preferred to threads just that I would tend to explore
a multi-process solution *first* before a multi-thread
solution.

Same here. That's one of the reasons that, though I know a fair bit
about multithreaded concurrency in theory, I've done very little with
multithreaded development in practice. In fact, all I've found need to
do with mutilthreaded dev so far is dealing with others' code, where the
code was implemented using multithreaded concurrency before I ever laid
eyes on it.

 

[Phlip]

+1
threads are a misfeature

Threads are worse than GOTO.

Yes, sometimes you need GOTO. (Locally catch-throw). Do your darndest to
avoid it!

 

[MenTaLguY]

Maybe not "entirely", but certainly close enough for government (or
corporate) work. I was under the impression we were talking about
massive-traffic server-based systems here, where throwing more hardware
at the problem (in the sense of extra blades, or whatever) is an option.
I did not think we were talking about something like a desktop app where
opportunities for parallelism are strictly limited -- in which case I'd
agree that throwing more hardware at the problem is a non-starter. Of
course, I don't know anyone who thinks endlessly adding processors to
your desktop system is the correct answer to a slow word processor.

I wasn't thinking of only desktop systems, but I think you're right:
many massive-traffic server-based systems _can_ be embarrassingly
parallel, since jobs are often relatively independent (e.g. individual
user sessions/HTTP requests), and in that context adding more work
usually translates into simply adding more jobs. It's going to depend
a lot on the nature of the problem domain, though: once the jobs
become sufficiently interdependent, you do start to hit a scalability
wall (often a problem in e.g. simulations or online games).

Anyway, I think I was wrong: Amdahl's law may not be appropriate
here -- we aren't just talking about making a fixed-size job go
faster, but what happens when more work is added. Unless you're doing
something like n-body simulations, the relative amount of
unparallelizable work can decrease as the absolute amount of work
increases, because the shared stuff that forces serialization can often
be partitioned, allowing more overall parallelism than was practical
with a smaller workload.

It's close enough (again), for many purposes, to "realistic". When you
can get roughly linear scaling up to 100 times as much scaling needs, as
opposed to trying to get similar scaling capabilities out of throwing
programmers (or programmer time) at the problem, that's certainly
"realistic" in my estimation.

I'd submit that while you're still able to get a two-orders-magnitude
increase in performance from simply improving your algorithms, you're
probably not to the point where you could scale linearly. On the other
hand, by the time you're scaling linearly, there's probably not much
else to squeeze out of the thing, aside from micro-optimizations which
may get you some fractions of an order of magnitude improvement.

Obviously I'm not saying that you should write crap code and throw
hardware at it. On the other hand, there's a sweet spot for effort spent
in developing good, performant code -- and beyond that point, you should
consider throwing hardware at the problem. In such circumstances, one of
the primary measures of quality code is "Does it scale in a roughly
linear manner when you add compute resources?"

Yes, agreed.

-mental

 

[Chad Perrin]

I'd submit that while you're still able to get a two-orders-magnitude
increase in performance from simply improving your algorithms, you're
probably not to the point where you could scale linearly. On the other
hand, by the time you're scaling linearly, there's probably not much
else to squeeze out of the thing, aside from micro-optimizations which
may get you some fractions of an order of magnitude improvement.

Algorithms are important to performance, of course, and there's a minimal
amount of attention you should always want to employ in writing quality
code. My point is that ultimately, as things continue to scale upward,
you tend to pass a point where algorithm tweaking helps sufficiently any
longer to bother with it very much at about the same time you notice that
no matter what else you do you're going to have to add more hardware
resources.

Yes, agreed.

It seems that on one hand I've been advocating writing code so you can
scale upward in hardware resources, and that throwing hardware at the
problem is eventually the only likely option you have left, while on the
other hand you're advocating for people writing good code in the first
place because crap code of sufficiently bad quality won't scale very well
no matter how much hardware you throw at it. In other words, I was
assuming reasonably good code as a baseline, and you were assuming
reasonably bad code as a baseline. These incompatible assumptions may be
influenced by our respective work environments.

I'll expand on my position, then:

1. Hire good programmers.

2. Have them write good code.

3. Throw hardware at the scaling problem, because your good code
written by good programmers can handle it.

 

[MenTaLguY]

Threads are worse than GOTO.

I think the biggest issue is not threads per se, but rather threads
with shared state. A non-shared-state thread could be preferable
to an OS process.

-mental

 

[M. Edward (Ed) Borasky]

Pop-quiz: Which of *your* core features would you have to eliminate with
three million simultaneous users?

R&D vacations

 

[Todd Benson]

Ruby will eventually (I think) become something more useful than PHP,
mostly because it is a language, like other cool ones (LUA, Erlang,
etc.) that attracts some very smart people.

Everyone seems to be focusing on the cost/benefit analysis. Have we
forgot about how fun it is to program?

Todd

 

[Todd Benson]

Ruby will eventually (I think) become something more useful than PHP,
mostly because it is a language, like other cool ones (LUA, Erlang,
etc.) that attracts some very smart people.

Everyone seems to be focusing on the cost/benefit analysis. Have we
forgot about how fun it is to program?

Todd

Hmm... I guess I happen to be in the wrong thread, but the point
still stands (-: (Yes, I stole that smiley from someone else).

Todd

 

[Robert Klemme]

Depends what you mean by "scaling". When I think of scaling, I think of
tens or hundreds of thousands of servers running
AOL/eBay/Amazon/Google-style apps. At that level, threads don't matter
much; what matters is that your app scales near-linearly with hardware
(what we used to call "dollar scalable").

What do you mean by scaling?

To me "scaling" just means "adjusting to higher load". What you have
in mind seems a concrete solution to provide scaling but that's just
one (albeit a very common one).

Kind regards

robert

 

[Chad Perrin]

A lot depends on your application requirements. If you design it from the
ground up to be "shared nothing", then you may well be lucky enough to
truly HAVE shared nothing. But you'll also have a pretty limited feature
set.

"Feature-rich" is overrated. Anyone who tries to be everything to
everyone will end up being not the right thing to pretty much everyone.
You only get into the kind of trouble you describe when you try to hard
to get *everyone* interested.

What's the big buzzword today? Social networking. What did we used to
call that? "Community." What was the single biggest sticky-paper
community feature? Buddy lists. Who does buddy lists besides the Big Guys
(who can throw money at it) and the really small guys (who fit on a single
server)? Nobody. Why? Doesn't scale linearly. Think about what it takes
to offer a feature that, for every simultaneous user, checks the list of
every other simultaneous user for people you know. Shared-nothing *that*.

The answer to that, from where I'm sitting, is to choose between focusing
on "social networking" and focusing on something else. If you're just
adding it as "yet another feature" to your application to become
buzzword-compliant, you'll become another dot-com startup has-been. Of
course, there's also always the business strategy of "look successful,
sell to someone big" without actually turning a positive buck along the
way -- and if that's what you want to do, you're on your own.

My area of expertise was the AOL mail system. And, looking back, there
were a number of core features we offered that simply couldn't be done in a
shared-nothing world over slow phone lines:

And some of the features were only important in an age where pipes (both
last-mile and LAN) were very narrow and disks and RAM were very small.
Spam, in particular, made the "one copy of each message" model obsolete,
because spammers wouldn't play by the rules.

But restricting yourself to only shared-nothing features means ruling out
an awful lot of features. Including anything depending on a database
index, or a table that fits completely in memory, or any sort of
rate-limiting or duplicate-detection or spam prevention, or in fact
anything that makes any assumptions at all about the state of any database
you're interacting with or relational integrity or any other transaction in
the system, ever. Including whether the disk drive holding the transaction
you just wrote to disk has disappeared in a puff of head crash.

You don't always have to write shared-nothing code to get near-linear
scalability -- and it's true that near-linear scalability is something
that only exists within certain ranges before you hit a cost or resource
requirement spike, but if you're smart you plan ahead for those kinds of
things. Things don't always go as planned, of course, but if you're
smart you plan for *that*, too, by setting aside "money for a rainy day"
and ensuring that, short of your main datacenters and every off-site
backup in the world being eliminated by meteor strikes simultaneously,
any major scaling issues will not require a sudden "right now" fix.

It was always the little things that bit us. Know why AOL screen names are
often "Jim293852"? Well, it started out as "The name 'Jim' is already
taken. Would you like 'Jim2'?". Guess how well that scales when the first
available Jim is "Jim35000"? Not very.

I'm curious how any of this is meant to support the position that a
faster-executing programming language that imposes greater hurdles on
programmer productivity will be a better investment for scalability than
designing a system that can absorb greater loads by adding hardware
resources.

Pop-quiz: Which of *your* core features would you have to eliminate with
three million simultaneous users?

Hopefully, by the time you have that many users, you're making enough
money to be able to manage that many users. If not, your business model
sucks.

 

[MenTaLguY]

In other words, I was assuming reasonably good code as a baseline,
and you were assuming reasonably bad code as a baseline. These
incompatible assumptions may be influenced by our respective work
environments.

Yes, I think that's the crux of it.

I'll expand on my position, then:

1. Hire good programmers.

2. Have them write good code.

3. Throw hardware at the scaling problem, because your good code
written by good programmers can handle it.

Perhaps we've also got slightly different audiences in mind --
"hire good programmers" is not advice I'd normally give to a
programmer.

-mental

 

[Chad Perrin]

Yes, I think that's the crux of it.


Perhaps we've also got slightly different audiences in mind --
"hire good programmers" is not advice I'd normally give to a
programmer.

Adjusted for programmers:

1. Be(come) a good programmer.

2. Write good code.

3. Let your boss throw hardware at the scaling problem, because the
good code you wrote as a good programmer can handle it.

Better?

 

[MenTaLguY]

Adjusted for programmers:

1. Be(come) a good programmer.

2. Write good code.

3. Let your boss throw hardware at the scaling problem, because the
good code you wrote as a good programmer can handle it.

Better?

Fair enough.

-mental

 

[Chad Perrin]

*chuckle* I do believe that's the first time in recorded history that
anyone has accused AOL of being feature-rich.

AOL has always been "feature-rich". It just isn't the right thing for
almost anyone at all, because when you get that "feature-rich" you get
very feature-targeted -- in that you're catering only to the people who
want all, or most, of what you provide. People who want little or none
of what you provide (beyond basics) will go somewhere else, because that
"somewhere else" doesn't impose a whole lot of overhead. The fact that
AOL features have often been broken, slow, and in-the-way kludgey never
changed the fact that there were a lot of them -- and, in fact, it's in
large part the sheer weight of features that made the feature set so
unusable to so many people.

Trying to trap people in an AOL-only internet, rather than letting them
seamlessly out into the Internet, was a "feature" -- it was just a
feature pretty much nobody wanted. Most of AOL's features have tended to
be much like that.

The point I was making with all the features you snipped was that it
*doesn't* take wild, pie-in-the-sky everything-to-everyone features to
prevent your application from scaling linearly. Any little thing can trip
you up. Most of the features I listed were either small facets of behavior
or byproducts of other design decisions. And some of them (e.g. saving
disk space) were actually "scaling" features themselves; what helps you
scale to 100x (fitting on the available disk drives) may hinder you at
10000x (when your servers are in different data centers).

. . but you can get pretty close to linear scalability within specific
ranges of scaling, especially if you *avoid* massive feature lists.
Sure, they don't have to be "wild, pie-in-the-sky" features, but you
missed my point with that statement. My point wasn't that one feature is
"everything to everyone", but that seventy features is trying to provide
exactly that without doing any one thing that, examined in a vacuum,
looks unreasonable.

In other words, if you want to minimize scalability hurdles, one of the
most important things you can do is pick a focus area.

I'm not really sure what that has to do with... well, with anything. But
then, my point probably wasn't all that clear to you, either. The point
was that there's always a tension between feature-set and scalability.

That was sorta my point, too -- except that I wasy saying that since
there's a tension, you need to pick a direction, and if your direction
kills scalability that's your own fault and not disproof of the fact that
near-linear scalability is possible. The fact of the matter is that the
same things that break linearality of scalabilty for your software are
the things that break linearality of scalability for everything else,
too. You may start watching your "everything to everyone" business plan
circling the drain, now.

On the other hand, I find it remarkable that a huge crop of "social
networking" sites have become immensely popular without the most obvious
social networking feature - who else is here? - because that feature just
doesn't scale. It would be like if e-commerce grew to its current levels
without real-time credit card processing, or if Flickr only let you upload
ASCII-art of photos because photos are too big to store.

I don't find that so odd. The "most obvious" social networking feature
is actually not all that great a feature for a new business venture. It
was solved a long damned time ago with technologies like IRC. It's not
new. The other stuff being implemented by all these "social networking"
sites *is* new, at least in an Internet context -- or presented in a new
manner.

I both agree and violently disagree. The problem with "if you're smart you
plan ahead" is that (a) you often won't know what your pain points will be
until shortly before you hit them, (b) even if you do, they may not be in
your control, and (c) you don't always know how fast you're going to grow.
It would be foolish for me to invest in a large Arizona data center in case
the traffic to my last-updated-in-2005 blog spikes 10000x next year. (And
I do keep promising my financial advisor that I'm selling the data center.)

But sometimes externalities do hit your business; it becomes "steam engine
time", and you're the steam engine. Verizon nee AT&T nee the Bell
Telephone Company had, literally, over a hundred years of experience
telling them how much their business would grow each year. And that worked
very well until 1995, when all of a sudden the online world was booming,
people were leaving their phones off the hook even when they weren't home,
and suddenly they ran out of dial tones.

Luckily for them, they were the phone company; nobody had anywhere to run.
But if they were in a competitive business, they'd be toast, because
someone would fill the need that they couldn't. Remember Friendster?
Great idea, great product, right time, couldn't scale as quickly as their
user base, slow site, toast.

I don't generally like to be so harsh, but . . . if you plan badly, your
plan fails. Sorry to burst the bubble for anyone who thinks that hard
work and good intentions should automatically translate into success.
This is not something that can be blamed on the potential scalability of
well-written software. The blame for that rests entirely at the feet of
those who made the planning decisions in the first place.

It isn't. Elsewhere in this thread, in fact, I was arguing that programmer
productivity is far more important to orders-of-magnitude scalability than
raw language performance. I agree with you on that. Where I disagreed was
that it was realistic for "many purposes" to assume linear scalability.
Show me any site design and I'll show you a dozen places it falls over at a
few orders of magnitude.

Show me where it falls over at a few orders of magnitude, and I'll show
you software that is being misused -- or, looked at from the other
direction, miswritten -- if it's being written well at all. If it's not
being written well at all, that's pretty much irrelevant to my point
anyway, since poor software development can kill anything.

This is why focus is important: when you're trying to be all things to
all people (the all-singing, all-dancing, dish-washing performing
monkey), there's no give in any area to make compromises so that in other
areas it'll scale, because there's nothing you don't need out of a
system.

 

[M. Edward (Ed) Borasky]

Ruby will eventually (I think) become something more useful than PHP,
mostly because it is a language, like other cool ones (LUA, Erlang,
etc.) that attracts some very smart people.

There are some languages that can *only* be used by "very smart people".
APL comes to mind, and I suspect there are those who could make the same
case for Forth, Haskell and Prolog. For "most of us", languages like
Python, Perl, PHP, Ruby and Lua are great *because* they're easy to
learn even if you're not a "very smart person".

Everyone seems to be focusing on the cost/benefit analysis. Have we
forgot about how fun it is to program?

I haven't ... but I don't think fun is language-specific. I can't think
of a single programming language I hated using, but then, I never used
RPG. That one I think would have sucked.

 

[Matt Lawrence]

There are some languages that can *only* be used by "very smart people". APL
comes to mind, and I suspect there are those who could make the same case
for Forth, Haskell and Prolog. For "most of us", languages like Python,
Perl, PHP, Ruby and Lua are great *because* they're easy to learn even if
you're not a "very smart person".

Huh? Forth was one of the easiest languages for me to learn. Ruby has
been a LOT more work.

-- Matt
It's not what I know that counts.
It's what I can remember in time to use.