how to get CPU usage

[Luca Scaljery]

Hi All

For the (RAILS) application I'm writing I need to know what the CPU
usage is, any modules in Ruby that can do this (for a linux server)?

In my case, the server has to deal with many AJAX calls all the time. So
when the CPU-usage gets too high, I would like to lower the
AJAX-call-frequency dynamically.

Thnx a lot in advance
LuCa

 

[Daniel Berger]

Hi All

For the (RAILS) application I'm writing I need to know what the CPU
usage is, any modules in Ruby that can do this (for a linux server)?

In my case, the server has to deal with many AJAX calls all the time. So
when the CPU-usage gets too high, I would like to lower the
AJAX-call-frequency dynamically.

Thnx a lot in advance

gem install sys-cpu

Regards,

Dan

 

[thefed]

gem install sys-cpu

Is there a way to control how much CPU Ruby uses **FROM WITHIN** ruby?

I know there's nice, renice, and ulimit, but I'd _really_ like to
keep this in ruby.

 

[Daniel Berger]

Is there a way to control how much CPU Ruby uses **FROM WITHIN** ruby?

I know there's nice, renice, and ulimit, but I'd _really_ like to
keep this in ruby.

Process.setrlimit

Regards,

Dan

 

[M. Edward (Ed) Borasky]

Hi All

For the (RAILS) application I'm writing I need to know what the CPU
usage is, any modules in Ruby that can do this (for a linux server)?

In my case, the server has to deal with many AJAX calls all the time. So
when the CPU-usage gets too high, I would like to lower the
AJAX-call-frequency dynamically.

Thnx a lot in advance
LuCa

Are you looking for the overall system utilization? There are lots of
command-line tools you can use to log basic performance variables and
capture them in a pipe with Ruby, but in the long run, if you're really
interested in server performance monitoring, you probably should pick up
a whole performance logging package. RRDtool is probably the best, and
it has Ruby bindings. So you can let it collect the data and interrogate
its database periodically for the performance variables you need.

If you're willing to sacrifice portability away from Linux, I can give
you some quick-and-dirty code that will go directly into /proc and
snapshot this once every five minutes. But I think that's a bad idea --
I think you should keep your options open and get something that will
run on Linux, FreeBSD or Open Solaris.

 

[thefed]

If you're willing to sacrifice portability away from Linux, I can give
you some quick-and-dirty code that will go directly into /proc and
snapshot this once every five minutes.

I'd love to see that code!

But what I'm actually trying to do is to actually _limit_ the CPU
usage from ruby - not shut it down if it uses to much.

I want to teach the duck the trick, not punish for doing it wrong!

IE, make it so all my programs never use more than 15% of my CPU.

Thanks,
Ari

 

[M. Edward (Ed) Borasky]

I'd love to see that code!

But what I'm actually trying to do is to actually _limit_ the CPU usage
from ruby - not shut it down if it uses to much.

I want to teach the duck the trick, not punish for doing it wrong!

IE, make it so all my programs never use more than 15% of my CPU.

Thanks,
Ari

Well ... from inside a Ruby script, open the following file:

/proc/self/stat

You will get a line something like this:

$ more /proc/self/stat
9931 (more) R 9923 9931 9923 34816 9931 4194304 281 0 1 0 0 0 0 0 20 0 1
0 10067 10 8445952 197 18446744073709551615 4194304 4223844
140735769651408 184467440737 09551615 47274997513552 0 0 0 134742022 0 0
0 17 0 0 0 2

Yeah, a bunch of numbers separated by spaces. I *think* this hasn't
changed in many kernel versions, but here's the C code that creates that
line from the source of the 2.6.23 kernel. If you want to look at yours,
the file name is /usr/src/linux/fs/proc/array.c

res = sprintf(buffer, "%d (%s) %c %d %d %d %d %d %u %lu \
%lu %lu %lu %lu %lu %ld %ld %ld %ld %d 0 %llu %lu %ld %lu %lu %lu %lu %lu \
%lu %lu %lu %lu %lu %lu %lu %lu %d %d %u %u %llu\n",
task->pid,
tcomm,
state,
ppid,
pgid,
sid,
tty_nr,
tty_pgrp,
task->flags,
min_flt,
cmin_flt,
maj_flt,
cmaj_flt,
cputime_to_clock_t(utime),
cputime_to_clock_t(stime),
cputime_to_clock_t(cutime),
cputime_to_clock_t(cstime),
priority,
nice,
num_threads,
start_time,
vsize,
mm ? get_mm_rss(mm) : 0,
rsslim,
mm ? mm->start_code : 0,
mm ? mm->end_code : 0,
mm ? mm->start_stack : 0,
esp,
eip,
/* The signal information here is obsolete.
* It must be decimal for Linux 2.0 compatibility.
* Use /proc/#/status for real-time signals.
*/
task->pending.signal.sig[0] & 0x7fffffffUL,
task->blocked.sig[0] & 0x7fffffffUL,
sigign .sig[0] & 0x7fffffffUL,
sigcatch .sig[0] & 0x7fffffffUL,
wchan,
0UL,
0UL,
task->exit_signal,
task_cpu(task),
task->rt_priority,
task->policy,
(unsigned long long)delayacct_blkio_ticks(task));
if (mm)
mmput(mm);


OK ... now what? Well, you read that line with Ruby, and the kernel very
nicely printed a bunch of space-separated fields. There's one field for
each line in the "sprintf". The ones you want are

cputime_to_clock_t(utime),
cputime_to_clock_t(stime),

These are, in order, the number of CPU ticks your Ruby process has
consumed since process creation in user mode and the number of CPU ticks
consumed since process creation in system mode. These start at zero and
increase every time the kernel looks at the process and sees what it has
done.

There's one more piece of the puzzle. Those counts are in CPU ticks,
known as "jiffies" in Linux. You want to convert them to seconds. The
conversion factor is called "HZ" and it's in a kernel header file. On my
Athlon64, it's in

/usr/include/asm-i386/param.h:#define HZ 100

So ... your script has to

1. Read /proc/self/stat
2. Split the line up into fields
3. Pick out the right two fields
4. Add them together
5. Find the definition of HZ in the kernel header file
6. Divide the total in step 4 by HZ

You now have the total number of CPU seconds your process has used.

Or ... if you want to build a C extension, you could call "getrusage",
which is a lot more portable, since it will run on almost all platforms.

Or, you could open your copy of the Pickaxe (2nd edition -- Ruby 1.8)
and go to page 566. There you will find a method called "process.times".



This reminds me of the joke about the student who was asked how to
measure the height of a building using a barometer.

 

[thefed]

Well ... from inside a Ruby script, open the following file:

Thanks!

But there's not anything I can do to limit the CPU% my script uses?

 

[Thomas Adam]

Thanks!

But there's not anything I can do to limit the CPU% my script uses?

No -- that has to be controlled at the process level, something your
program can't do.

-- Thomas Adam

 

[Gary Wright]

No -- that has to be controlled at the process level, something your
program can't do.

Perhaps something like Process::setrlimit is something the OP could use?

Gary Wright

 

[thefed]

Perhaps something like Process::setrlimit is something the OP could
use?

I looked at setrlimit, but it seems to quit the process if it
violates the conditions. Will it actually prevent ruby from going
over 15% without killing it?

 

[Gary Wright]

I looked at setrlimit, but it seems to quit the process if it
violates the conditions. Will it actually prevent ruby from going
over 15% without killing it?

I don't know if setrlimit is appropriate for your use or the OP's use
but it works like...

For RLIMIT_CPU, the limit is the maximum number of CPU seconds.
When the process gets to the (lower) soft limit the OS will send
the SIGXCPU signal, which the process can catch and then do something
(like shutdown). If the (higher) hard limit is reached, the OS
will terminate the process. I just tried the following code
on Mac OS X and it worked just fine. Your milage may vary depending
on how well your OS supports hard/soft resource limits.

soft, hard = *Process.getrlimit(Process::RLIMIT_CPU)
Process.setrlimit(Process::RLIMIT_CPU, 10, hard)

trap ("SIGXCPU") { puts 'received XCPU, shutting down'; exit }

loop {}

 

[Gary Wright]

trap ("SIGXCPU") { puts 'received XCPU, shutting down'; exit }

On Mac OS X, SIGXCPU gets sent every second or so once you've
exceeded the hard limit. You can still ignore or catch the
signal at that point---the os won't just kill the process.

I'm not sure if this behavior follows the applicable standards
or is just OS X's take on the situation.

Gary Wright

 

[M. Edward (Ed) Borasky]

I looked at setrlimit, but it seems to quit the process if it violates
the conditions. Will it actually prevent ruby from going over 15%
without killing it?

My last post was semi-facetious ... as it turns out, everything you need
to do what you want to do is built in to Ruby. But as a working
performance engineer, I'm not convinced that what you are trying to do
is either necessary or without risk to the correctness of your
application's behavior.

In other words:

1. Is there in fact a performance problem with your application? There
are many ways to define "performance problem", but what I'm talking
about here is whether the *end-users* experience unacceptable response
times for the tasks they are trying to get done. If they are happy with
response times, you're done -- there is no problem.

2. So ... assume they are unhappy. If you look at a task's lifetime,
you'll generally see that it starts with a user pressing a button or
clicking a link after having typed something in somewhere. That sends an
HTTP request to the web server, which vibrates around in the guts of
Rails, possibly does some CRUD with the database, generates an update to
the screen, and sends a bunch of stuff back to the user.

What you need to do is break this life cycle down into its components
... network time to the servers, processor time, disk time and network
time from the servers back to the end user. The good news is that you
don't really need to worry about memory at this point -- Ruby and the OS
will worry about this for you.

3. You will usually find that one component -- network traffic over slow
links, processor time in the Ruby interpreter or the database, or disk
time in the database -- is the largest one. We call this the *primary
bottleneck*.

Now let's say, since you originally asked how to reduce your Ruby usage
to 15 percent of the CPU, that you've determined that the primary
bottleneck is the Ruby processor time. If you limit that to 15 percent
of the CPU, your application will become *less responsive* than it was
when it was allowed to use as much as it needed!

Instead, what you need to do is profile the Ruby script, using the
profiling tools built in to Ruby/Rails. This is covered in a number of
books, so I won't go into it here. The point is that you need to define
the problem, if indeed there is one, and use a *systematic* approach to
solving it.

 

[thefed]

Instead, what you need to do is profile the Ruby script, using the
profiling tools built in to Ruby/Rails. This is covered in a number of
books, so I won't go into it here. The point is that you need to
define
the problem, if indeed there is one, and use a *systematic*
approach to
solving it.

So ruby methods won't *try* to use as much CPU as necessary, just as
much as needed?

IE, a puts statement wont take up 100% of my CPU for a split second,
it will only use 1%?

 

[M. Edward (Ed) Borasky]

So ruby methods won't *try* to use as much CPU as necessary, just as
much as needed?

IE, a puts statement wont take up 100% of my CPU for a split second, it
will only use 1%?

Actually, all of that is handled by the operating systems, and most of
them do a better job of scheduling than an application designer can.
Operating systems manage the processor(s), I/O devices, memory and
processes for you. There are cases where you can get raw speed at the
expense of portability, but in general you're far better off letting the
operating system do the managing.

On a *totally* unrelated note, I want to share something "cute" I got
today when I installed Wireshark.