write_noblock is blocking?

C

Christoffer Lernö

Anyone know what's up with TCPSocket#write_noblock actually blocking
when sending a packet that exceeds the buffer size of the socket in
1.8.6 on Mac.

The behaviour seems slightly counter-productive.


/C
 
E

Eric Hodel

Anyone know what's up with TCPSocket#write_noblock actually =20
blocking when sending a packet that exceeds the buffer size of the =20
socket in 1.8.6 on Mac.

The behaviour seems slightly counter-productive.

How long does it block? I see ruby occasionally being delayed 10ms =20
when using write_nonblock, but not repeatably for the same packet =20
sizes. Probably my benchmark script is wrong.

Could you supply a benchmark script that illustrates your problem?

#write_nonblock first does rb_io_check_closed() which might call down =20=

to fseek(3) then write(2) before calling rb_io_set_nonblock(). I'm =20
not even close to being a sockets expert though, I just read the man =20
pages.
 
7

7stud --

Eric said:
That's rather harsh for a one letter typo.

Harsh? In what way?

I wasn't aware there was a similarly named method. I looked in the
index of pickaxe2, and there was no write_noblock method listed, and now
that I look again there is no write_nonblock method listed either. I
also checked the standard library documentation for BasicSocket, Socket,
and TCPSocket, and there was no write_noblock method listed, and now
that I look again, there is no write_nonblock method either. In fact,
there is no method that starts with 'write'. I also searched google and
came up with 0 hits for write_noblock, which has to be a first for me.

Of course, because the ruby documentation is so deplorable, maybe I
should have just assumed it was a commonly used method.
 
A

ara.t.howard

Harsh? In what way?

responding that way when you didn't know
I wasn't aware there was a similarly named method. I looked in the
index of pickaxe2, and there was no write_noblock method listed, =20
and now
that I look again there is no write_nonblock method listed either. I
also checked the standard library documentation for BasicSocket, =20
Socket,
and TCPSocket, and there was no write_noblock method listed, and now
that I look again, there is no write_nonblock method either. In fact,
there is no method that starts with 'write'. I also searched =20
google and
came up with 0 hits for write_noblock, which has to be a first for me.

searching !=3D knowing

besides, google pulls 'ruby write_noblock' up on the first hit

http://www.google.com/search?q=3Druby=20
+noblock&ie=3Dutf-8&oe=3Dutf-8&aq=3Dt&rls=3Dorg.mozilla:en-=20
US:eek:fficial&client=3Dfirefox-a
Of course, because the ruby documentation is so deplorable, maybe I
should have just assumed it was a commonly used method.

now you are being harsh again.

cfp:~ > PAGER=3Dcat ri IO|grep non
pos, pos=3D, print, printf, putc, puts, read, read_nonblock,
write_nonblock


cfp:~ > PAGER=3Dcat ri IO
-------------------------------------------------------------- Class: IO
Class +IO+ is the basis for all input and output in Ruby. An I/O
stream may be _duplexed_ (that is, bidirectional), and so may use
more than one native operating system stream.

Many of the examples in this section use class +File+, the only
standard subclass of +IO+. The two classes are closely associated.

As used in this section, _portname_ may take any of the following
forms.

* A plain string represents a filename suitable for the
underlying operating system.

* A string starting with ``+|+'' indicates a subprocess. The
remainder of the string following the ``+|+'' is invoked as a
process with appropriate input/output channels connected to =20=

it.

* A string equal to ``+|-+'' will create another Ruby =20
instance as
a subprocess.

Ruby will convert pathnames between different operating system
conventions if possible. For instance, on a Windows system the
filename ``+/gumby/ruby/test.rb+'' will be opened as
``+\gumby\ruby\test.rb+''. When specifying a Windows-style =20
filename
in a Ruby string, remember to escape the backslashes:

"c:\gumby\ruby\test.rb"

Our examples here will use the Unix-style forward slashes;
+File::SEPARATOR+ can be used to get the platform-specific
separator character.

I/O ports may be opened in any one of several different modes,
which are shown in this section as _mode_. The mode may either =20
be a
Fixnum or a String. If numeric, it should be one of the operating
system specific constants (O_RDONLY, O_WRONLY, O_RDWR, O_APPEND =20=

and
so on). See man open(2) for more information.

If the mode is given as a String, it must be one of the values
listed in the following table.

Mode | Meaning
-----+--------------------------------------------------------
"r" | Read-only, starts at beginning of file (default mode).
-----+--------------------------------------------------------
"r+" | Read-write, starts at beginning of file.
-----+--------------------------------------------------------
"w" | Write-only, truncates existing file
| to zero length or creates a new file for writing.
-----+--------------------------------------------------------
"w+" | Read-write, truncates existing file to zero length
| or creates a new file for reading and writing.
-----+--------------------------------------------------------
"a" | Write-only, starts at end of file if file exists,
| otherwise creates a new file for writing.
-----+--------------------------------------------------------
"a+" | Read-write, starts at end of file if file exists,
| otherwise creates a new file for reading and
| writing.
-----+--------------------------------------------------------
"b" | (DOS/Windows only) Binary file mode (may appear with
| any of the key letters listed above).

The global constant ARGF (also accessible as $<) provides an
IO-like stream which allows access to all files mentioned on the
command line (or STDIN if no files are mentioned). ARGF provides
the methods +#path+ and +#filename+ to access the name of the file
currently being read.

------------------------------------------------------------------------


Includes:
---------
Enumerable(all?, any?, collect, detect, each_cons, each_slice,
each_with_index, entries, enum_cons, enum_slice, enum_with_index,
find, find_all, grep, group_by, include?, include_any?, index_by,
inject, injecting, map, max, member?, min, partition, reject,
select, sort, sort_by, sum, to_a, to_json, to_set, zip),
File::Constants()


Constants:
----------
SEEK_CUR: INT2FIX(SEEK_CUR)
SEEK_END: INT2FIX(SEEK_END)
SEEK_SET: INT2FIX(SEEK_SET)


Class methods:
--------------
for_fd, foreach, new, open, pipe, popen, read, readlines, select,
sysopen


Instance methods:
-----------------
<<, binmode, block_scanf, close, close_read, close_write, closed?,
each, each_byte, each_line, eof, eof?, fcntl, fileno, flush, =20
fsync,
getc, gets, inspect, ioctl, isatty, lineno, lineno=3D, open, pid,
pos, pos=3D, print, printf, putc, puts, read, read_nonblock,
readbytes, readchar, readline, readlines, readpartial, reopen,
rewind, scanf, seek, soak_up_spaces, stat, sync, sync=3D, sysread,
sysseek, syswrite, tell, to_i, to_io, tty?, ungetc, write,
write_nonblock



i know ruby-core is *always* looking for people to contribute for =20
docs. you certainly get what you pay for with bloody open source eh?


a @ http://codeforpeople.com/
 
E

Eric Hodel

responding that way when you didn't know
Exactly.


now you are being harsh again.

cfp:~ > PAGER=3Dcat ri IO|grep non
cfp:~ > PAGER=3Dcat ri IO

Also,

$ ri -l | egrep "write.*block"
IO#write_nonblock
i know ruby-core is *always* looking for people to contribute for =20
docs. you certainly get what you pay for with bloody open source eh?

Absolutely. The main reason that ruby documentation is "so =20
deplorable" is because the people complaining about it are rarely the =20=

ones doing anything to improve it.
 
7

7stud --

ara.t.howard said:
On Oct 13, 2007, at 6:31 PM, 7stud -- wrote:

searching != knowing

Maybe in your native language they don't have question marks for
punctuation. If you don't know what they mean, perhaps you shouldn't
criticize?
besides, google pulls 'ruby write_noblock' up on the first hit

http://www.google.com/search?q=ruby
+noblock&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-
US:eek:fficial&client=firefox-a

Now that google has had time to index this page, the hits you claim to
be the Holy Grail for "write no_block" are actually links back to this
thread. I wonder if you post any information on this forum that isn't
erroneous.
now you are being harsh again.

It sounds like you are suffering a case of sour grapes because I
corrected you in that other thread. If you are going to post poorly
written examples, which are also not responsive to a poster's question,
you will get corrected. If that hurts your feelings, maybe you should
construct better examples--or just not post at all.

Absolutely. The main reason that ruby documentation is "so
deplorable" is because the people complaining about it are rarely the
ones doing anything to improve it.

Yes, when a language is poorly designed and/or documented, blame the
people who are trying to learn it.
 
C

Christoffer Lernö

How long does it block? I see ruby occasionally being delayed 10ms =20=
when using write_nonblock, but not repeatably for the same packet =20
sizes. Probably my benchmark script is wrong.

Could you supply a benchmark script that illustrates your problem?

#write_nonblock first does rb_io_check_closed() which might call =20
down to fseek(3) then write(2) before calling rb_io_set_nonblock=20
(). I'm not even close to being a sockets expert though, I just =20
read the man pages.

Thanks. Know it "should work" made me try to analyze the situation. =20
Here is an example code:

require 'socket'
a =3D TCPServer.new(5000)
socket =3D TCPSocket.new("localhost", 5000)

# Using a.accept will give a socket that does not exhibit problems,
# Only TCPServer#accept_nonblock seems to create this
# odd situation.

other =3D a.accept_nonblock
puts "Start Write"

# Lower values, say "X" * 2000, writes all in one sweep and does
# not cause it block

other.write_nonblock("X" * 300000)
puts "End Write"



It looks like write_nonblock together with accept_nonblock causes =20
some issues.
Now I could be using these wrong as both are new (in 1.8.6?)


/Christoffer=
 
E

Eric Hodel

Thanks. Know it "should work" made me try to analyze the situation. =20=
Here is an example code:

[...]

It looks like write_nonblock together with accept_nonblock causes =20
some issues.

Now I could be using these wrong as both are new (in 1.8.6?)

I couldn't reproduce with your this test using:

$ ruby -v
ruby 1.8.6 (2007-09-23 patchlevel 5000) [powerpc-darwin8.10.0]

I'll to try again tomorrow though, since I'm tired.

I experimented a bit with your benchmark, and found that writing a =20
large string would only write as much as the socket buffer would =20
accept (81660 bytes, in my case).

Maybe you need to set the TCP_NODELAY socket option?

socket.setsockopt Socket::IPPROTO_TCP, Socket::TCP_NODELAY, 1

I once made write_nonblock raise Errno::EAGAIN, but I accidentally =20
deleted that change. I'll play some more tomorrow when I'm not so =20
tired.
 
C

Christoffer Lernö

On Oct 13, 2007, at 14:55 , Christoffer Lern=F6 wrote:
Anyone know what's up with TCPSocket#write_noblock actually =20
blocking when sending a packet that exceeds the buffer size of =20
the socket in 1.8.6 on Mac.

The behaviour seems slightly counter-productive.

How long does it block? I see ruby occasionally being delayed =20
10ms when using write_nonblock, but not repeatably for the same =20
packet sizes. Probably my benchmark script is wrong.

Could you supply a benchmark script that illustrates your problem?

#write_nonblock first does rb_io_check_closed() which might call =20
down to fseek(3) then write(2) before calling rb_io_set_nonblock=20
(). I'm not even close to being a sockets expert though, I just =20
read the man pages.

Thanks. Know it "should work" made me try to analyze the =20
situation. Here is an example code:

[...]

It looks like write_nonblock together with accept_nonblock causes =20
some issues.

Now I could be using these wrong as both are new (in 1.8.6?)

I couldn't reproduce with your this test using:

$ ruby -v
ruby 1.8.6 (2007-09-23 patchlevel 5000) [powerpc-darwin8.10.0]

I'll to try again tomorrow though, since I'm tired.

I experimented a bit with your benchmark, and found that writing a =20
large string would only write as much as the socket buffer would =20
accept (81660 bytes, in my case).

Maybe you need to set the TCP_NODELAY socket option?

socket.setsockopt Socket::IPPROTO_TCP, Socket::TCP_NODELAY, 1

I once made write_nonblock raise Errno::EAGAIN, but I accidentally =20
deleted that change. I'll play some more tomorrow when I'm not so =20
tired.

Could this be due to me running

ruby 1.8.6 (2007-03-13 patchlevel 0) [i686-darwin8.10.1] ?

The funny thing was that only the socket returned by accept_nonblock =20
had this behaviour, that is

require 'socket'
a =3D TCPServer.new(5000)
socket =3D TCPSocket.new("localhost", 5000)
other =3D a.accept
puts "Start Write"
other.write_nonblock("X" * 300000)
puts "End Write"

Works.

require 'socket'
a =3D TCPServer.new(5000)
socket =3D TCPSocket.new("localhost", 5000)
other =3D a.accept_nonblock
puts "Start Write"
other.write_nonblock("X" * 300000)
puts "End Write"

Doesn't work.

What's so special about a socket returned by accept_nonblock?


/Christoffer=
 
T

Tanaka Akira

Christoffer Lern=F6 said:
Could this be due to me running

ruby 1.8.6 (2007-03-13 patchlevel 0) [i686-darwin8.10.1] ?

The funny thing was that only the socket returned by accept_nonblock =20
had this behaviour, that is

I think it is a Mac OS X problem.

Your example works without problem on GNU/Linux and FreeBSD
box but not on Mac OS X.

% cat z1.rb=20
require 'socket'
require 'fcntl'
a =3D TCPServer.new(5000)
socket =3D TCPSocket.new("localhost", 5000)
other =3D a.accept
p other.fcntl(Fcntl::F_GETFL) & Fcntl::O_NONBLOCK
puts "Start Write"
p other.write_nonblock("X" * 300000)
puts "End Write"
% ./ruby -v z1.rb
ruby 1.8.6 (2007-03-13 patchlevel 0) [powerpc-darwin8.3.0]
0
Start Write
81560
End Write

The IO returned from accept is not nonblocking mode.
So write_nonblock makes it nonblocking mode at first.

% cat z2.rb=20
require 'socket'
require 'fcntl'
a =3D TCPServer.new(5000)
socket =3D TCPSocket.new("localhost", 5000)
other =3D a.accept_nonblock
p other.fcntl(Fcntl::F_GETFL) & Fcntl::O_NONBLOCK
puts "Start Write"
p other.write_nonblock("X" * 300000)
puts "End Write"
% ./ruby -v z2.rb=20
ruby 1.8.6 (2007-03-13 patchlevel 0) [powerpc-darwin8.3.0]
4
Start Write
^Cz2.rb:8: Interrupt

The IO returned from accept_nonblock seems nonblocking mode.
So write_nonblock doesn't make it nonblocking mode.
But actually it is not nonblocking mode, I think.

Workaround:

Index: ext/socket/socket.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- ext/socket/socket.c (revision 13670)
+++ ext/socket/socket.c (working copy)
@@ -1464,6 +1464,24 @@
return init_inetsock(sock, Qnil, arg1, Qnil, Qnil, INET_SERVER);
}
=20
+static void
+make_fd_nonblock(int fd)
+{
+ int flags;
+#ifdef F_GETFL
+ flags =3D fcntl(fd, F_GETFL);
+ if (flags =3D=3D -1) {
+ rb_sys_fail(0);
+ }
+#else
+ flags =3D 0;
+#endif
+ flags |=3D O_NONBLOCK;
+ if (fcntl(fd, F_SETFL, flags) =3D=3D -1) {
+ rb_sys_fail(0);
+ }
+}
+
static VALUE
s_accept_nonblock(VALUE klass, OpenFile *fptr, struct sockaddr *sockaddr, =
socklen_t *len)
{
@@ -1475,6 +1493,7 @@
if (fd2 < 0) {
rb_sys_fail("accept(2)");
}
+ make_fd_nonblock(fd2);
return init_sock(rb_obj_alloc(klass), fd2);
}
=20
Anyway nonblocking flag of accepted fd is not portable.
http://cr.yp.to/docs/unixport.html

So setting it unconditionally is good thing for portability.
--=20
Tanaka Akira
 
C

Christoffer Lernö

Christoffer Lern=F6 said:
Could this be due to me running

ruby 1.8.6 (2007-03-13 patchlevel 0) [i686-darwin8.10.1] ?

The funny thing was that only the socket returned by accept_nonblock
had this behaviour, that is

I think it is a Mac OS X problem.

Your example works without problem on GNU/Linux and FreeBSD
box but not on Mac OS X.

% cat z1.rb
require 'socket'
require 'fcntl'
a =3D TCPServer.new(5000)
socket =3D TCPSocket.new("localhost", 5000)
other =3D a.accept
p other.fcntl(Fcntl::F_GETFL) & Fcntl::O_NONBLOCK
puts "Start Write"
p other.write_nonblock("X" * 300000)
puts "End Write"
% ./ruby -v z1.rb
ruby 1.8.6 (2007-03-13 patchlevel 0) [powerpc-darwin8.3.0]
0
Start Write
81560
End Write

The IO returned from accept is not nonblocking mode.
So write_nonblock makes it nonblocking mode at first.

% cat z2.rb
require 'socket'
require 'fcntl'
a =3D TCPServer.new(5000)
socket =3D TCPSocket.new("localhost", 5000)
other =3D a.accept_nonblock
p other.fcntl(Fcntl::F_GETFL) & Fcntl::O_NONBLOCK
puts "Start Write"
p other.write_nonblock("X" * 300000)
puts "End Write"
% ./ruby -v z2.rb
ruby 1.8.6 (2007-03-13 patchlevel 0) [powerpc-darwin8.3.0]
4
Start Write
^Cz2.rb:8: Interrupt

The IO returned from accept_nonblock seems nonblocking mode.
So write_nonblock doesn't make it nonblocking mode.
But actually it is not nonblocking mode, I think.

Yeah, that seems to be it.

This workaround in the ruby code gets around the problem:

require 'socket'
require 'fcntl'
a =3D TCPServer.new(5000)
socket =3D TCPSocket.new("localhost", 5000)
other =3D a.accept_nonblock
other.fcntl(Fcntl::F_SETFL, other.fcntl(Fcntl::F_GETFL) & ~=20
(Fcntl::O_NONBLOCK))
puts "Start Write"
other.write_nonblock("X" * 300000)
puts "End Write"


/C
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
474,269
Messages
2,571,348
Members
48,026
Latest member
ArnulfoCat

Latest Threads

Top