Missing packets in multicast receiver.

[James W. Mills]

Hello list.

I need to use multicast in a project I am working on. I would like to
have a multicast sender send a large amount of multicast data, and a
multicast receiver receive it.

I used the example I found here:
http://onestepback.org/index.cgi/Tech/Ruby/MulticastingInRuby.red
to get started.

What I want initially is to be able to blast packets to a multicast
group. I wrote a simple multicast sender that prints a message every
(almost) MB of data it sends. I also wrote a simple receiver that does
the same thing for every (almost) MB of data it receives. Here they are.

Sender:
##############################
require 'socket'

MULTICAST_ADDR = "225.0.0.1"
PORT= 1200

begin
socket = UDPSocket.open
socket.setsockopt(Socket::IPPROTO_IP, Socket::IP_TTL, [1].pack('i'))
data_source = File.open '/dev/zero'
data_length = 0
loop do
data = data_source.read(1024)
data_length += data.length
if (data_length % 102400) == 0
puts data_length
end
socket.send(data, 0, MULTICAST_ADDR, PORT)
end
ensure
socket.close
end
##############################

Receiver:

##############################
require 'socket'
require 'ipaddr'

MULTICAST_ADDR = "225.0.0.1"
PORT = 1200

ip = IPAddr.new(MULTICAST_ADDR).hton + IPAddr.new("0.0.0.0").hton
sock = UDPSocket.new
sock.setsockopt(Socket::IPPROTO_IP, Socket::IP_ADD_MEMBERSHIP, ip)
sock.bind(Socket::INADDR_ANY, PORT)
start = Time.now
data_length = 0
begin
while true
msg, info = sock.recvfrom(1024)
data_length += msg.length
if data_length % 102400 == 0
puts data_length
end
end
ensure
sock.close
puts "Ran for #{Time.now - start}"
end
##############################

If you start the receiver, then start the sender, you will see that the
numbers do not line up at all. The receiver gets one MB for every 10 or
so MB that is sent.

Now, please don't fillet me over the coals for this, but I have the
receiver written in perl as well. The perl code does the same thing as
the Ruby code was meant to. Here it is.

##############################
use IO::Socket::Multicast;

# create a new UDP socket ready to read datagrams on port 1100
my $s = IO::Socket::Multicast->new(LocalPort=>1200);

# Add a multicast group
$s->mcast_add('225.0.0.1');

# now receive some multicast data

$data = "";
$t=0;

while(true) {
$s->recv($data,1024);
$t += length $data;
if($t % 102400 == 0) {
warn "$t";
}
}
##############################

If you run the perl receiver against the Ruby sender, the programs print
out the same messages, as expected. I am very curious to know what I am
doing wrong with the Ruby receiver that is causing it to miss so much
data. I suspect the line:

sock.setsockopt(Socket::IPPROTO_IP, Socket::IP_ADD_MEMBERSHIP, ip)

in the receiver might hold the answer, but so far nothing I have tried
has made any difference. I find it interesting that it is using TCP
instead of UDP/DGRAM, but I was unable to get it to work with IPPROTO_UDP...


Help is very much appreciated!

Thanks,
~james

 

[Douglas Wells]

Hello list.

I need to use multicast in a project I am working on. I would like to
have a multicast sender send a large amount of multicast data, and a
multicast receiver receive it.

Hi,

You seem to have a number of the networking ideas properly in your
head, but you also seem to be some erroreous fundamental concepts.
So, I'm going to go over some basics. (Please note that I'm also
going to rearrange your post for purposes of exposition.)

First, I should note that I didn't notice any useful clues about
your execution environment (other than your use of /dev/zero),
including OS and whether you were communicating between two different
networks hosts or looping back on one host. These differences
would introduce various effects on your symptoms.

Second, I should note that I ran your programs (all three) on my
system after making a few fundamental changes (more about that
later) and found that the ruby and perl receivers behave just about
the same. (Just for the record, I was running on FreeBSD using a
loop-around configuration)

So, let me start with your concluding analysis:

If you run the perl receiver against the Ruby sender, the programs print
out the same messages, as expected. I am very curious to know what I am
doing wrong with the Ruby receiver that is causing it to miss so much
data. I suspect the line:

sock.setsockopt(Socket::IPPROTO_IP, Socket::IP_ADD_MEMBERSHIP, ip)

in the receiver might hold the answer, but so far nothing I have tried
has made any difference. I find it interesting that it is using TCP
instead of UDP/DGRAM, but I was unable to get it to work with IPPROTO_UDP...

Here you show a fundamental understanding about the IP stack.
First, there's nothing here that has anything to do with TCP. If
there were, it would have used Socket::IPPROTO_TCP. IP (Internet
Protocol) underlies *both* TCP and UDP. In addition, multicast,
whose official name is "IP Multicast," has many differences from
UDP.

What I want initially is to be able to blast packets to a multicast
group. I wrote a simple multicast sender that prints a message every
(almost) MB of data it sends. I also wrote a simple receiver that does
the same thing for every (almost) MB of data it receives. Here they are.

Your use of the term "blast" here is telling. You need to understand
that IP Multicast (and UDP) are "unreliable" protocols. When you
transmit something using TCP, which is a "reliable" protocol, the
operating system transmits that the data and then holds onto a
copy until the receiver acknowledges successful receipt. At some
point, the OS will decide that you have consumed your fair share
of buffer space, and it will block you from sending any more data
until the receiver acknowledges delivery of some earlier data (and
the OS can safely discard the copy that it has been storing).

In the case of IP Multicast (and UDP), there is no back-channel
to provide such data. Thus, there will never be any acknowledgment
of received data and no cue to discard stored data. Instead, most
network stack implementations (that I am aware of) discard the
data immediately after it is transmitted over the wire.

Consider the case where your processor can generate data faster
than the network can transmit it, which is almost certainly the
case in your environment. Now consider what happens when you use
up your "fair share" of buffer space. What is the OS to do? It
boils to down to three choices: 1) block the process, 2) discard
some data silently, and 3) report an error. All three options have
been taken by various OSs at some point in the past. Today most
systems implement either (2) or (3). My system (FreeBSD) implements
(3), but previous BSD systems have implemented (2). I don't
immediately remember what Linux does.

Now let's look at your sender loop:

loop do
data = data_source.read(1024)
data_length += data.length
if (data_length % 102400) == 0
puts data_length
end
socket.send(data, 0, MULTICAST_ADDR, PORT)
end

You are looping here over and over with no intervening pause.
You're almost certainly spending most of your time calling into
the OS and having it discard data. The fact that your system
doesn't report any error on the socket.send makes me think that
your OS implements technique (2) above and silently files the data
in the bit bucket.

So why does the Perl program behave "better?" I don't know, and
frankly there are so many possible reasons that it's not worth my
while chasing. In order to make your program work (as described
above), I introduced a millisecond sleep every time the OS reported
that it had discarded data. (And I still flooded my screen with
notifications of the discards.)

Instead, you should take this exercise as an early warning that
your "blaster" application won't work as currently designed. If
you pore through networking chatter, you'll find lots of sage
advice by the networking grey beards that if you're going to use
unreliable datagrams instead of TCP, you will have to work our for
yourself how to achieve the level of reliability that you need for
your application. Since TCP is not an option for you, you're going
to have to solve the problem. Typically, in order to send large
amounts of data via datagrams (including via IP multicast), you
will need to monitor the amount of data that you are sending,
rate-limit it as appropriate, and possibly negotiate reserved
bandwidth if that should prove necessary.

You will also need to figure out what to do about dropped packets,
because the system WILL drop some packets -- even on single node
and/or dedicated network systems. You will either need to send
redundant data (as most audio/video systems do), or create an
auxiliary data correction channel (as some of the P2P file
distribution use).

~james

Good luck, - dmw