Limitate speed of a socket-based data transferring

[billie]

Hi all. I'm writing a TCP-based application that I will use to trasfer
binary files through the network. This piece of code represents how do
I get a file from a remote peer and save it on my local hard drive:

file_obj = open('downloaded.ext', 'wb')
while 1:
buf = sock.recv(2048)
if len(buf) == 0:
break
file_obj.write(buf)
file_obj.close()
sock.close()

I would like to know how could be possible to limit the file transfer
speed (for example: don't write more than 50 Kb/sec).
Some ideas?

Best regards

 

[Diez B. Roggisch]

Hi all. I'm writing a TCP-based application that I will use to trasfer
binary files through the network. This piece of code represents how do
I get a file from a remote peer and save it on my local hard drive:

file_obj = open('downloaded.ext', 'wb')
while 1:
buf = sock.recv(2048)
if len(buf) == 0:
break
file_obj.write(buf)
file_obj.close()
sock.close()

I would like to know how could be possible to limit the file transfer
speed (for example: don't write more than 50 Kb/sec).
Some ideas?

If you are on unix, use trickle. If you must limit it from within your
own code, I can only assume that computing the transfer rate so far and
introducing timeouts might help - but I never did such a thing, nor do I
know what that means for example for the network stack.

But maybe even then trickle may help you to get an idea, as it is a
user-space program AFAIK. So they might have some information (at least
the code... ) out there that could be of use.


Diez

 

[Tim Williams]

If you are on unix, use trickle. If you must limit it from within your
own code, I can only assume that computing the transfer rate so far and
introducing timeouts might help - but I never did such a thing, nor do I
know what that means for example for the network stack.

But maybe even then trickle may help you to get an idea, as it is a
user-space program AFAIK. So they might have some information (at least
the code... ) out there that could be of use.

You could wrap buf = sock.recv(xxx) in a data counter and sleep loop
so that you burst to no more than 50KB/s average.

Completely untestest and for illustration only

file_obj = open('downloaded.ext', 'wb')
interval = 1.0 # seconds eg. 0.5 or 2.0
# smaller the interval, the less bursty and smoother the throughput
max_speed = 51200 # 50k * 1024 = bytes
data_count = 0 # keep track of the amount of data transferred
time_next = time.time() + interval
while 1:
buf = sock.recv(512) # smaller chunks = smoother, more accurate
if len(buf) == 0:
break
data_count += len(buf)
if data_count >= max_speed * interval:
data_count = 0
sleep_for = time_next - time.time()
if sleep_for > 0:
time.sleep(sleep_for)
time_next = time.time() + interval
file_obj.write(buf)
file_obj.close()
sock.close()

 

[Dennis Lee Bieber]

Hi all. I'm writing a TCP-based application that I will use to trasfer
binary files through the network. This piece of code represents how do
I get a file from a remote peer and save it on my local hard drive:

I would like to know how could be possible to limit the file transfer
speed (for example: don't write more than 50 Kb/sec).
Some ideas?

I'm not sure you /can/ limit the /receive/ speed. The sender will
send at whatever rate they are capable of, so packets may just become
backlogged on your receiving socket waiting for you to read them if you
add some sort of delay to your reading loop.

--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/

 

[Grant Edwards]

I'm not sure you /can/ limit the /receive/ speed.

Sure you can. Just read data from the socket at the max speed
you want to receive. The receive buffer for that socket will
fill up and the TCP window will start close up and throttle the
sender.

The sender will send at whatever rate they are capable of, so
packets may just become backlogged on your receiving socket

When that happens, the sending end of the socket will throttle
down to match the rate at which data is being read from the
socket.

 

[Steve Holden]

Sure you can. Just read data from the socket at the max speed
you want to receive. The receive buffer for that socket will
fill up and the TCP window will start close up and throttle the
sender.




When that happens, the sending end of the socket will throttle
down to match the rate at which data is being read from the
socket.

Of course this depends crucially on the window size. Since the addition
of the window scaling TCP option it's been possible to specify very
large windows, which are useful over high-bandwidth high-delay links.

The remote (send) throttling will only start to cut in when the window
is full (since the whole point of the sliding window mechanism is to
allow continuous transmission in the face of acknowledgment delay).

regards
Steve

 

[Grant Edwards]

Of course this depends crucially on the window size. Since the
addition of the window scaling TCP option it's been possible
to specify very large windows, which are useful over
high-bandwidth high-delay links.

True. If the window size is large compared to the amount of
data being transferred, then the throttling won't happen.

The remote (send) throttling will only start to cut in when
the window is full (since the whole point of the sliding
window mechanism is to allow continuous transmission in the
face of acknowledgment delay).

Yup.

 

[Tim Williams]

True. If the window size is large compared to the amount of
data being transferred, then the throttling won't happen.


Yup.

If the OP is also writing the client, the throttling can be at the
client side to control the speed of writing to the socket. But if
multiple clients connect to the server concurrently then this may not
help either!