How to check client shutdown?

[Paul Pittlerson]

I'm currently learning about the socket module. My question is how can I detect if a connection is closed from the other side, for example a KeyboardInterrupt as I frequently use. My code below:

##########################################################
#server script:

class client(threading.Thread):
def __init__(self, connection):
super(client, self).__init__()

self.recv = connection.recv

print dir(connection)

def start(self):
self.run()

def run(self):
while True:
data = self.recv(2000)

if data:
msg = self.decode(data)
print msg
#break;

def decode(self, data):
return pickle.loads(data)


class bullet_server:
def __init__(self):
self.server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

def start(self):
server_address = ("localhost", 9006)
print '/--server running--/'
print server_address
self.server.bind(server_address)
self.server.listen(5)

while True:
connection, client_address = self.server.accept()

c = client(connection)
c.start()


if __name__ == '__main__':
server = bullet_server()
server.start()


##########################################################
#client script:

# set IP and port
host = '127.0.0.1'
port = 9006

# connect to server
connection = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
connection.connect((host, port))

def loop():
print 'Enter message:'
msg = raw_input()
encoded = pickle.dumps(msg)
connection.send(encoded)
loop()

loop()

 

[Chris Angelico]

I'm currently learning about the socket module. My question is how can I detect if a connection is closed from the other side, for example a KeyboardInterrupt as I frequently use. My code below:

Once the remote end has terminated (as it will presumably do in this
case), any attempt to write to the socket will eventually result in an
error. You should be able to see that by simple experimentation. Be
aware that it might take a bit of time for the error to filter back to
you; it depends on whether the remote end actually sends back a RST
(reset) packet or not. (In a sane system, hitting Ctrl-C should result
in a clean shutdown and proper socket closure, but your server needs
to handle the case where it doesn't get one.) Play around with it, you
should be able to figure out what it's doing for you.

A couple of tangential comments about your script:

##########################################################
#server script:

def decode(self, data):
return pickle.loads(data)

MAJOR security issue here. You are taking data from a networked source
and running it through a trusting system (pickle). This is NOT
recommended.

Also: You're using a stream socket, and depending - I think - on
getting an entire message in a single read() call. This is definitely
not guaranteed, though when you're working with localhost it's likely
to be true. To make this reliable, you'll need to do something like
prefix the message with its length (eg a Hollerith string), or use
some system like JSON that lets you detect the end of the message.

From the client:

def loop():
print 'Enter message:'
msg = raw_input()
encoded = pickle.dumps(msg)
connection.send(encoded)
loop()

loop()

Why recurse? Why not simply loop? No Python has tail-call optimization
(as far as I know), so this will eventually blow up with a
RuntimeError: maximum recursion depth exceeded.

ChrisA

 

[MRAB]

I'm currently learning about the socket module. My question is how
can I detect if a connection is closed from the other side, for
example a KeyboardInterrupt as I frequently use. My code below:

[snip]
When reading from a socket, it'll return as much data as is available,
up to the given limit if specified.

If _no_ data is available, then it'll block, unless the socket has been
closed at the other end, in which case it'll return zero bytes (an
empty bytestring).

 

[Grant Edwards]

I'm currently learning about the socket module. My question is how
can I detect if a connection is closed from the other side,

recv() will return an empty value of ''.

send() will eventually throw an exception. [You may be able to call
send() once or twice after the connection has been closed without
getting the exception.] On Linux the exception is socket.error with
errno 32 (broken pipe).

 

[Paul Pittlerson]

MAJOR security issue here. You are taking data from a networked source

and running it through a trusting system (pickle). This is NOT

recommended.

Security issue!? Do you mean someone could enter devious python h4xx into the chat or something? I had no idea using pickle was so dangerous, but I don't know any other method of transmitting data in python

Also: You're using a stream socket, and depending - I think - on

getting an entire message in a single read() call. This is definitely

not guaranteed, though when you're working with localhost it's likely

to be true. To make this reliable, you'll need to do something like

prefix the message with its length (eg a Hollerith string), or use

some system like JSON that lets you detect the end of the message.

I'm guessing the fix is to have a separate thread which only job is to sendinfo about the size of the next data transmission. What is the actual downside of having the server set to anticipate a message length which is knownto be more than will be sent (or be allowed to be sent?), for example connection.recv(10000). Does not the receiver know the size after the fact? Is it impacting performance somehow (I haven't noticed anything in my tests)

 

[Andrew Berg]

Security issue!? Do you mean someone could enter devious python h4xx into the chat or something? I had no idea using pickle was so dangerous, but I don't know any other method of transmitting data in python

JSON, XML, or any other format that doesn't have the capacity to serialize directly executable Python code. Pickles are okay for internal
data from trusted sources, but don't accept pickles from anyone you don't trust.
JSON is simple, easy, and can handle the common data types, and there is a module for it in the stdlib. It's also a standard format you can
use in pretty much any language, and it's human readable.

 

[Irmen de Jong]

Security issue!? Do you mean someone could enter devious python h4xx into the chat or
something? I had no idea using pickle was so dangerous, but I don't know any other
method of transmitting data in python

Shameless plug: have a look at my remote object library Pyro:
http://pythonhosted.org/Pyro4/

It transmits arbitrary objects to remote machines by what seem to be normal method
calls. In the past, it was restricted to using pickle as serialization format, but since
a couple of releases, it now defaults to another -safe- serializer. Because of the same
reason Chris is warning you about: unpickling data from untrusted sources can result in
arbitrary code execution in your server.


Main thing is: don't use pickle in your networking code unless you can guarantee the
trustworthiness of your sources. Instead, use another serialization format that is safe
(such as marshal, json, serpent).

I'm guessing the fix is to have a separate thread which only job is to send info
about the size of the next data transmission.

I'm not sure what you're proposing here. What's a separate thread got to do with things?

? What is the actual downside of having

the server set to anticipate a message length which is known to be more than will be
sent (or be allowed to be sent?), for example connection.recv(10000). Does not the
receiver know the size after the fact? Is it impacting performance somehow (I haven't
noticed anything in my tests)

The issue is that recv() is not guaranteed to return you the full amount of data that is
requested. It may very well just return a single byte, and leave the rest for later. The
argument is an upper bound on the amount of data you receive. So to make your recv
reliable, you need to have a means of deciding when the 'full' amount of data has been
collected. As Chris already suggested, this is usually done by putting the recv() in a
loop and collecting data until it reaches a length that you precisely know beforehand,
or by detecting a special end-of-message marker in the data stream, such as a newline.


Irmen de Jong

 

[Chris Angelico]

? What is the actual downside of having

The issue is that recv() is not guaranteed to return you the full amount of data that is
requested. It may very well just return a single byte, and leave the rest for later. The
argument is an upper bound on the amount of data you receive. So to make your recv
reliable, you need to have a means of deciding when the 'full' amount of data has been
collected. As Chris already suggested, this is usually done by putting the recv() in a
loop and collecting data until it reaches a length that you precisely know beforehand,
or by detecting a special end-of-message marker in the data stream, such as a newline.

Right. When you use TCP sockets, there's no boundaries, so you could
get two pickles in one recv, or you could get one and a half, or
anything. It depends partly on your buffer sizes and things; if you're
sending very short messages (less than a kilobyte), and have long
delays between them, chances are you'll get one write in one read; but
it's not guaranteed.

ChrisA

 

[Irmen de Jong]

Right. When you use TCP sockets, there's no boundaries, so you could
get two pickles in one recv, or you could get one and a half, or
anything. It depends partly on your buffer sizes and things; if you're
sending very short messages (less than a kilobyte), and have long
delays between them, chances are you'll get one write in one read; but
it's not guaranteed.

ChrisA

More horror stories: the same is also true for send().

Thankfully, Python provides a sendall(). But that doesn't work on some systems when the
socket is in non-blocking mode. Meh.

Irmen