Sockets: code works locally but fails over LAN

[n00m]

1.
Python 2.3.4

2.
Win98 and Win2k Professional

 

[Dennis Lee Bieber]

No, my guess is that you're running an old version of Python.
The constant was added in the source on 27 Nov 2003; I'm not

Are you sure of that 2003?

C:\Documents and Settings\Dennis Lee Bieber>python
ActivePython 2.3.5 Build 236 (ActiveState Corp.) based on
Python 2.3.5 (#62, Feb 9 2005, 16:17:08) [MSC v.1200 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: 'module' object has no attribute 'SHUT_WR'

Would appear to be new with 2.4.x (I believe the reason I'm still on
2.3.x is that Plone didn't like 2.4.x and I couldn't seem to find a
compatible set of paths to have Plone work with its Python without
conflicting with a separate ActiveState install. As it is, it took some
games to get AS into the Plone directory.)
--

 

[Bryan Olson]

Bryan Olson declaimed the following in comp.lang.python:



Are you sure of that 2003?

Yes, but that's when it went into the source, in the sense of
first being edited into a file. It was Revision 1.279 of
socketmodule.c.

http://cvs.sourceforge.net/viewcvs....src/Modules/socketmodule.c?rev=1.312&view=log

 

[n00m]

Bryan;
Look at how I corrected your the very first version
(see added arguments in both functions). And now it
really can handle multiple connections!


import socket, thread

sqls_host, sqls_port = '127.0.0.1', 1433
proxy_host, proxy_port = '127.0.0.1', 1434

# How I tested it:
# sqls_host, sqls_port = 'www.google.com', 80

def VB_SCRIPT(s2, cn):
while 1:
data = cn.recv(4096)
if not data: return
s2.sendall(data)
print 'VB_SCRIPT:' + data + '\n'

def SQL_SERVER(s2, cn):
while 1:
data = s2.recv(4096)
if not data: return
cn.sendall(data)
print 'SQL_SERVER:' + data + '\n'

s1 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s1.bind((proxy_host, proxy_port))
s1.listen(5)

while 1:
cn, addr = s1.accept()
s2 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s2.connect((sqls_host, sqls_port))
thread.start_new_thread(VB_SCRIPT,(s2, cn))
thread.start_new_thread(SQL_SERVER,(s2, cn))


Without these corrections I got these error messages
when I launched SIMULTANEOUSLY 3 instances of my.vbs:


Unhandled exception in thread started by
Unhandled exception in thread started by
Traceback (most recent call last):
Traceback (most recent call last):
File "D:\Python23\00\socket_Br10.py", line 18, in SQL_SERVER
File "D:\Python23\00\socket_Br10.py", line 13, in VB_SCRIPT
data = s2.recv(4096)
s2.sendall(data)
socket File "<string>", line 1, in sendall
..socketerror.: error: (10054, 'Connection reset by peer')
(10054, 'Connection reset by peer')

 

[Bryan Olson]

> Bryan;
> Look at how I corrected your the very first version
> (see added arguments in both functions). And now it
> really can handle multiple connections!

Ah, yes, I see. (In my defense, I had already fixed that bug in
my second version.)

 

[n00m]

Ah, yes, I see. (In my defense, I had already fixed that bug in
my second version.)

1.
Yes! I myself noticed that, but your 2nd version looks
a bit more verbose.
2.
This all means... what? ONLY send() vs sendall() matters?
Sometimes send() really sends ALL and my version works too!
I must give it a thorough testing!
3.
See how it looks in SQL Server Profiler (it's its utility for
tracing client/server events) WHEN I started 5 copies of .vbs.
http://free.7host02.com/n00b/SQL_Profiler.gif

Btw, without s2.shutdown(1) those vbs' do NOT disconnect from
the server (see DISCONNECT events on the gif).


The latest python version:
==============================
import socket, thread

sqls_host, sqls_port = '127.0.0.1', 1433
proxy_host, proxy_port = '127.0.0.1', 1434

# How I tested it:
# sqls_host, sqls_port = 'www.google.com', 80

def VB_SCRIPT(s2, cn):
while 1:
data = cn.recv(4096)
if not data:
s2.shutdown(1)
return
s2.sendall(data)
print 'VB_SCRIPT:' + data + '\n'

def SQL_SERVER(s2, cn):
while 1:
data = s2.recv(4096)
if not data: return
cn.sendall(data)
print 'SQL_SERVER:' + data + '\n'

s1 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s1.bind((proxy_host, proxy_port))
s1.listen(5)

while 1:
cn, addr = s1.accept()
s2 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s2.connect((sqls_host, sqls_port))
thread.start_new_thread(VB_SCRIPT,(s2, cn))
thread.start_new_thread(SQL_SERVER,(s2, cn))


The vbs text:
==============
Set cn = CreateObject("ADODB.Connection")
cn.Open _
"Provider=sqloledb;Data Source=127.0.0.1,1434;" & _
"Network Library=DBMSSOCN;Initial Catalog=pubs;" & _
"User ID=qwe;Password=asdasd;"
cn.Execute _
"select 'AAAAAAAAAAAAAAA';" & _
"waitfor delay '00:00:02'; raiserror('XXX',10,1) with nowait;" & _
"waitfor delay '00:00:02'; raiserror('YYY',10,1) with nowait;" & _
"waitfor delay '00:00:02'; raiserror('ZZZ',10,1) with nowait;" & _
"select 'BBBBBBBBBBBBBBB';"
cn.Close
Set cn = Nothing

 

[Phill Atwood]

Newbie question:

I have a dictionary called "rec". And a list called "db". db is my
database. rec are to be records in the database. In a loop I'm trying to
create a record and add it to my database. The key stmt is

db.append( rec )

This works initially but it seems that instead of copying the values in
rec and adding it to db, only a reference of rec is added to db. Thus
when I reassign rec with new data and then do another append I end up
with two records in my database but now the both contain the same data,
namely the latest version or rec.

So how do I add a dictionary into a list by value rather than by reference?

The complete code is here:

=========================================================
#!/usr/bin/python

import sys

inputfile = sys.argv[1]

keywords = [ 'place-name=',
'addr-line1=',
'addr-line2=',
'addr-city=',
'addr-state=',
'addr-country=',
'addr-postalcode=',
'mailaddr-line1=',
'mailaddr-line2=',
'mailaddr-city=',
'mailaddr-state=',
'mailaddr-country=',
'mailaddr-postalcode=',
'place-phone1=',
'place-phone2=',
'place-fax=',
'place-email1=',
'place-email2=' ]

def csv_it():
db = [] # the entire database of records
rec = {} # a single rec: a dictionary of field names and data pairs
fields = [] # list of field names collected so far for current record
for line in open(inputfile):
kword = getkeyword(line) # get keyword (or field name)
if kword:
# We've got a line with a keyword in it
if kword == keywords[0]:
# Starting a new record, so save current record
db.append( rec )
printdb(db, "stage 1")

# Now clear current record
fields = []
rec.clear()
dummylist = []
for i in range(1,len(keywords)+1):
dummylist.append('')
for k,d in zip(keywords,dummylist):
rec[k] = d
printdb(db, "stage 2")

# make sure we are not encountering a duplicate key word
if kword in fields:
print "Record contains duplicate field"
print line,
sys.exit()
fields.append(kword)
# collect our data and store it in the current record
data = line[line.find('=')+1:]
datalstrip = data.lstrip()
datarstrip = datalstrip.rstrip()
rec[kword] = datarstrip

printdb(db,"stage 3")
db.append( rec ) # don't forget whatever we have in our last record

# dump the database in comma separated value form
outstring = ''
for k in keywords:
outstring += '"'
outstring += k
outstring += '"'
outstring += ','
print outstring
for r in db:
outstring = ''
for k in keywords:
if r[k]:
outstring += '"'
outstring += r[k]
outstring += '"'
outstring += ','
print outstring


def getkeyword(line):
equal_index = line.find('=')
if equal_index >= 0:
kword = line[0:equal_index+1]
if kword in keywords:
return kword
else:
print "Invalid keyword at line:"
print line
sys.exit()
return None

def printdb( db, text ):
print " "
print "db %s = " % text
print db
print " "



if __name__ == '__main__':
csv_it()

====================================================

Thanks
Phill

 

[Steve Holden]

Newbie question:

I have a dictionary called "rec". And a list called "db". db is my
database. rec are to be records in the database. In a loop I'm trying to
create a record and add it to my database. The key stmt is

db.append( rec )

This works initially but it seems that instead of copying the values in
rec and adding it to db, only a reference of rec is added to db. Thus
when I reassign rec with new data and then do another append I end up
with two records in my database but now the both contain the same data,
namely the latest version or rec.

So how do I add a dictionary into a list by value rather than by reference?

The complete code is here:

[code snipped]

I see you do a rec.clear() to clear out the contents of one record
before creating a new one. Python only *ever* uses references, so why
not just use

rec = {}

instead of rec.clear()?

regards
Steve

 

[Terry Reedy]

def csv_it():
db = [] # the entire database of records
rec = {} # a single rec: a dictionary of field names and data pairs

Move this line

fields = [] # list of field names collected so far for current record
for line in open(inputfile):

to here

kword = getkeyword(line) # get keyword (or field name) ....
# Now clear current record
fields = []
rec.clear()

and delete this

dummylist = []

and I think you will have what you want (or close).
The initial fields = [] line could be moved also, I believe, without
thoroughly reading your code.

Terry J. Reedy

 

[Bryan Olson]

Phill Atwood wrote:
[...]

> So how do I add a dictionary into a list by value rather than
> by reference?

Is rec.items() what you want? It returns a list of (key, value)
tuples.

> The complete code is here:

[...]

Looks like you could use Python's ConfigParser module.

http://docs.python.org/lib/module-ConfigParser.html

 

[n00m]

I was trying to test the send() vs sendall() like this:

x=send(data)
print len(data)-x > 0 ? (when the code fails)

but I could not reproduce the failures anymore.
As if the lan got "refreshed" after the first
using of sendall() instead of send().

Btw, why we need send() if there is sendall()?

 

[Bryan Olson]

> Btw, why we need send() if there is sendall()?

Mostly because sendall() can block, even if you do all the
select() and setblocking() magic. That's no problem in the
threaded architecture we're using, but a deal-breaker for a
single-threaded server.

 

[n00m]

Thanks, Bryan, for the details!

Btw, the newest oops in the topic's subject is:
the code does not work in the case of:

sqls_host, sqls_port = '192.168.0.8', 1433
proxy_host, proxy_port = '192.168.0.3', 1434
## proxy_host, proxy_port = '127.0.0.1', 1434
## proxy_host, proxy_port = '', 1434

I.e. when both Python and vbs script run on one machine
(with ip = 192.168.0.3) and SQL Server runs on another
(with ip = 192.168.0.8)

How namely it does not work:
in the idle window only one line is printed:

VB_SCRIPT:
.........

that's all. No errors from Python. After timeout expires
I get an error message from VBS (smth like preHandShake()
failed; I've never seen it before).

I just wonder MUST (or not) it work at all (IN THEORY)?

PS: again, without Python vbs and sql server contact with
each other PERFECTLY.

 

[Bryan Olson]

n00m wrote:
[...]

> Btw, the newest oops in the topic's subject is:
> the code does not work in the case of:
>
> sqls_host, sqls_port = '192.168.0.8', 1433
> proxy_host, proxy_port = '192.168.0.3', 1434
> ## proxy_host, proxy_port = '127.0.0.1', 1434
> ## proxy_host, proxy_port = '', 1434
>
> I.e. when both Python and vbs script run on one machine
> (with ip = 192.168.0.3) and SQL Server runs on another
> (with ip = 192.168.0.8)
>
> How namely it does not work:
> in the idle window only one line is printed:
>
> VB_SCRIPT:
.........
>
> that's all. No errors from Python. After timeout expires
> I get an error message from VBS (smth like preHandShake()
> failed; I've never seen it before).
>
> I just wonder MUST (or not) it work at all (IN THEORY)?

No theoretically-must-or-even-should-work solution is generally
possible. Protocols can thwart proxies by using addresses that
they transfer within their payload-data. That's an essential
feature of some security protocols, such as SSL, and an
unfortunate consequence of some old or badly-designed protocols.

Were I a betting man (outside of Texas Hold'em, where I am), I'd
wager that your problem is more basic. The code you're running,
the settings you're setting, or -- well -- something there, is
not right.