Character set woes with binary data

[Michael B. Trausch]

I am attempting to piece together a Python client for Fotobilder, the
picture management server on Livejournal.

The protocol calls for binary data to be transmitted, and I cannot seem
to be able to do it, because I get this error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "scrapbook.py", line 181, in UploadSinglePicture
{Request['UploadPic.Meta.Filename']: pic_mem})
File "scrapbook.py", line 237, in ComposeMIME
return(self.EncodeMIME(fields, files))
File "scrapbook.py", line 226, in EncodeMIME
body = eol.join(L)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 0:
ordinal not in range(128)
When putting the MIME segments (listed line-by-line in a Python list)
together to transmit them. The files are typically JPG or some other
binary format, and as best as I understand the protocol, the binary data
needs to be transmitted directly (this is evidenced by looking at the
tcp-stream of an existing client for uploading files).

It seems that Python thinks it knows better than I do, though. I want
to send this binary data straightaway to the server.

This is a hex dump of what one file looks like being uploaded to the
server (partial; the file is 3.8 MB):

000001CB ff d8 ff e1 3b fc 45 78 69 66 00 00 49 49 2a 00 ....;.Ex
if..II*.
000001DB 08 00 00 00 09 00 0f 01 02 00 10 00 00 00 7a
00 ........ ......z.
000001EB 00 00 10 01 02 00 10 00 00 00 aa 00 00 00 12
01 ........ ........
000001FB 03 00 01 00 00 00 01 00 00 00 1a 01 05 00 01
00 ........ ........
0000020B 00 00 da 00 00 00 1b 01 05 00 01 00 00 00 e2
00 ........ ........
0000021B 00 00 28 01 03 00 01 00 00 00 02 00 00 00 31
01 ..(..... ......1.
0000022B 02 00 1e 00 00 00 ea 00 00 00 13 02 03 00 01
00 ........ ........
0000023B 00 00 02 00 00 00 69 87 04 00 01 00 00 00 54
01 ......i. ......T.
0000024B 00 00 ac 12 00 00 48 65 77 6c 65 74 74 2d 50 61 ......He
wlett-Pa
0000025B 63 6b 61 72 64 00 00 00 00 00 00 00 00 00 00 00
ckard... ........
0000026B 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 ........ ........
0000027B 00 00 00 00 00 00 50 68 6f 74 6f 73 6d 61 72 74 ......Ph
otosmart
0000028B 20 4d 35 32 35 00 00 00 00 00 00 00 00 00 00 00
M525... ........
0000029B 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 ........ ........
000002AB 00 00 00 00 00 00 e6 00 00 00 01 00 00 00 e6
00 ........ ........
000002BB 00 00 01 00 00 00 56 65 72 73 69 6f 6e 20 31 2e ......Ve
rsion 1.
000002CB 34 31 30 30 2c 53 4e 3a 43 4e 36 34 31 44 33 31 4100,SN:
CN641D31
000002DB 4a 35 53 00 00 00 00 00 00 00 00 00 00 00 00 00
J5S..... ........
000002EB 00 00 00 00 00 00 ff ff ff ff ff ff ff ff ff
ff ........ ........
000002FB ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
ff ........ ........
0000030B ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
ff ........ ........
0000031B ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
ff ........ ........
0000032B 27 00 9a 82 05 00 01 00 00 00 96 08 00 00 9d 82
'....... ........

Is there any way to tell Python to ignore the situation and treat the
entire thing as simply a stream of bytes? I cannot seem to find one,
though I have found a great many posts on this mailing list regarding
issues in the past. It doesn't look like translating the file to base64
is an option for me.

— Mike

--
Michael B. Trausch
(e-mail address removed)
Phone: (404) 592-5746
Jabber IM:
(e-mail address removed)
(e-mail address removed)
Demand Freedom! Use open and free protocols, standards, and software!

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQBGD2uF0kE/IBnFmjARAn4BAJ4658n45aZmSUUlwt8tXrW1CuYg1ACcD2l6
xurwY+hB406KiEJ9RFQnA8k=
=gfl/
-----END PGP SIGNATURE-----