How to encode binary for http.post()

[Randy Lawrence]

I'm having problems posting a binary string (around 256 characters)
using http.post().

When I encode the binary string using Base64.encode64() and then
http.post() the data, the "+" characters are replaced by " " so the data
is no longer valid.

I've tried URI.escape(Base64(encode64(data))) to post and
URI.unescape(data) to receive but the same problem appears.

Any suggestions?

 

[Mark Hubbart]

I'm having problems posting a binary string (around 256 characters)
using http.post().

When I encode the binary string using Base64.encode64() and then
http.post() the data, the "+" characters are replaced by " " so the
data is no longer valid.

I've tried URI.escape(Base64(encode64(data))) to post and
URI.unescape(data) to receive but the same problem appears.

Any suggestions?

The rfc that describes the base64 encoding has a slightly different
version of the encoding that is url/filename-safe. It is rather simple
to tr the characters around to get the right results:

http://www.faqs.org/rfcs/rfc3548.html

something like this:

require 'base64'
=> nil
def Base64.encode64url(str)
Base64.encode64(str).tr("+/","-_")
end
=> nil
def Base64.decode64url(str)
Base64.decode64(str.tr("-_","+/"))
end
=> nil
(data = (0..255).inject(""){|a,v|a<<v}).size # all bytes from 0 to 255
=> 256
(encoded = Base64.encode64url(data)).count "+/" # any prohibited
characters?
=> 0
# raise error if encoded data is incorrect
x=0; Base64.decode(encoded).each_byte{|b| raise if x!=b; x=b+1}; 0
=> 0

I wonder if this should be added to the module? In the rfc, it seems to
be a standard extension, for use in filenames, urls, etc.

Additionally, it might be nice to have a base32 implementation shipped.
I've coded it myself twice, I recall, and I think it takes all of 20
loc at most.

cheers,
Mark

 

[Randy Lawrence]

The rfc that describes the base64 encoding has a slightly different
version of the encoding that is url/filename-safe. It is rather simple
to tr the characters around to get the right results:

http://www.faqs.org/rfcs/rfc3548.html

something like this:

require 'base64'
=> nil
def Base64.encode64url(str)
Base64.encode64(str).tr("+/","-_")
end
=> nil
def Base64.decode64url(str)
Base64.decode64(str.tr("-_","+/"))
end
=> nil
(data = (0..255).inject(""){|a,v|a<<v}).size # all bytes from 0 to 255
=> 256
(encoded = Base64.encode64url(data)).count "+/" # any prohibited
characters?
=> 0
# raise error if encoded data is incorrect
x=0; Base64.decode(encoded).each_byte{|b| raise if x!=b; x=b+1}; 0
=> 0

I wonder if this should be added to the module? In the rfc, it seems to
be a standard extension, for use in filenames, urls, etc.

Additionally, it might be nice to have a base32 implementation shipped.
I've coded it myself twice, I recall, and I think it takes all of 20 loc
at most.

cheers,
Mark

I ran into problems precisely because of this issue. I had "+" showing
up as " ".

I simply replace spaces in Base64 encoded string to "+" as a workaround
but would love something more appropriate/standard to use.

 

[Mark Hubbart]

I ran into problems precisely because of this issue. I had "+"
showing up as " ".

I simply replace spaces in Base64 encoded string to "+" as a
workaround but would love something more appropriate/standard to use.

I'm not sure what you mean... those two functions, added to the Base64
module, should solve the issue. They just implement more of the Base64
standard. The RFC describes base64 encoding, then it gives an
alternative encoding for use in urls and filenames, where the + and /
are replaced with - and _.

cheers,
Mark

 

[Mark Hubbart]

Oops. Sorry to reply to my own post. I made a mistake.

something like this:

require 'base64'
=> nil
def Base64.encode64url(str)
Base64.encode64(str).tr("+/","-_")
end
=> nil
def Base64.decode64url(str)
Base64.decode64(str.tr("-_","+/"))
end
=> nil
(data = (0..255).inject(""){|a,v|a<<v}).size # all bytes from 0 to
255
=> 256
(encoded = Base64.encode64url(data)).count "+/" # any prohibited
characters?
=> 0
# raise error if encoded data is incorrect

=> 0

the last line should be:
x=0; Base64.decode64url(encoded).each_byte{|b| raise if x!=b; x=b+1};
0
=> 0

(I copied from various places in my testing window; I didn't check for
consistency)

cheers,
Mark