background on String#unpack?

[Giles Bowkett]

I'm just looking for historical context on this method. It looks like
one of those old-school Unix-isms that must have been incredibly
useful in the past but never comes up in the course of run-of-the-mill
Web programming. I'm pretty sure I first saw it ten years ago in Perl
and never once used it myself. I think I did copy/paste it in some
encryption code or something related, but I'm not sure. Just
wondering, what's it for? Who uses it, and why?

--
Giles Bowkett

Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com/

 

[Joel VanderWerf]

I'm just looking for historical context on this method. It looks like
one of those old-school Unix-isms that must have been incredibly
useful in the past but never comes up in the course of run-of-the-mill
Web programming. I'm pretty sure I first saw it ten years ago in Perl
and never once used it myself. I think I did copy/paste it in some
encryption code or something related, but I'm not sure. Just
wondering, what's it for? Who uses it, and why?

Array#pack and String#unpack are useful for handling _binary_ network
protocols and file formats.

 

[Giles Bowkett]

encryption code or something related, but I'm not sure. Just

Array#pack and String#unpack are useful for handling _binary_ network
protocols and file formats.

So would this be accurate? Use cases would include implementing
graphics formats and extracting text from proprietary-format word
processor files?

--
Giles Bowkett

Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com/

 

[James Edward Gray II]

So would this be accurate? Use cases would include implementing
graphics formats and extracting text from proprietary-format word
processor files?

Right. I wrote about that some in the summary to this quiz:

http://www.rubyquiz.com/quiz136.html

James Edward Gray II

 

[Rick DeNatale]

I'm just looking for historical context on this method. It looks like
one of those old-school Unix-isms that must have been incredibly
useful in the past but never comes up in the course of run-of-the-mill
Web programming. I'm pretty sure I first saw it ten years ago in Perl
and never once used it myself. I think I did copy/paste it in some
encryption code or something related, but I'm not sure. Just
wondering, what's it for? Who uses it, and why?

String#unpack, along with its partner Array#pack is used for dealing
with binary input and output streams respectively.

 

[Thomas Adam]

So would this be accurate? Use cases would include implementing
graphics formats and extracting text from proprietary-format word
processor files?

Anything that's encoded (URL-encoding, for instance), binary encoding.
Binary data sent down from pipes/sockets, etc.

Just look at the formatting options for it.

-- Thomas Adam

 

[Tim Hunter]

So would this be accurate? Use cases would include implementing
graphics formats and extracting text from proprietary-format word
processor files?

I had occasion to use these methods in code dealing with IPTC (image)
metadata.

 

[Giles Bowkett]

So would this be accurate? Use cases would include implementing

Anything that's encoded (URL-encoding, for instance), binary encoding.
Binary data sent down from pipes/sockets, etc.

Just look at the formatting options for it.

I think that's one of the times I've used it. URL-decoding in Perl
before the days of CGI.pm. I think I've seen it or used it in
JavaScript that way as well, actually, for URLs.

--
Giles Bowkett

Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com/

 

[Giles Bowkett]

Here's a real world example where I am trying to figure out how to use

string#unpack. I'm using the ruby-net-ldap library to pull info from Active
Directory. One of the fields, objectguid, comes back as a binary field. I
need to be able to compare it to (I think, I'm no expert here) what looks
like a hexidecimal representation of this field. So, I'm trying to figure
out how to convert this binary field I get back from Active Directory, into
a hexidecimal field. I think I need to use the unpack method, but I haven't
figured out what format string to pass the method yet.

Didn't mean to hijack the thread, just thought it kind of tied in with what
you were asking.

Here's a Perl snippet that seems to do it:

$binary = unpack("B32", pack("N", hex($hex)));

presumably the Ruby equivalent would be similar. although actually I
think this converts hex to binary instead of converting binary to hex.

There's a Perl tutorial on it
(http://perldoc.perl.org/perlpacktut.html) but I don't know how much
the syntax for pack/unpack statements in Perl and Ruby conform or
differ. Seems like an old-school Unix DSL, like regexen.

--
Giles Bowkett

Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com/

 

[Stefan Rusterholz]

I'm just looking for historical context on this method. It looks like
one of those old-school Unix-isms that must have been incredibly
useful in the past but never comes up in the course of run-of-the-mill
Web programming. I'm pretty sure I first saw it ten years ago in Perl
and never once used it myself. I think I did copy/paste it in some
encryption code or something related, but I'm not sure. Just
wondering, what's it for? Who uses it, and why?

--
Giles Bowkett

Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com/

Since it came up here and is somewhat related: I recently wrote out of
fun some wrapper methods to make it a bit nicer to read/write binary
datastructures. IMHO it's a bit cumbersome to do things like:
size = socket.read(4).unpack("I").first
meta = SomeStruct.new(*socket.read(x).unpack(format))
data = socket.read(size)

The wrappers are attached and here: http://pastie.caboo.se/111365
Use it at your own risk ;-)

Regards
Stefan

Attachments:
http://www.ruby-forum.com/attachment/790/packed.rb

 

[Joel VanderWerf]

Since it came up here and is somewhat related: I recently wrote out of
fun some wrapper methods to make it a bit nicer to read/write binary
datastructures. IMHO it's a bit cumbersome to do things like:
size = socket.read(4).unpack("I").first
meta = SomeStruct.new(*socket.read(x).unpack(format))
data = socket.read(size)

See also:

binaryparse
bindata
bitstruct

The first two are gems, the last is tgz, available at
http://redshift.sourceforge.net/bit-struct (that one's mine).