Ruby Marshal unexpected results

[Aaron Vegh]

Hi there,
For a Rails project, I'm trying to serialize some data in a hash, using
Marshal. However, this data is getting serialized in unexpected ways.
I'm not sure if there's a problem with Marshal, my data, or me.

In Rails, I have a parameter that is a hash of data from a form:

{"305"=>"Complete trim detail at top of stringer where basement
staircase meets first floor level. ",
"25"=>"The vertical joint between the two adjacent masonry wythes at
the ... do not appear to be bonded or tied together. Cracking of the
mortar is resulting. \t\t\t\t\t\t
",
"728"=>"The mantle above factory built fireplace in the family room has
been installed in an un-workmanlike manner. Gaps between the mantle and
wall are too excessive. "}

In my controller, I serialize the data with Marshal:

data = Marshal.dump(p)

But when I save it, I get an error. Here's the beginning of it:

SQLite3::SQLException: unrecognized token:
"'C:HashWithIndifferentAccess{"305"": UPDATE "reports" SET
"updated_at" = '2009-10-07 14:49:26', "data" =
'C:HashWithIndifferentAccess..."

I logged the result of that Marshal call, and this is what I got:

C:HashWithIndifferentAccess305"`Complete trim detail at top of stringer
where basement staircase meets first floor level. "25"?The vertical
joint between the two adjacent masonry wythes at the ... do not appear
to be bonded or tied together. Cracking of the mortar is resulting.
728"?The mantle above factory built fireplace in the family room has
been installed in an un-workmanlike manner. Gaps between the mantle and
wall are too excessive.

Which to me doesn't look at all like what Marshal should be producing.
So what gives?

Thanks,
Aaron.

 

[7stud --]

Hi there,
For a Rails project, I'm trying to serialize some data in a hash, using
Marshal. However, this data is getting serialized in unexpected ways.
I'm not sure if there's a problem with Marshal, my data, or me.

In Rails, I have a parameter that is a hash of data from a form:

{"305"=>"Complete trim detail at top of stringer where basement
staircase meets first floor level. ",
"25"=>"The vertical joint between the two adjacent masonry wythes at
the ... do not appear to be bonded or tied together. Cracking of the
mortar is resulting. \t\t\t\t\t\t
",
"728"=>"The mantle above factory built fireplace in the family room has
been installed in an un-workmanlike manner. Gaps between the mantle and
wall are too excessive. "}

In my controller, I serialize the data with Marshal:

data = Marshal.dump(p)

But when I save it, I get an error. Here's the beginning of it:

SQLite3::SQLException: unrecognized token:
"'C:HashWithIndifferentAccess{"305"": UPDATE "reports" SET
"updated_at" = '2009-10-07 14:49:26', "data" =
'C:HashWithIndifferentAccess..."

I logged the result of that Marshal call, and this is what I got:

C:HashWithIndifferentAccess305"`Complete trim detail at top of stringer
where basement staircase meets first floor level. "25"?The vertical
joint between the two adjacent masonry wythes at the ... do not appear
to be bonded or tied together. Cracking of the mortar is resulting.
728"?The mantle above factory built fireplace in the family room has
been installed in an un-workmanlike manner. Gaps between the mantle and
wall are too excessive.

Which to me doesn't look at all like what Marshal should be producing.
So what gives?

Thanks,
Aaron.

require 'rubygems'
require 'sqlite3'

db = SQLite3:atabase.new( "mytest.db" )
db.execute("drop table mytable")
db.execute("create table mytable (data TEXT)")

h = {
"305"=>"Complete trim detail at top of stringer where basement
staircase meets first floor level. ",

"25"=>"The vertical joint between the two adjacent masonry wythes at
the ... do not appear to be bonded or tied together. Cracking of the
mortar is resulting. \t\t\t\t\t\t
",

"728"=>"The mantle above factory built fireplace in the family room has
been installed in an un-workmanlike manner. Gaps between the mantle and
wall are too excessive. "
}

str = Marshal.dump(h)
p str

--output:--
"\004\b{\b\"\b305\"`Complete trim detail at top of stringer where
basement\nstaircase meets first floor level. \"\a25\"\001\241The
vertical joint between the two adjacent masonry wythes at\nthe ... do
not appear to be bonded or tied together. Cracking of the\nmortar is
resulting. \t\t\t\t\t\t\n\"\b728\"\001\242The mantle above factory
built fireplace in the family room has\nbeen installed in an
un-workmanlike manner. Gaps between the mantle and\nwall are too
excessive. "


command = "insert into mytable (data) values ('#{str}')"
db.execute(command)
command = "insert into mytable (data) values ('hello world')"
db.execute(command)

rows = db.execute("select * from mytable")
p rows

--output:--
[["\004\b{\b\"\b305\"`Complete trim detail at top of stringer where
basement\nstaircase meets first floor level. \"\a25\"\001\241The
vertical joint between the two adjacent masonry wythes at\nthe ... do
not appear to be bonded or tied together. Cracking of the\nmortar is
resulting. \t\t\t\t\t\t\n\"\b728\"\001\242The mantle above factory
built fireplace in the family room has\nbeen installed in an
un-workmanlike manner. Gaps between the mantle and\nwall are too
excessive. "], ["hello world"]]

 

[Aaron Vegh]

7stud -- wrote:

[ perfectly-functional code demonstrating Marshal and SQLite3 at work]

Nice looking code! But that doesn't solve my problem, I'm afraid. I've
used Marshal before many times. My question isn't "how do I use it", but
"why the heck is it not serializing my data properly?

Apologies if I don't spy the solution within your code, but you didn't
provide any commentary with your code...

Thanks,
Aaron.

 

[John W Higgins]

[Note: parts of this message were removed to make it a legal post.]

Aaron,

The short answer appears to be that you are not using a hash but rather the
HashWithIndifferentAccess class which apparently in combination with Marshal
spits out characters (I'm guessing the tick at the end of this section right
here "\004\b{\b\"\b305\"`Complete) which are not conducive to storage within
SQLite.

My quick and dirty suggestion would be to switch your data back to a raw
hash prior to SQLite storage and then reverse the process on the way back
out of the database.

For example,

data = Marshal.dump(p.to_hash)

and

p = HashWithIndifferentAccess.new(data)

John

 

[Aaron Vegh]

My quick and dirty suggestion would be to switch your data back to a raw
hash prior to SQLite storage and then reverse the process on the way
back out of the database.

That was a good suggestion! It pushed me further along, that's for sure.
Your suggestion that it may be related to bad characters really tipped
me off. I'm now employing strip on the text that does into the form
that's submitted, so no whitespace surrounds the content I'm processing.
But there does appear to be a rogue character that is hanging up the
SQLite database. Here's the current result of my serialization:

740"?The mantle...

The quote-question mark is what appears to be causing the problem, but I
can't identify where that's coming from. I'm really scratching my head
over this one.

Thanks,
Aaron.

 

[Ryan Davis]

For a Rails project, I'm trying to serialize some data in a hash,
using
Marshal. However, this data is getting serialized in unexpected ways.
I'm not sure if there's a problem with Marshal, my data, or me.

1) this is a rails question, not a ruby question. You'd be better off
asking over there.

2) Doesn't activerecord already support object serialization out of
the box?