Using File.open block to manage svnadmin dump stream

[Brad C.]

While working on a script to manage dumps from a subversion repository
of mine, I found that when I used a File.open block to work with the
stream returned by svnadmin dump, I would get errors regarding a
malformed header in the dump file. For instance:

# The following block results in bad dump files, not accepted by
svnadmin load
File.open("path/to/todays/dumpfile", "a") do |file|
file << `svnadmin dump "path/to/repository" -r #{revision}
--incremental`
end

# However, a strait command expression does work, like
`svnadmin dump "path/to/repository" -r #{revision} --incremental >>
"path/to/todays/dumpfile"`

Might anyone know why the use of a File.open block would result in a
dump file not accepted by svnadmin load, while a strait command
expression works? I tried using the << and write methods of File, but
still no luck. Perhaps I don't understand a more appropriate way to
write the stream?

Thanks for taking a look.
Brad

 

[Stephen Waits]

# The following block results in bad dump files, not accepted by
svnadmin load
File.open("path/to/todays/dumpfile", "a") do |file|
file << `svnadmin dump "path/to/repository" -r #{revision}
--incremental`
end

# However, a strait command expression does work, like
`svnadmin dump "path/to/repository" -r #{revision} --incremental >>
"path/to/todays/dumpfile"`

Can you do it both ways and then diff the two files to see what's
actually different?

Also, try using write (or, failing that, puts) instead of << to output
to the file.

--Steve

 

[Pia Kraft]

While working on a script to manage dumps from a subversion repository
of mine, I found that when I used a File.open block to work with the
stream returned by svnadmin dump, I would get errors regarding a
malformed header in the dump file. For instance:

# The following block results in bad dump files, not accepted by
svnadmin load
File.open("path/to/todays/dumpfile", "a") do |file|
file << `svnadmin dump "path/to/repository" -r #{revision}
--incremental`
end

# However, a strait command expression does work, like
`svnadmin dump "path/to/repository" -r #{revision} --incremental >>
"path/to/todays/dumpfile"`

Might anyone know why the use of a File.open block would result in a
dump file not accepted by svnadmin load, while a strait command
expression works? I tried using the << and write methods of File, but
still no luck. Perhaps I don't understand a more appropriate way to
write the stream?

Thanks for taking a look.
Brad

I'm only guessing but the different file size suggests that our old
friend LF vs. CR/LF rears its ugly head again.

Pia Kraft

 

[Brad C.]

Can you do it both ways and then diff the two files to see what's
actually different?

Also, try using write (or, failing that, puts) instead of << to output
to the file.

--Steve

Being on a windows XP machine, I ran COMP to compare the files and it
only stated that the files were of different sizes. I also compared the
files with TextPad, which output:

Compare: (<)C:\subversion\dump\test\test_cmdexpress (431 bytes)
with: (>)C:\subversion\dump\test\test_rblocksave (462 bytes)
The files are identical

I also ran subversion's own diff against the files which output:

Index: test_cmdexpress
===================================================================
--- test_cmdexpress (revision 0)
+++ test_cmdexpress (revision 0)
@@ -0,0 +1,31 @@
+SVN-fs-dump-format-version: 2
+
+UUID: 6d10496b-807e-eb4a-8eb1-db2388672f7a
+
+Revision-number: 32
+Prop-content-length: 120
+Content-length: 120
+
+K 7
+svn:log
+V 18
+changed banana.txt
+K 10
+svn:author
+V 7
+bc13935
+K 8
+svn:date
+V 27
+2006-04-04T15:34:30.679827Z
+PROPS-END
+
+Node-path: banana.txt
+Node-kind: file
+Node-action: change
+Text-content-length: 14
+Text-content-md5: 485def7fa8bf59e9d2ea55653a999a22
+Content-length: 14
+
+I am a banana.
+
Index: test_rblocksave
===================================================================
--- test_rblocksave (revision 0)
+++ test_rblocksave (revision 0)
@@ -0,0 +1,31 @@
+SVN-fs-dump-format-version: 2
+
+UUID: 6d10496b-807e-eb4a-8eb1-db2388672f7a
+
+Revision-number: 32
+Prop-content-length: 120
+Content-length: 120
+
+K 7
+svn:log
+V 18
+changed banana.txt
+K 10
+svn:author
+V 7
+bc13935
+K 8
+svn:date
+V 27
+2006-04-04T15:34:30.679827Z
+PROPS-END
+
+Node-path: banana.txt
+Node-kind: file
+Node-action: change
+Text-content-length: 14
+Text-content-md5: 485def7fa8bf59e9d2ea55653a999a22
+Content-length: 14
+
+I am a banana.
+

...Which I don't see any differences in.

I tried generating the file with File#write, File#<<, and File#puts,
with both "a" and "w" modifiers, but all come to be the larger file size
of 462 bytes, and are not accepted by svnadmin load, which errors:

svnadmin: Found malformed header block in dumpfile stream

I'm using ruby 1.8.4 (2005-12-24) [i386-mswin32], Windows XP.

 

[Brad C.]

.. btw

Testing without a svn dump results in different file sizes just the
same. However in this case the ruby generated file is smaller:

echo Test > testcmd.txt
(7 byte file)

File.open("c:/testrb.txt","w") do |f|
f << "Test"
end
(4 byte file)

If it matters much. Just trying to understand why these differences
happen

 

[gwtmp01]

I'm only guessing but the different file size suggests that our old
friend LF vs. CR/LF rears its ugly head again.

I thought the OP was on a Linux box where there is no difference
between a binary
and text file but if the problem is on a Windows system then you have
to explicitly
open the file in binary mode to avoid the LF CR/LF issues of text I/O.

Gary Wright

 

[Brad C.]

but if the problem is on a Windows system then you have
to explicitly
open the file in binary mode to avoid the LF CR/LF issues of text I/O.

Gary Wright

Well, I tried this to no avail (I assume this is using a binary mode
file):

File.open("c:/testrb.txt","w") do |f|
f.binmode # I'm not really sure if this puts f in binary mode or not.
f << "Test"
end

Still results in different file sizes.

 

[Brad C.]

I should have posted an example. Your code is opening the file in text
mode. You have to add a 'b' to the mode argument of open to force the
file to be opened in binary mode:


File.open("c:/testrb.txt","wb") do |f|
# write data here via the File object f
end

Gary Wright

Ah, I should have been able to come up with that. Thanks. I tried this
and the file sizes remain different though. Might this have to do with
the version of ruby I'm using? Apart from running on windows :\

I guess one could create files via strait command expressions, then
modify them via ruby's File class, unless that resaves differently. Is
this something that's normal -different file sizes when creating files
in various ways?

 

[Logan Capaldo]

Ah, I should have been able to come up with that. Thanks. I tried this
and the file sizes remain different though. Might this have to do with
the version of ruby I'm using? Apart from running on windows :\

I guess one could create files via strait command expressions, then
modify them via ruby's File class, unless that resaves differently. Is
this something that's normal -different file sizes when creating files
in various ways?

------------------------------------------------------------------ IO#<<
ios << obj => ios
------------------------------------------------------------------------
String Output---Writes obj to ios. obj will be converted to a
string using to_s.

$stdout << "Hello " << "world!\n"

produces:

Hello world!

As you can see, << is more like write or print than puts, while echo
will append a new line

try file.puts "Test" and see if your file sizes match up then