Encrypt large files with ruby openssl or ????

[Damjan Rems]

It looks like ruby openssl is not intended to encrypt large amount of
data (like XML documents) because of:

`private_encrypt': data too large for key size (OpenSSL:Key::RSAError)

Since there obvious are tools which do the job my question is. Can and
how it be done with ruby?


Thank you

TheR

 

[Christopher Brown]

You can certainly do something like the following for any size
document, streaming it in as you go:

#!/usr/bin/env ruby

require 'openssl'

key = "1234567890123456"
alg = "AES-128-CBC"
iv = "6543210987654321"

aes = OpenSSL::Cipher::Cipher.new(alg)
aes.encrypt
aes.key = key
aes.iv = iv

File.open("foo.enc",'w') do |enc|

File.open("foo") do |f|
loop do
r = f.read(4096)
break unless r
cipher = aes.update(r)
enc << cipher
end
end

enc << aes.final
end

Cheers,
Chris

 

[Jan Svitok]

It looks like ruby openssl is not intended to encrypt large amount of
data (like XML documents) because of:

`private_encrypt': data too large for key size (OpenSSL:Key::RSAError)

Since there obvious are tools which do the job my question is. Can and
how it be done with ruby?

Well, large data is encrypted another way. Usual keysizes are
1024-4096 bits, i.e. 128-512 bytes. With plain RSA, you can
encrypt/sign/whatever just as many bytes, as the key size. So, if you
have 1024 bit key, and you want to encrypt data larger that 128, you
have to do something more.

The simplest thing to do is to cut the data into 128 blocks and
encrypt each separately. This has several drawbacks: 1. It's reaaally
slow. You have to do 1024 bit exponentiation for each block (i.e. 1024
bits ^ 1024bits). 2. You still have to detect reordered, missing or
inserted blocks.

So the solution is: encrypt data with symmetric cipher (AES, IDEA,
DES, 3DES, Blowfish, etc.) and then encrypt the symmetric key with
RSA. Send over the wire both RSA-encrypted AES key, and AES-encrypted
data. This way it will be faster - AES is by an order of magnitude
faster than RSA.

Now the problem is somewhat escalated: AES still has 128-256 bit keys,
so the blocks are even smaller. This time we cut the data into this
smaller blocks, and chain them together. By chaining I mean that a
part of output of a block is fed into encryption of another. This way
the blocks depend on previous one(s). There are 4 usual types of
chaining: ECB, CFB, CBC, OFB and counter mode. Each one has different
properties, and you have to choose the right one.

Chances are, that you don't need to implement all off this, just call
proper library method. All you have to do is choose the parameters -
key sizes, cipher, chaining mode. I don't know the ruby OpenSSL
binding so I won't help you here. A quick look at ruby doc hasn't
revealed anything.

You can read more about this in an excellent book by A. Menezes et
al.: Handbook of Applied Cryptography - it's available on the web for
free [1]

[1] http://www.cacr.math.uwaterloo.ca/hac/

 

[Damjan Rems]

I am sorry. It looks I have been asking wrong questions, because I
didn't know how or where to start. And this is basicly all I have got in
my directions.

I need to encript and sign XML document as defined by XML security
standards.

XML Signature Syntax, XML-DSIG, http://www.w3.org/2000/09/xmldsig#
Digest method, SHA-1, http://www.w3.org/2000/09/xmldsig#sha1
Signature method, RSA-SHA-1, http://www.w3.org/2000/09/xmldsig#rsa-sha1

I thought this would be a lot easier.

by TheR

 

[Roland Schmitt]

Hi,

I am sorry. It looks I have been asking wrong questions, because I
didn't know how or where to start. And this is basicly all I have got in
my directions.

I need to encript and sign XML document as defined by XML security
standards.

XML Signature Syntax, XML-DSIG, http://www.w3.org/2000/09/xmldsig#
Digest method, SHA-1, http://www.w3.org/2000/09/xmldsig#sha1
Signature method, RSA-SHA-1, http://www.w3.org/2000/09/xmldsig#rsa-sha1

I thought this would be a lot easier.

by TheR

you can do all the basic cryptographic operations (sign, encrypt, hash)
with ruby-openssl.
But that is not enough, you also need a xml canonicalizer, defined by
http://www.w3.org/TR/xml-c14n. There is nothing like this in pure ruby.
One idea is using the libxml-ruby-bindings for this
(http://rubyforge.org/projects/libxml/), as the libxml
(http://xmlsoft.org/) supports canonicalisation.
Another idea is implementing SWIG-bindings to something like xmlsec
(http://www.aleksey.com/xmlsec/), which implements the XML security
standards you refering to.

Regards,
Roland