Slow RAILS testing

[kevin cline]

My RAILS unit tests take approximately one minute to run. This may
not sound like much, but it's a real drag on incremental development.
I imagine most of the slowness in spent in applying the database
schema to the test database. Is there a memory-resident database that
would speed up my tests?

 

[George Malamidis]

Hello,

Do all your tests need to access the database? I find this approach
( http://nutrun.com/weblog/rails-fast-test-suite/ ) useful in terms
of instant feedback and application code responsibility boundaries.

Thanks,
George

 

[Alex Young]

My RAILS unit tests take approximately one minute to run. This may
not sound like much, but it's a real drag on incremental development.
I imagine most of the slowness in spent in applying the database
schema to the test database. Is there a memory-resident database that
would speed up my tests?

I think you can run SQLite in-memory. The Rails people would know more,
though:

http://groups.google.com/group/rubyonrails-talk

I tend to factor tests so that they don't use Rails' built-in fixtures,
though - you really don't need to test the database interface, so it's
pure overhead.

 

[Travis D Warlick Jr]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

My RAILS unit tests take approximately one minute to run. This may
not sound like much, but it's a real drag on incremental development.
I imagine most of the slowness in spent in applying the database
schema to the test database. Is there a memory-resident database that
would speed up my tests?

You can use Heap tables with MySQL if you wish. This SQL statement will convert
a table into a Heap (memory-resident) table:

ALTER TABLE tablename TYPE=HEAP;
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFG6cnYWvapaOIz2YYRAuuvAJ4wwlypxThIaHCkVy7t4V1iDdgEHQCfZ5w+
wYBYFf9cGzqs56/O04qhG9I=
=JLqS
-----END PGP SIGNATURE-----

 

[Gaspard Bucher]

I could more then double the speed of my tests by avoiding fixtures
reload on each test and using transactions. Have a look at
http://dev.zenadmin.org/browser/trunk/test/test_helper.rb and
http://dev.zenadmin.org/browser/trunk/test/zena_test.rb.

It sure helped me (and I will now try Travis' HEAP idea on top of that).

Gaspard

 

[Gaspard Bucher]

HEAP does not support TEXT columns... Too bad.

Gaspard

 

[Phlip]

ALTER TABLE tablename TYPE=HEAP;

Ain't that ENGINE=HEAP?

HEAP does not support TEXT columns... Too bad.

Suppose one wrote a rake task db:heap_mode, to switch the test environment
DB to the faster system. Could that task programmatically access schema.rb,
find the TEXT columns, and downgrade them to long strings?

 

[Travis D Warlick Jr]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ain't that ENGINE=HEAP?

Both is valid.

Suppose one wrote a rake task db:heap_mode, to switch the test
environment DB to the faster system. Could that task programmatically
access schema.rb, find the TEXT columns, and downgrade them to long
strings?

One possible rails issue: I believe the entire test database structure is erased
and recreated everytime, so it couldn't be a rake task, it would have to be
integrated into the test task.

A database issue: you would have to beware of your table size. Every HEAP table
with at least 1 VARCHAR is at least 50KB due to how the VARCHARs are stored in
memory, and the larger you make your VARCHARs the larger your tables get (eg. a
table with 1 field of VARCHAR(20000) gives a table size of ~170KB).

If anyone wants to do this, note that the largest record size using the HEAP
engine is 64KB, and the largest single VARCHAR is ~21000.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFG69BFWvapaOIz2YYRAv5cAJ0Z29aPLUVD8q/coTpULjzkzMahuACfUxqy
Sbr0ZNzCUHCFbrGI4FBQJMk=
=6GQA
-----END PGP SIGNATURE-----

 

[Phlip]

One possible rails issue: I believe the entire test database structure is
erased
and recreated everytime, so it couldn't be a rake task, it would have to
be
integrated into the test task.

Yup; I just got that.

Then I put the conversion into the top of the source file of the first test
suite in alphabetic order (please nobody comment on the sustainability!),
and got a huge spew of test faults, as if I had no fixtures. The converter
was just this:

require File.dirname(__FILE__) + '/../test_helper'

def setup_database
tables = %w( accessories chats fighter_images props rings rounds users )
tables.each do |table|
ActiveRecord::Base.connection.execute("
ALTER TABLE #{ table } ENGINE=HEAP
")
end
end
setup_database

class AccessoryTest < Test::Unit::TestCase
...

And note, per the memory consumption issues you mentioned, that you don't
need to put all the fixtures in the list - if it worked!

 

[Gaspard Bucher]

I tried this, but HEAP does not support transactions either. From what
I see, it feels like it's going to be much slower to reload the
fixtures then to do a ROLLBACK...