reccommended work flow for unit tests and databases in ruby

[Adam Akhtar]

How does one go around creating unit tests for database tables during
developement. Is there a rock solid work flow plan that everyone uses.
Im new to dbs and relatively so with unit tests. I have my project and
it usually starts with a rough idea of what fields need to be in the
table. As i code away items in the db change so it makes it difficult to
write unit tests that will withstand these changes.

I know i need a test version of a database with static items but what id
like to do is for this test version to always reflect whats happening in
the development one. So if I add a field in the development one, the
test one gets one too.

I could just go and code this and probably get something that works but
is probably ugly. I dont like to reinvent the wheel, a bad one at that
so if anyone has some tips or pointers in the right direction id really
appreciate it.

This is for small projects so it doesnt have to industrial strength
bullet proof ninja stuff, small but elegant will be good enough

 

[Phlip]

How does one go around creating unit tests for database tables during
developement. Is there a rock solid work flow plan that everyone uses.

A myth of software development is you can plan out and invent your database
schema before writing your code. This myth is so pervasive that some communities
(>cough< Oracle) actively prevent programmers from changing their schemas.

Refactoring databases is a Best Practice, and all projects should include
support scripts that overcome our database platform limitations.

Im new to dbs and relatively so with unit tests. I have my project and
it usually starts with a rough idea of what fields need to be in the
table. As i code away items in the db change so it makes it difficult to
write unit tests that will withstand these changes.

Investigate (and use!) the ActiveRecord system exemplified in Rails fixtures and
migrations. When your program starts, only only _only_ put in the database the
very few fields you are actually using. Rails's "script/generate model" will
write a migration which adds that table to the db. Write tests for the model's
behavior, and these will require fields in the table. Edit the model's migration
file to add them. Repeat in tiny cycles, testing and integrating as often as
possible. Never integrate if the tests break - revert if necessary. (This is
more efficient than debugging!)

I know i need a test version of a database with static items but what id
like to do is for this test version to always reflect whats happening in
the development one. So if I add a field in the development one, the
test one gets one too.

Put sample data into the fixture files, and use it in the tests. Your scripts
should then be able to migrate your test, development, and production databases
in lockstep.

If you are not writing a Rails project, research how to get all these benefits
without its framework. Rails can import fixture data into the development
database with 'rake db:migrate'.

When you deploy to a live site, at this point stop editing your initial
migration files, and only add new files that add and adjust your fields.

I could just go and code this and probably get something that works but
is probably ugly. I dont like to reinvent the wheel, a bad one at that
so if anyone has some tips or pointers in the right direction id really
appreciate it.

This is for small projects so it doesnt have to industrial strength
bullet proof ninja stuff, small but elegant will be good enough

These techniques - after you spend a little time implementing them - keep
potentially big projects very small and manageable. I would use them on projects
of any size!

 

[David A. Black]

Hi --

How does one go around creating unit tests for database tables during
developement. Is there a rock solid work flow plan that everyone uses.
Im new to dbs and relatively so with unit tests. I have my project and
it usually starts with a rough idea of what fields need to be in the
table. As i code away items in the db change so it makes it difficult to
write unit tests that will withstand these changes.

I know i need a test version of a database with static items but what id
like to do is for this test version to always reflect whats happening in
the development one. So if I add a field in the development one, the
test one gets one too.

I could just go and code this and probably get something that works but
is probably ugly. I dont like to reinvent the wheel, a bad one at that
so if anyone has some tips or pointers in the right direction id really
appreciate it.

This is for small projects so it doesnt have to industrial strength
bullet proof ninja stuff, small but elegant will be good enough

Rails has a bunch of Rake tasks that copy the schema from one database
to another. I don't know whether your project is a Rails application,
but even if it isn't, you might be able to integrate one of those
fairly easily, so that when you run your tests, the test database will
pick up any changes in the development database.


David

--
David A. Black / Ruby Power and Light, LLC
Ruby/Rails consulting & training: http://www.rubypal.com
Coming in 2009: The Well-Grounded Rubyist (http://manning.com/black2)

http://www.wishsight.com => Independent, social wishlist management!

 

[Phlip]

Rails has a bunch of Rake tasks that copy the schema from one database
to another. I don't know whether your project is a Rails application,
but even if it isn't, you might be able to integrate one of those
fairly easily, so that when you run your tests, the test database will
pick up any changes in the development database.

However...

Every once in a while I feel the urge to think outside those ActiveRecord
scripts' box. For example, I want to write a method nab_fixtures('models.yml'),
and it will reach out to a yml file and load it, on command, without clawing its
way thru the uber-kewt fixtures() directive at the top of my suite.

When I feel that urge, I often have to repress it, because when I try to re-use
the code inside fixtures(), it's all "refactored" into a maze of fragile and
un-reusable methods. When I rewrite the system that loads a yml file into a
table myself, I replace fixtures() bugs with my own!

How the heck does anyone use ActiveRecord, with fixtures and migrations, outside
Rails? How, for example, can that magic luvvable plugable Merb do it??

 

[David A. Black]

Hi --

However...

Every once in a while I feel the urge to think outside those ActiveRecord
scripts' box.

Rake will let you do that

For example, I want to write a method
nab_fixtures('models.yml'), and it will reach out to a yml file and load it,
on command, without clawing its way thru the uber-kewt fixtures() directive
at the top of my suite.

When I feel that urge, I often have to repress it, because when I try to
re-use the code inside fixtures(), it's all "refactored" into a maze of
fragile and un-reusable methods. When I rewrite the system that loads a yml
file into a table myself, I replace fixtures() bugs with my own!

I'd have to see the code to know exactly what's going on. Is it a
problem reconciling foreign key references?


David

--
David A. Black / Ruby Power and Light, LLC
Ruby/Rails consulting & training: http://www.rubypal.com
Coming in 2009: The Well-Grounded Rubyist (http://manning.com/black2)

http://www.wishsight.com => Independent, social wishlist management!

 

[Phlip]

I'd have to see the code to know exactly what's going on. Is it a
problem reconciling foreign key references?

It's a general, structural question, so I will ask it a different way.

This thread seems to propose one can borrow ActiveRecord, plus fixtures and
migrations, and use them in a Ruby project that is (shocked gasp) not Rails.

If so, then doing this, in Rails, should be easy, too!

class MyUnitTestSuite < ActiveSupport::TestCase
fixtures :foos

def setup
fixtures :bars
end
...
end

For whatever reason, we want to load some fixtures the old-fashioned way, and
some fixtures optionally. Maybe in the top of a test case, or maybe
conditionally in a setup. Or whatever.

If we promise the OP they can borrow ActiveRecord facilities, then borrowing
them should indeed be easy. But whenever I try to call or re-use or re-wrap the
methods inside fixtures, they screw up in horrible ways. This _might_ be a sign
of coupling!

 

[Matt Harrison]

Investigate (and use!) the ActiveRecord system exemplified in Rails
fixtures and migrations. When your program starts, only only _only_ put
in the database the very few fields you are actually using. Rails's
"script/generate model" will write a migration which adds that table to
the db. Write tests for the model's behavior, and these will require
fields in the table. Edit the model's migration file to add them. Repeat

This sounds great, but I've read in several places (sorry can't find
references) that you should create a new migration to make changes to an
existing table.

I've tried before to modify an existing migration but unless I totally
wipe the DB, it doesn't take notice, as the schema version isn't
incremented.

Is this expected behavior or am I doing something wrong?

 

[Phlip]

This sounds great, but I've read in several places (sorry can't find
references) that you should create a new migration to make changes to an
existing table.

If you have not yet deployed a migration to your production site, you can go
back into it and tweak it. This is still within the inner TDD cycle, because
your tests will cover the new versions of the tables.

I've tried before to modify an existing migration but unless I totally
wipe the DB, it doesn't take notice, as the schema version isn't
incremented.

Right - rake db:reset, and rake db:migrate:reset.

Tweaking existing migrations before deploying them (such as adding each field in
a new table, as you TDD it) is a convenience that prevents your migrate folder
from filling up with many trivial variations.

 

[Matt Harrison]

If you have not yet deployed a migration to your production site, you
can go back into it and tweak it. This is still within the inner TDD
cycle, because your tests will cover the new versions of the tables.


Right - rake db:reset, and rake db:migrate:reset.

Tweaking existing migrations before deploying them (such as adding each
field in a new table, as you TDD it) is a convenience that prevents your
migrate folder from filling up with many trivial variations.

Well I'm just getting into TDD and thats going well, but I did notice
that when running rake:test, that it complains there are migrations that
need to be run before the tests will be performed, even though the tests
are running on a different database.

Matt

 

[Phlip]

Well I'm just getting into TDD and thats going well, but I did notice

that when running rake:test, that it complains there are migrations that
need to be run before the tests will be performed, even though the tests
are running on a different database.

That's because the migrations (should!) happen on the Development database, and
this gets cloned into the Test database just before each test run...

 

[Matt Harrison]

That's because the migrations (should!) happen on the Development
database, and this gets cloned into the Test database just before each
test run...

So on the development database, it's safe to add the columns or whatever
manually, update the migration to reflect the changes and test/code as
usual. Then when the production deployment happens it uses the
migrations from scratch?

Thanks

Matt

 

[Phlip]

So on the development database, it's safe to add the columns or whatever
manually, update the migration to reflect the changes and test/code as
usual. Then when the production deployment happens it uses the
migrations from scratch?

Not safe to add columns "manually". Add them to your migrations, rake
db:migrate:reset to rebuild your databases, manually test, deploy, and lock down
those migrations.

The deploy script will migrate on the server, so all your existing data don't
get erased!

 

[Matt Harrison]

Not safe to add columns "manually". Add them to your migrations, rake
db:migrate:reset to rebuild your databases, manually test, deploy, and
lock down those migrations.

The deploy script will migrate on the server, so all your existing data
don't get erased!

Ok, I'll have a go at that, thanks for clearing that up.

Matt

 

[Adam Akhtar]

hi guys, sorry this is actually a railess app. Just ruby and the
console. Ive found a few links though that show how to use active record
without rails though here including setting up migrations and fixutres.

http://rails.aizatto.com/category/activerecord/activerecord-without-rails/


and there is also this piece here

http://wiki.rubyonrails.org/rails/pages/HowToUseActiveRecordOutsideRails

does anyone know of anyother links?

From what i can tell those links dont discuss how to implement the
automatic generation of a test version of a development db that rails
does everytime you run a test. If anyone has a link etc id really
appreciate it!.

Thanks for the input so far

 

[David A. Black]

Hi --

hi guys, sorry this is actually a railess app. Just ruby and the
console. Ive found a few links though that show how to use active record
without rails though here including setting up migrations and fixutres.

http://rails.aizatto.com/category/activerecord/activerecord-without-rails/


and there is also this piece here

http://wiki.rubyonrails.org/rails/pages/HowToUseActiveRecordOutsideRails

does anyone know of anyother links?

automatic generation of a test version of a development db that rails
does everytime you run a test. If anyone has a link etc id really
appreciate it!.

I don't have a link to a complete setup but what I'd recommend is have
a look at the Rake tasks that come with Rails, and adapt them. It's
non-trivial enough that I can't offer to do it on the spot, but
reasonably straightforward and will give you some Rake practice
anyway.


David

--
David A. Black / Ruby Power and Light, LLC
Ruby/Rails consulting & training: http://www.rubypal.com
Coming in 2009: The Well-Grounded Rubyist (http://manning.com/black2)

http://www.wishsight.com => Independent, social wishlist management!