A way to find out when a constant gets defined?

[Josh Cheek]

[Note: parts of this message were removed to make it a legal post.]

Hi, I'd like to be able to find out when a constant gets defined. I think I
have a gem that vendored Psych, so it doesn't get listed in my dependencies,
but causes the most unusual and difficult to track bugs (because other
things, like rake tasks that don't load the entire environment, don't get
Psych loaded, and thus behave differently when doing things like serializing
data).


Here is my reasoning:

# I do not even have psych installed (it isn't listed)
$ gem list psych

*** LOCAL GEMS ***


# Psych is not a dependency (no output)
$ cat Gemfile.lock | grep -i psych


# When I load irb with my app, Psych is somehow defined
$ bundle exec irb
ruby-1.9.2-p180 :001 > Psych
=> Psych
ruby-1.9.2-p180 :002 > exit


# When I load irb without my app, Psych is not defined
$ irb
ruby-1.9.2-p180 :001 > Psych
NameError: uninitialized constant Object:sych
from (irb):1
from /Users/josh/.rvm/rubies/ruby-1.9.2-p180/bin/irb:16:in `<main>'
ruby-1.9.2-p180 :002 > exit


# For each file in my dir, exempting files in ./git, list the lines matching
/psych/i
# prints nothing, so Psych is not coming from my code.
$ find . -name \* | ruby -ne 'puts $_ unless /^\.\/\.git/' | xargs grep -i
psych



I don't really know what to do, I tried `$ bundle package` and then
untarring and gunzipping all the gems, and then grepping them for it, but
didn't find anything. I can't figure out where this is coming from, so I'd
like to be able to monitor the constant Psych to find out when it gets set.

 

[Josh Cheek]

[Note: parts of this message were removed to make it a legal post.]

Use set_trace_func lambda {|*a| p a} before the first require and try
to see whether you can find out from there psych code is loaded.

This worked. I figured out that it must be coming from Bundler itself.
Running set_trace_func against `require "bundler"` finds this:


require 'pp'

set_trace_func lambda { |event, filename, target_line, id, bnd, classname|
next unless event == 'class'
next unless File.exist? filename
crnt_line = (1..1/0.0).each
File.foreach filename do |line|
next unless crnt_line.next == target_line && line =~ /psych/i
puts "IN FILE #{filename.inspect}"
puts "ON LINE #{target_line}"
puts "THE LINE: #{line.inspect}"
begin
raise 'gettin the backtrace'
rescue
pp $!.backtrace
end
puts '-' * 100
end
}

require 'bundler'
# >> IN FILE
"/Users/josh/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/yaml.rb"
# >> ON LINE 39
# >> THE LINE: "module Psych\n"
# >> ["-:13:in `block (2 levels) in <main>'",
# >> "-:7:in `foreach'",
# >> "-:7:in `block in <main>'",
# >> "/Users/josh/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/yaml.rb:39:in
`<modulesych>'",
# >> "/Users/josh/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/yaml.rb:39:in
`<top (required)>'",
# >>
"/Users/josh/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in
`require'",
# >>
"/Users/josh/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in
`require'",
# >>
"/Users/josh/.rvm/gems/ruby-1.9.2-p180/gems/bundler-1.1.pre.5/lib/bundler.rb:10:in
`<top (required)>'",
# >> "<internal:lib/rubygems/custom_require>:33:in `require'",
# >> "<internal:lib/rubygems/custom_require>:33:in `rescue in require'",
# >> "<internal:lib/rubygems/custom_require>:29:in `require'",
# >> "-:21:in `<main>'"]
# >>
----------------------------------------------------------------------------------------------------



Looking at its source, it comes from requiring yaml, a quick test:
$ ruby -e 'require "yaml"; p Psych'
Psych



So I guess thats where it comes from. My errors are intermittent (and omg is
that frustrating), but if I can nail down a reproducible example of the
problem, I'll post it.

 

[Josh Cheek]

[Note: parts of this message were removed to make it a legal post.]

Okay, I think I found the ultimate source of the problem was that RVM said
it should hijack rake and run bin/rake (
https://rvm.beginrescueend.com/integration/bundler/), but the Bundler team
didn't like that, so the RVM team took it back out (I've asked that they
update the website to reflect this). I updated my RVM at some point and
didn't realize I'd lost this functionality.

This allowed one rake task to populate the DB with data serialized form
Syck, but when my app went to use it, it tried to deserialize with Psych,
and blew up.

The fix I found was to use https://github.com/mpapis/rubygems-bundler to
regain the functionality that was removed from RVM where it runs `bundle
exec rake ...` when I type "rake ..." while in a dir containing a Gemfile.

 

[Robert Klemme]

Okay, I think I found the ultimate source of the problem was that RVM said
it should hijack rake and run bin/rake (
https://rvm.beginrescueend.com/integration/bundler/), but the Bundler team
didn't like that, so the RVM team took it back out (I've asked that they
update the website to reflect this). I updated my RVM at some point and
didn't realize I'd lost this functionality.

This allowed one rake task to populate the DB with data serialized form
Syck, but when my app went to use it, it tried to deserialize with Psych,
and blew up.

The fix I found was to use https://github.com/mpapis/rubygems-bundler to
regain the functionality that was removed from RVM where it runs `bundle
exec rake ...` when I type "rake ..." while in a dir containing a Gemfile.

Thanks for the summary, Josh! Frankly, I find the Ruby ecosystem is
becoming increasingly complex. I think we should carefully watch out
to stop this tendency because these are the exact kind of issues which
will make people turn away when they try out Ruby and run into them.

Kind regards

robert

 

[Steve Klabnik]

[Note: parts of this message were removed to make it a legal post.]

but the Bundler team
didn't like that, so the RVM team took it back out

Whoah, is this discussion somewhere? I literally just moved all my stuff to
use binstubs. Or, do you mean that the rvm functionality to add ./bin to the
$PATH?

 

[Steve Klabnik]

[Note: parts of this message were removed to make it a legal post.]

Frankly, I find the Ruby ecosystem is
becoming increasingly complex.

All software is becoming increasingly complex. It has been, ever since we
started using assembly rather than machine code. Such is the nature of the
beast.

That said, it's our job to manage this complexity. This is a fundamentally
hard problem, and none of it is _necceasary_ to do Ruby development. If the
tools are too much work, just don't use those tools. Isolate exists for a
reason...

 

[Robert Klemme]

All software is becoming increasingly complex. It has been, ever since we
started using assembly rather than machine code. Such is the nature of th= e
beast.

Of course. But there's a difference between complexity required to
solve complex problems and complexity introduced without need.
Unfortunately the latter seems to be quite ubiquitous these days...

That said, it's our job to manage this complexity. This is a fundamentall= y
hard problem, and none of it is _necceasary_ =A0to do Ruby development. I= f the
tools are too much work, just don't use those tools. Isolate exists for a
reason...

You mean http://rubygems.org/gems/isolate I guess. Isn't it ironic
that we introduced gem to have a single repository with automated
updating etc. and now add another tool which reverses the effect?
Greetings from DLL hell...

Kind regards

robert

--=20
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

 

[Steve Klabnik]

[Note: parts of this message were removed to make it a legal post.]

Of course. But there's a difference between complexity required to
solve complex problems and complexity introduced without need.
Unfortunately the latter seems to be quite ubiquitous these days...

Maybe for you. rvm/bundler pretty much saved my sanity. There is absolutely
a need for them in the Ruby ecosystem. In fact, I've even heard people say
"I don't want to use a programming language without something like rvm ever
again."

Isn't it ironic

that we introduced gem to have a single repository with automated

updating etc. and now add another tool which reverses the effect?

Not really. rubygems is about managing packages that are installed on your
system and their versions. isolate/bundler are about managing which packages
and versions are used with your application. The tools are complimentary.

Greetings from DLL hell...

That's specifically what isolate/bundler help with.

 

[Josh Cheek]

[Note: parts of this message were removed to make it a legal post.]

Whoah, is this discussion somewhere? I literally just moved all my stuff to
use binstubs. Or, do you mean that the rvm functionality to add ./bin to
the
$PATH?

The latter. IDK if it is documented, it was explained to me by mpapis, the
author of rubygems-bundler, in irc://irc.freenode.net/rvm Exact quote was
"joshcheek, we are sorry but because of some bundler team complains we had
to remove the integration"

 

[Steve Klabnik]

[Note: parts of this message were removed to make it a legal post.]

Thanks. Ill jump in there and ask sometime.

 

[Josh Cheek]

[Note: parts of this message were removed to make it a legal post.]

Okay, I think I found the ultimate source of the problem was that RVM said
it should hijack rake and run bin/rake (
https://rvm.beginrescueend.com/integration/bundler/), but the Bundler team
didn't like that, so the RVM team took it back out (I've asked that they
update the website to reflect this). I updated my RVM at some point and
didn't realize I'd lost this functionality.

This allowed one rake task to populate the DB with data serialized form
Syck, but when my app went to use it, it tried to deserialize with Psych,
and blew up.

The fix I found was to use https://github.com/mpapis/rubygems-bundler to
regain the functionality that was removed from RVM where it runs `bundle
exec rake ...` when I type "rake ..." while in a dir containing a Gemfile.

To prevent this from happening to me again, I'm working on a gem
bundler-bouncer (https://github.com/JoshCheek/bundler-bouncer). At the top
of any entrance into your app (ie Rakefiel) you `require "bundler/bouncer"`
and it will exit the app if Bundler isn't running.

If anyone here uses Bundle, or frequently authors gems, I wouldn't mind
feedback: Have I followed gem conventions appropriately? Am I testing
properly? Anything you see from your experience that I don't have yet? If
so, point it out; help me learn.