Monkeypatching is Destroying Ruby

[Gary Wright]

I think this is the worst kind of monkey-patching - breaking existing
functionality. I didn't realize facets went to this level.
Another reason
to never touch it.

What existing functionality? Standard Ruby doesn't define Hash#-.

One of the strange aspects of this discussion is that monkey patching
core classes is seen as dangerous *except* when Matz and Co. decide
to add something to the core classes. Then it is seen as an example
of elegant library design. Fortunately, Matz is pretty good at
making those decisions.

I don't think the problem is with the ability to modify core classes
per se but with the difficulty of managing library evolution in a
distributed context.

Gary Wright

 

[Florian Gilcher]

What existing functionality? Standard Ruby doesn't define Hash#-.

One of the strange aspects of this discussion is that monkey
patching core classes is seen as dangerous *except* when Matz and
Co. decide to add something to the core classes. Then it is seen as
an example of elegant library design. Fortunately, Matz is pretty
good at making those decisions.

I don't think the problem is with the ability to modify core classes
per se but with the difficulty of managing library evolution in a
distributed context.

Gary Wright

Extending the Core-Classes is not dangerous//bad. It just requires
care. If Matz and Co. decide to add functionality to core classes,
they are not monkey-patching. Usually, this change is well documented
in a changelog that should be read if you update the language to a new
major version.

I see the problem that everybody does what he wants (because he is
free) but there is no common sense on when and how to do it (well,
thats why we are discussing ). As i said: some kind of opt-in
functionality would be nice.

Greetings
Florian Gilcher

 

[Gary Wright]

Extending the Core-Classes is not dangerous//bad. It just requires
care. If Matz and Co. decide to add functionality to core classes,
they are not monkey-patching. Usually, this change is well
documented in a changelog that should be read if you update the
language to a new major version.

Let me avoid the phrase 'monkey-patching' for the moment.

If you imagined a version of Ruby that did not permit
local additions to core classes then the only source
of change to those classes would be the evolution of
the language itself. But since that isn't the case,
core classes can acquire new methods in several ways:
-- new release of Ruby
-- local additions in a code base I manage
-- 'foreign' additions in a gem or other dependency

It pretty much doesn't matter where the conflicting
change originated, it is still a problem, if you
want to integrate the code bases.

I can read the release notes/code of the Ruby
distribution or I can read the documentation/code
of a gem that I'm dependent on but the end result
is the same: a conflict that has to be resolved.

If you want to say that changes that originate from
Matz are 'blessed' and not 'monkey-patching' fine but
that doesn't change the fact that I've still got
a conflict that has to be resolved.

Of course I could point out that this isn't just a
problem with conflicting method names in core classes.
It is possible to have conflicts in the class/module
namespace also.

So I believe the problem is a much more general one of
how to manage the composition (i.e. integration) of
independently evolving distributed code bases. The
'monkey-patching' is bad school of thought is basically
an argument to localize all evolution of the libraries
and language in some sort of central standards committee.
No doubt that would work, but it has its own set of
problems.

Gary Wright

 

[Trans]

Facets defines Hash#- based on [key,value] pairs and not keys. An
argument can be made for either approach but you can't integrate code
bases that have different expectations for Hash#-.

BTW, there is a reason for that. You can do:

ahash - otherhash.keys

I think this is the worst kind of monkey-patching - breaking existing
functionality. I didn't realize facets went to this level. Another reason
to never touch it.

Maybe you should learn your Ruby before you bad mouth someone else's
work. Because you don't what you are talking about. There is no
"existing functionality". Thee is no Hash#- defined in Ruby! It's
strictly an added method. I said it before and I'll repeat it again.
FACETS DOES NOT OVERRIDE ANY BUILT-IN METHODS.

Moreover, about a dozen or so methods that were defined in Facets have
now become standard parts of Ruby as of 1.9. Facets isn't some crazy
poison juice box. It's a way to provide some tertiary standardization
around common Ruby idioms.

T.

 

[Eric Mahurin]

[Note: parts of this message were removed to make it a legal post.]

FACETS DOES NOT OVERRIDE ANY BUILT-IN METHODS.

You don't have to yell, but I'm glad to hear this. BTW, "redefine" is
probably the better term to use. Derived classes "override" methods in
their base class.

In the context of the previous methods, I falsely jumped to the conclusion
that Hash#- was part of the std ruby distribution since a conflict in what
the functionality should be was being discussed. I apologize. I don't keep
track of what all methods are available in each class (especially String,
Array, and Hash).

It's a way to provide some tertiary standardization
around common Ruby idioms.

Yep. I think your library is providing things that YOU think should be part
of Ruby. If somebody else provides methods for what they think should be
part of Ruby, your library and their code most likely won't play together.

 

[Gary Wright]

Yep. I think your library is providing things that YOU think
should be part
of Ruby. If somebody else provides methods for what they think
should be
part of Ruby, your library and their code most likely won't play
together.

Which was exactly my point. Distributed library development is
difficult.

Another way to think about this entire problem is in the same way we
view distributed source code control systems. How do you manage
conflicts between independently evolved code bases? At some point
when you try to merge two files/classes/methods that are in conflict
you have to decide how to resolve the conflict. The fact that the
attempted merge is happening at runtime is interesting but the
conflict/merge problem exists even if the attempted merge was
occurring at compile time, or even at the design stage (i.e. can I
use a particular 3rd party gem to implement xyz?).

Gary Wright

 

[Trans]

You don't have to yell, but I'm glad to hear this. BTW, "redefine" is
probably the better term to use. Derived classes "override" methods in
their base class.

In the context of the previous methods, I falsely jumped to the conclusion
that Hash#- was part of the std ruby distribution since a conflict in what
the functionality should be was being discussed. I apologize. I don't keep
track of what all methods are available in each class (especially String,
Array, and Hash).


Yep. I think your library is providing things that YOU think should be part
of Ruby. If somebody else provides methods for what they think should be
part of Ruby, your library and their code most likely won't play together.

The majority of Facets' comes directly from the field. Either by
direct contribution or through my pursing the works of others. The
rdocs atest to that by the fact that it cites dozens of authors. Sure,
some of them are strictly my ideas (or my implementation of other's
ideas). But I work hard to focus the library for general usecases and
fairly obvious definitions.

T.

 

[Florian Frank]

Hmm I do not think that one can overrule freeze; I never found a way,
anyone else?

main>> class A; end
# => nil
main>> A.freeze
# => A
main>> class A; def foo;end ;end
TypeError: can't modify frozen class
from (irb):8
main>> A = A.dup
(irb):9: warning: already initialized constant A
# => A
main>> class A; def foo;end ;end
# => nil

This doesn't work, if $SAFE == 4, though.

 

[Clifford Heath]

FACETS DOES NOT OVERRIDE ANY BUILT-IN METHODS.

I enjoy using Facets, but it does cause problems in combination
with other common frameworks, particularly Rails. It'd be really
good to see some effort being put to getting common behaviors for
methods provided by commonly-used frameworks. It's not really
enough just to avoid modifying the built-ins.

Someone I helped recently had tried to use Treetop (which uses
Facets) with Rails, and I think it was that works differently.
I found that the size of the collision was very small because
Treetop doesn't use many Facets methods, but in the case of wider
collisions, it could have been very awkward.

It's nice when everything just works, so framework authors should
try for compatibility with other frameworks. Perhaps there's a
case to be made for operating a registry/catalog of core class
extensions made by frameworks that wish to remain compatible?

Clifford Heath.

 

[Marc Heiler]

I actually think we should avoid the term "monkey-patching" at all.

It seems to imply a somewhat negative, and what is even more important,
inaccurate term. The first thought one gets at reading "monkey-patch"
will be that it is associated with something inherently bad/negative.

Look at the term "global variable" instead. It does not imply that
using it is bad instantly (which in fact it is not, it is just
that there are many better ways than global variables in most
cases. Same with class vars.) and it even gives a tiny little
help, by stating that its scope is "global".

I enjoy using Facets, but it does cause problems in combination
with other common frameworks, particularly Rails.

I guess that would not be totally unexpected, rails seems to be
non regular in many situations, for the better or worse.
Rails, for one example, even insists(ed?) on evaling my .irbrc and
in my humble opinion, Rails has exactly 0 valid reasons to look
there - even moreso as Rails choked on some stuff inside it,
whereas the normal irb has had exactly 0 problems with it. Huh ... ?

By the way, about .freeze - I think .freeze is a little bit... I mean,
it
does not have many valid use cases? Once the object is frozen,
it is kinda useless from that point on since it will no longer
change, thus losing advantages of code and starting to behave more
like a static configuration/Marshalled code.
I cant really recall having used .freeze the last 3 years hmm

I think your library is providing things that YOU think should
be part of Ruby.

Personally I also sometimes come to the point where I ask why
a certain idiom or solution is not part of official ruby.
And in a few situations I think the most popular idioms
should be considered for conclusion, I think that happened
in a few rails idioms which became part to ruby in 1.9.x

But for me, this is happily no problem at all, since you can
freely extend Ruby to your liking. (Though I am still often
confused when to use alias, and when to use alias_method instead...)

That is an advantage you get compared to i.e. python more
quickly and readily in Ruby IMHO.

Although I do not use facets (I don't really have a
use case, for personal stuff i use my own files, and for
distributing stuff I try to avoid all my own modifications
and more relying on "official" ruby code/way instead), I think
there is in fact a use case for something like facets.
And I am quite confident that the situation would change
if something like that would be included in default ruby too.
At least it would be a more standardized solution
But that is just my personal opinion, let's see what can or
will happen all the way up to 2.0

Regards.

 

[James Britt]

I enjoy using Facets, but it does cause problems in combination
with other common frameworks, particularly Rails. It'd be really
good to see some effort being put to getting common behaviors for
methods provided by commonly-used frameworks.

It's just too much work.

I suspect, though, that the very thing being derided is exactly what
would make such a byzantine scheme unnecessary.

If there is a combination of tools you would like to use, but they
collide in some way, it is quite possible that deliberate file-loading
order, combined with re-opening/selective re-definition of the offending
classes, would allow you to carry on.

Such class orchestration could become a meta-library; those who want it
can build it, while the library authors can continue to scratch their
own itch.

--
James Britt

"The trouble with the world is that the stupid are cocksure and the
intelligent are full of doubt."
- Bertrand Russell

 

[Trans]

I enjoy using Facets,
Thanks.

but it does cause problems in combination
with other common frameworks, particularly Rails. It'd be really
good to see some effort being put to getting common behaviors for
methods provided by commonly-used frameworks. It's not really
enough just to avoid modifying the built-ins.

Someone I helped recently had tried to use Treetop (which uses
Facets) with Rails, and I think it was that works differently.
I found that the size of the collision was very small because
Treetop doesn't use many Facets methods, but in the case of wider
collisions, it could have been very awkward.

I do put in effort to get Facets to not collide with ActiveSupport.
But being one person --and only an occasional Rails user, it's rather
difficult to catch everything. Moreover, ActiveSupport is squarely
geared toward Rails, which means many of the extensions are just
Railisms. Take for instance #camelcase. In Rails that not only
converts a snakecase string to camelcase, but will replace '::' with
'/' for the sake of creating pathnames out of class names.
Understandable for use by ActiveRecord, but is that what any one
thinks of as camel-casing? Not really. A better name for Rails version
would be #pathize IMHO. But I have no control over that. While this is
a rather minor case, for any given case I have to weigh whether it is
better to stay compatible with ActiveSupport or not. That's the thing
really. With Facets I have to look toward total generality --
considering what would probably work best for any Ruby program,
including Rails. ActiveSupport's doesn't have a reciprocal focus.

It's nice when everything just works, so framework authors should
try for compatibility with other frameworks. Perhaps there's a
case to be made for operating a registry/catalog of core class
extensions made by frameworks that wish to remain compatible?

That's an interesting idea for sure. I've considered progressing
Facets in that direction for a while. But it's a fairly big job and
one has to ask if its really worth the effort. It might just be better
if more people would contribute to Facets, since that would largely
have the same effect.

T.

 

[Trans]

If there is a combination of tools you would like to use, but they
collide in some way, it is quite possible that deliberate file-loading
order, combined with re-opening/selective re-definition of the offending
classes, would allow you to carry on.

Before I once again ditch the effort, I just want to point out that I
have tried to make Facets more flexible in this regard, but there
always issues. For instance, David Black recommend wrapping extensions
in modules so they could be used per object via #extend. This would
also make rdoc'ing a little easier, adn allow other extension to get
"in front" of Facets depending on load order. All good ideas, but it
has the following issues:

1) Extending a whole class would require an extra "class X; include E;
end" call, or a secondary file for every extension file that would do
it for you. For backward compatibility I would have to do the latter,
which means a large set of new files to maintain.

2) You can't add aliases for core/standard methods via a module. It
requires some #included tricks to do so and that doesn't RDoc.

3) Modules can't include extension modules b/c of the Double Module
Inclusion Problem. That means extensions to Enumerable, for instance,
won't be any better off then they are now. This adds an asymmetry to
the system, not to mention the RDocs.

4) To offer the same granularity to extending objects that Facets now
offers to extending class/modules would require the creation of many
modules. Is it worth the extra memory overhead?

It's too bad that Ruby isn't more flexible about what we can do with
classes. If it were possible to include a class, I wouldn't have to
mess with any of this. Anyone could load an extension into a
protective module space and reuse it to their hearts content.

The other option is to use a special #extension method. Eg.
extension :Array, :foomethod do ... I've seen other play with this
idea, and it has some merits. But it has some issues as well.

1) You can't get suitable RDocs from it at all. I know we shouldn't
program for the sake of docs, but in real life good docs are very
important.

2) It can produce a lot of blocks, so like the module idea the
corresponding memory footprint question arises.

Perhaps by combining the two approaches in some way it would be
possible to largely mitigate their short comings. But in either case
the memory footprint question remains. Is it worth a few hundred
modules and/or blocks for this level of control?

Of course the bottom line question though is: Would people really use
it? Or, in reality, is it just easier to fix library conflicts
manually as they occasionally come up?

T.

 

[Clifford Heath]

I do put in effort to get Facets to not collide with ActiveSupport.
But ... Take for instance #camelcase. In Rails that not only
converts a snakecase string to camelcase, but will replace '::' with
'/' for the sake of creating pathnames out of class names.
Understandable for use by ActiveRecord, but is that what any one
thinks of as camel-casing? Not really. A better name for Rails version
would be #pathize IMHO. But I have no control over that.

All true, but a dialog with the Rails guys might yield some compromise.

That's an interesting idea for sure. ... But it's a fairly big job and
one has to ask if its really worth the effort.

I agree. However, it's not too hard to find where there might be
conflicts. For example, the following program yields a list of
methods (class and instance) added (but not those monkey-patched)
during a require. Facets and ActiveSupport have 45 extensions in
common. It wouldn't take too long to look through the implementations
looking for behavioural differences.

I have this program installed as "extensions":

#! /usr/bin/env ruby

# Return a hash for each defined class containing a array of two arrays,
# the first containing the class methods and the second the instance methods
def methods_by_class
Module.
constants.
map{|klass| eval(klass)}.
select{|klass| klass.is_a? Class}.
inject({}){|h,klass|
h[klass] = [
klass.methods-klass.superclass.methods,
klass.instance_methods-(klass.superclass ? klass.superclass.instance_methods : [])
]
h
}
end

before = methods_by_class

ARGV.each{|a| require a }

after = methods_by_class

# Print the difference between the before and after method lists:
before.keys.sort_by{|k| k.to_s}.each{|k|
class_diff = after[k][0]-before[k][0]
instance_diff = after[k][1]-before[k][1]
next if class_diff.empty? && instance_diff.empty?
puts((class_diff.sort.map{|c| "#{k}."+c} +
instance_diff.sort.map{|c| "#{k}#"+c}
)*"\n")
}

 

[Trans]

I do put in effort to get Facets to not collide with ActiveSupport.
But ... Take for instance #camelcase. In Rails that not only
converts a snakecase string to camelcase, but will replace '::' with
'/' for the sake of creating pathnames out of class names.
Understandable for use by ActiveRecord, but is that what any one
thinks of as camel-casing? Not really. A better name for Rails version
would be #pathize IMHO. But I have no control over that.

All true, but a dialog with the Rails guys might yield some compromise.

That's an interesting idea for sure. ... But it's a fairly big job and
one has to ask if its really worth the effort.

I agree. However, it's not too hard to find where there might be
conflicts. For example, the following program yields a list of
methods (class and instance) added (but not those monkey-patched)
during a require. Facets and ActiveSupport have 45 extensions in
common. It wouldn't take too long to look through the implementations
looking for behavioural differences.

I have this program installed as "extensions":

#! /usr/bin/env ruby

# Return a hash for each defined class containing a array of two arrays,
# the first containing the class methods and the second the instance methods
def methods_by_class
Module.
constants.
map{|klass| eval(klass)}.
select{|klass| klass.is_a? Class}.
inject({}){|h,klass|
h[klass] = [
klass.methods-klass.superclass.methods,
klass.instance_methods-(klass.superclass ? klass.superclass.instance_methods : [])
]
h
}
end

before = methods_by_class

ARGV.each{|a| require a }

after = methods_by_class

# Print the difference between the before and after method lists:
before.keys.sort_by{|k| k.to_s}.each{|k|
class_diff = after[k][0]-before[k][0]
instance_diff = after[k][1]-before[k][1]
next if class_diff.empty? && instance_diff.empty?
puts((class_diff.sort.map{|c| "#{k}."+c} +
instance_diff.sort.map{|c| "#{k}#"+c}
)*"\n")
}

Thanks for this, btw. I working on the next release of Facets, taking
into consideration this entire conversation, and I'm using this code
to ensure compatibility with ActiveSupport.

T.

 

[Clifford Heath]

...
# Print the difference between the before and after method lists:
before.keys.sort_by{|k| k.to_s}.each{|k|
class_diff = after[k][0]-before[k][0]
instance_diff = after[k][1]-before[k][1]
next if class_diff.empty? && instance_diff.empty?
puts((class_diff.sort.map{|c| "#{k}."+c} +
instance_diff.sort.map{|c| "#{k}#"+c}
)*"\n")
}

Thanks for this, btw. I working on the next release of Facets, taking
into consideration this entire conversation, and I'm using this code
to ensure compatibility with ActiveSupport.

Glad to be able to help. Note that this code only detects added
methods, not changed ones, but since Facets doesn't change any
that shouldn't matter in this case. It would however be possible
to gather the actual Method objects into a hash and detect when
a name binds to a different Method object - to detect monkey
patching. Hmm, a new feature for "extensions.rb"

Clifford Heath.