assigning to hash keys when there is a default value?

[Robert Klemme]

From: (e-mail address removed) [mailto:[email protected]] On Behalf Of Russell= Norris:
# I learned that x ||=3D y means set x to y unless x, so I don't see =

the

there's the bug: you've changed the meaning of x ||=3D y, or of foo<o=

p>=3Dbar for that matter.

# bug. and i don't see this as being at all unpredictable.

you just said you've _learned that x||=3Dy means set x to y unless x.=

If '"x+=3D1" will always be "x=3Dx+1"', what's the problem in having x ||= =3D
1 always be x =3D x || 1?

That an assignment of x=3Dx is useless (basically a nop in the standard
case of x being a variable). Please see also one of my earlier
postings.

Kind regards

robert

 

[Rick DeNatale]

2007/9/3, (e-mail address removed) <[email protected]>:

I can't point my finger on it but I believe x||=y is equivalent to
"x=y unless x" instead of "x=x||y". It seems to be more reasonable to
skip the assignment altogether if the value is true equivalent
already. That would also explain behavior much better.

Robert,

Although I can't find the documentation quickly, although I'm 95%
certain that it should be in the pickaxe somewhere, I'm pretty sure
that you are correct.

I've just looked at parse.y and eval.c for ruby1.8.6 and it would appear that:

h[2] ||= 10

gets compiled to a NODE_OP_ASGN_OR node with h[2] as the lhs and 10 as
the rhs. Here's the code from eval.c which evaluates such a node:

case NODE_OP_ASGN_OR:
if ((node->nd_aid && !is_defined(self, node->nd_head, 0)) ||
!RTEST(result = rb_eval(self, node->nd_head))) {
node = node->nd_value;
goto again;
}
break;

So what happens is that the lhs is only evaluated if the the lhs
(node->nd_head) is not defined || it evaluates to an untrue value.

In the case of h[5] the default value for the hash means that it will
evaluate to 5, and the assignment is not done.

I for one, am glad that it works this way. The ruby idiom

x ||= y

is heavily used for lazy initialization/caching. While most often,
it's the rhs which is expensive to compute and therefore the thing we
want to short-circuit, since x= can in general be a method, and might
just be expensive, then optimizing the case where it boils down to x =
x as a nop, makes sense.

 

[Robert Klemme]

The problem with that is that || doesn't work that way, imo.

x || whatever

only does whatever if x isn't true. Excuse me if I'm sounding like a
broken record but the more we talk about this the more I'm convinced
that this "bug" only exists if you expect || to act like + does just
because it's an "operator" instead of allowing for the fact that ||
isn't a method like + and friends.

|| is special. It takes another bus to ||= than + does, heh.

Sorry for the silliness there.

I wonder why nobody commented on my attempt to point at the practical
implications. Sure I understand that it would be more consistent if
x||=y were equivalent to x = x||y but there are two questions here
IMHO that need to be answered: 1. which solution has advantages in
practice and 2. does it matter at all /in practice/ which approach is
taken?

Kind regards

robert

 

[dblack]

Hi --

2007/9/3, (e-mail address removed) <[email protected]>:

I can't point my finger on it but I believe x||=y is equivalent to
"x=y unless x" instead of "x=x||y". It seems to be more reasonable to
skip the assignment altogether if the value is true equivalent
already. That would also explain behavior much better.

Robert,

Although I can't find the documentation quickly, although I'm 95%
certain that it should be in the pickaxe somewhere, I'm pretty sure
that you are correct.

I've just looked at parse.y and eval.c for ruby1.8.6 and it would appear that:

h[2] ||= 10

gets compiled to a NODE_OP_ASGN_OR node with h[2] as the lhs and 10 as
the rhs. Here's the code from eval.c which evaluates such a node:

case NODE_OP_ASGN_OR:
if ((node->nd_aid && !is_defined(self, node->nd_head, 0)) ||
!RTEST(result = rb_eval(self, node->nd_head))) {
node = node->nd_value;
goto again;
}
break;

So what happens is that the lhs is only evaluated if the the lhs
(node->nd_head) is not defined || it evaluates to an untrue value.

In the case of h[5] the default value for the hash means that it will
evaluate to 5, and the assignment is not done.

I for one, am glad that it works this way. The ruby idiom

x ||= y

is heavily used for lazy initialization/caching. While most often,
it's the rhs which is expensive to compute and therefore the thing we
want to short-circuit, since x= can in general be a method, and might
just be expensive, then optimizing the case where it boils down to x =
x as a nop, makes sense.

I think The method always gets called, though:

class C
attr_reader :x
def x=(n)
puts "C#x="
true
end
end

c = C.new
c.x ||= 3 # C#x=

*Unless*, of course, the object is a Hash which has either (a) a key
corresponding to the indicated value, or (b) a default value with
boolean truth value.

Sigh. I really wish it were otherwise. What an annoying exception to
the rule.

Also, in the famous:

h = Hash.new(1)
h[5] ||= 10

case, it definitely isn't doing the "x = x" equivalent, since that
would set the 5 key to 1.

I don't know.... However many times I look at it, I just can't see
this:

h[5] ||= 10

as *not* meaning that I expect h to end up having a 5 key, one way or
another.


David

--
* Books:
RAILS ROUTING (new! http://www.awprofessional.com/title/0321509242)
RUBY FOR RAILS (http://www.manning.com/black)
* Ruby/Rails training
& consulting: Ruby Power and Light, LLC (http://www.rubypal.com)

 

[Rick DeNatale]

Hi --

2007/9/3, (e-mail address removed) <[email protected]>:

x ||= y is, I think, always supposed to be exactly equivalent to
x = x || y, ...

I can't point my finger on it but I believe x||=y is equivalent to
"x=y unless x" instead of "x=x||y". It seems to be more reasonable to
skip the assignment altogether if the value is true equivalent
already. That would also explain behavior much better.

Robert,

Although I can't find the documentation quickly, although I'm 95%
certain that it should be in the pickaxe somewhere, I'm pretty sure
that you are correct.

I've just looked at parse.y and eval.c for ruby1.8.6 and it would appear that:

h[2] ||= 10

gets compiled to a NODE_OP_ASGN_OR node with h[2] as the lhs and 10 as
the rhs. Here's the code from eval.c which evaluates such a node:

case NODE_OP_ASGN_OR:
if ((node->nd_aid && !is_defined(self, node->nd_head, 0)) ||
!RTEST(result = rb_eval(self, node->nd_head))) {
node = node->nd_value;
goto again;
}
break;

So what happens is that the lhs is only evaluated if the the lhs
(node->nd_head) is not defined || it evaluates to an untrue value.

In the case of h[5] the default value for the hash means that it will
evaluate to 5, and the assignment is not done.

I for one, am glad that it works this way. The ruby idiom

x ||= y

is heavily used for lazy initialization/caching. While most often,
it's the rhs which is expensive to compute and therefore the thing we
want to short-circuit, since x= can in general be a method, and might
just be expensive, then optimizing the case where it boils down to x =
x as a nop, makes sense.

I think The method always gets called, though:

class C
attr_reader :x
def x=(n)
puts "C#x="
true
end
end

c = C.new
c.x ||= 3 # C#x=

*Unless*, of course, the object is a Hash which has either (a) a key
corresponding to the indicated value, or (b) a default value with
boolean truth value.

No, in the case you posited, the assignment happened and C#x= got
called because c.x returned nil.

irb(main):001:0> class D
irb(main):002:1> def x
irb(main):003:2> @x || 5
irb(main):004:2> end
irb(main):005:1> def x=(v)
irb(main):006:2> puts "x=called"
irb(main):007:2> @x = v
irb(main):008:2> end
irb(main):009:1> end
=> nil
irb(main):010:0> d = D.new
=> #<D:0xb7b47528>
irb(main):011:0> d.x ||= 10
=> 5
irb(main):012:0> d.x
=> 5
irb(main):013:0> d.x=10
x=called
=> 10
irb(main):014:0> d.x
=> 10
irb(main):015:0>

Sigh. I really wish it were otherwise. What an annoying exception to
the rule.

Except that the 'rule' wasn't as you thought. The rule is that

x ||= y

is the same as

x = y unless x

Also, in the famous:

h = Hash.new(1)
h[5] ||= 10

case, it definitely isn't doing the "x = x" equivalent, since that
would set the 5 key to 1.

And that's as expected because h[5] returns the default value and
doesn't affect the state of the hash a whit, it doesn't create a 5
key. If you want the default to affect the hash you need something
like

hsh = Hash.new {|h,k| h[k] = 10}

$ fri Hash.new
-------------------------------------------------------------- Hash::new
Hash.new => hash
Hash.new(obj) => aHash
Hash.new {|hash, key| block } => aHash
------------------------------------------------------------------------
Returns a new, empty hash. If this hash is subsequently accessed
by a key that doesn't correspond to a hash entry, the value
returned depends on the style of new used to create the hash. In
the first form, the access returns nil. If obj is specified, this
single object will be used for all default values. If a block is
specified, it will be called with the hash object and the key, and
should return the default value. It is the block's responsibility
to store the value in the hash if required.

h = Hash.new("Go Fish")
h["a"] = 100
h["b"] = 200
h["a"] #=> 100
h["c"] #=> "Go Fish"
# The following alters the single default object
h["c"].upcase! #=> "GO FISH"
h["d"] #=> "GO FISH"
h.keys #=> ["a", "b"]

Note the value of h.keys at the end of the RI example.

I don't know.... However many times I look at it, I just can't see
this:

h[5] ||= 10

as *not* meaning that I expect h to end up having a 5 key, one way or
another.

Maybe just one more try?! <G>

 

[7stud --]

I don't like the idea that the syntactic sugar is

actually not a reliable drop-in replacement for the thing it's
sugaring.

Apparently, ||= is sugar free.

 

[dblack]

Hi --

No, in the case you posited, the assignment happened and C#x= got
called because c.x returned nil.

Right; I got that wrong.

Except that the 'rule' wasn't as you thought. The rule is that

x ||= y

is the same as

x = y unless x

OK, then: What an annoying exception to what should be the rule

Also, in the famous:

h = Hash.new(1)
h[5] ||= 10

case, it definitely isn't doing the "x = x" equivalent, since that
would set the 5 key to 1.

And that's as expected because h[5] returns the default value and
doesn't affect the state of the hash a whit, it doesn't create a 5
key. If you want the default to affect the hash you need something
like

hsh = Hash.new {|h,k| h[k] = 10}

That one I didn't get wrong I didn't say that retrieving the
default value creates a key (which it doesn't, since the default
value, whether nil or what you set it to, is specifically the default
value for keys that don't exist). My point was that this:

h = Hash.new(1)
h[5] ||= 10

does not map to "x = x", assuming that x stands for h[5]. h[5] = h[5]
*does* set a key; as I said, it would set the 5 key to 1. In fact
this whole thread is really about the fact that hash defaults, which
don't set keys, can be true, which short-circuits the ||= thing. I do
think it's the only such case, and probably fairly edge, though
obviously I'd like to see it do otherwise.


David

--
* Books:
RAILS ROUTING (new! http://www.awprofessional.com/title/0321509242)
RUBY FOR RAILS (http://www.manning.com/black)
* Ruby/Rails training
& consulting: Ruby Power and Light, LLC (http://www.rubypal.com)

 

[Peña, Botp]

From: Robert Klemme [mailto:[email protected]]=20
# 2007/9/4, Yossef Mendelssohn <[email protected]>:
# > If '"x+=3D1" will always be "x=3Dx+1"', what's the problem in=20
# having x ||=3D
# > 1 always be x =3D x || 1?
# That an assignment of x=3Dx is useless (basically a nop in the =
standard
# case of x being a variable). =20

consider,

irb(main):074:0* h=3DHash.new
=3D> {}
irb(main):075:0> h[5] =3D h[5]
=3D> nil
irb(main):076:0> h
=3D> {5=3D>nil}
irb(main):062:0* h=3DHash.new(1)
=3D> {}
irb(main):063:0> h[5] =3D h[5]
=3D> 1
irb(main):064:0> h
=3D> {5=3D>1}

thus h[k] =3D h[k] could mean
h[k] =3D h.default or
h[k] =3D nil

definitely not useless and not noop.

seeking enlightenment -botp

 

[Robert Klemme]

From: Robert Klemme [mailto:[email protected]]
# 2007/9/4, Yossef Mendelssohn <[email protected]>:
# > If '"x+=3D1" will always be "x=3Dx+1"', what's the problem in
# having x ||=3D
# > 1 always be x =3D x || 1?
# That an assignment of x=3Dx is useless (basically a nop in the standard
# case of x being a variable).

consider,

irb(main):074:0* h=3DHash.new
=3D> {}
irb(main):075:0> h[5] =3D h[5]
=3D> nil
irb(main):076:0> h
=3D> {5=3D>nil}
irb(main):062:0* h=3DHash.new(1)
=3D> {}
irb(main):063:0> h[5] =3D h[5]
=3D> 1
irb(main):064:0> h
=3D> {5=3D>1}

thus h[k] =3D h[k] could mean
h[k] =3D h.default or
h[k] =3D nil

definitely not useless and not noop.

I said "noop in the standard case of a variable". We can certainly
debate about usefulness or uselessness. I concede that it's not free
of side effects in the case of a Hash, but the only noticeable side
effect is the change of the key set. Assigning with the Hash's
default is not visible through Hash#[] although it is with #fetch
which seems to be rarely used from what I see. So I still say that
usefulness of self assignment is very limited.

Kind regards

robert

 

[dblack]

--1926193751-478355187-1188989391=:1699
Content-Type: MULTIPART/MIXED; BOUNDARY="1926193751-478355187-1188989391=:1699"

This message is in MIME format. The first part should be readable text,
while the remaining parts are likely unreadable without MIME-aware tools.

--1926193751-478355187-1188989391=:1699
Content-Type: TEXT/PLAIN; charset=X-UNKNOWN; format=flowed
Content-Transfer-Encoding: QUOTED-PRINTABLE

Hi --

From: Robert Klemme [mailto:[email protected]]
# 2007/9/4, Yossef Mendelssohn <[email protected]>:
# > If '"x+=3D1" will always be "x=3Dx+1"', what's the problem in
# having x ||=3D
# > 1 always be x =3D x || 1?
# That an assignment of x=3Dx is useless (basically a nop in the standar= d
# case of x being a variable).

consider,

irb(main):074:0* h=3DHash.new
=3D> {}
irb(main):075:0> h[5] =3D h[5]
=3D> nil
irb(main):076:0> h
=3D> {5=3D>nil}
irb(main):062:0* h=3DHash.new(1)
=3D> {}
irb(main):063:0> h[5] =3D h[5]
=3D> 1
irb(main):064:0> h
=3D> {5=3D>1}

thus h[k] =3D h[k] could mean
h[k] =3D h.default or
h[k] =3D nil

definitely not useless and not noop.

I said "noop in the standard case of a variable". We can certainly
debate about usefulness or uselessness. I concede that it's not free
of side effects in the case of a Hash, but the only noticeable side
effect is the change of the key set. Assigning with the Hash's
default is not visible through Hash#[] although it is with #fetch
which seems to be rarely used from what I see. So I still say that
usefulness of self assignment is very limited.

I'd be happy for unnecessary assignments to be optimized away, but I
like the conceptual clarity of the x =3D x || y interpretation for x ||=3D
y.


David

--=20
* Books:
RAILS ROUTING (new! http://www.awprofessional.com/title/0321509242)
RUBY FOR RAILS (http://www.manning.com/black)
* Ruby/Rails training
& consulting: Ruby Power and Light, LLC (http://www.rubypal.com)
--1926193751-478355187-1188989391=:1699--
--1926193751-478355187-1188989391=:1699--

 

[dblack]

--1926193751-1451961309-1188989512=:1699
Content-Type: MULTIPART/MIXED; BOUNDARY="1926193751-1451961309-1188989512=:1699"

This message is in MIME format. The first part should be readable text,
while the remaining parts are likely unreadable without MIME-aware tools.

--1926193751-1451961309-1188989512=:1699
Content-Type: TEXT/PLAIN; charset=X-UNKNOWN; format=flowed
Content-Transfer-Encoding: QUOTED-PRINTABLE

Hi --

From: Robert Klemme [mailto:[email protected]]
# 2007/9/4, Yossef Mendelssohn <[email protected]>:
# > If '"x+=3D1" will always be "x=3Dx+1"', what's the problem in
# having x ||=3D
# > 1 always be x =3D x || 1?
# That an assignment of x=3Dx is useless (basically a nop in the standard
# case of x being a variable).

consider,

irb(main):074:0* h=3DHash.new
=3D> {}
irb(main):075:0> h[5] =3D h[5]
=3D> nil
irb(main):076:0> h
=3D> {5=3D>nil}
irb(main):062:0* h=3DHash.new(1)
=3D> {}
irb(main):063:0> h[5] =3D h[5]
=3D> 1
irb(main):064:0> h
=3D> {5=3D>1}

thus h[k] =3D h[k] could mean
h[k] =3D h.default or
h[k] =3D nil

definitely not useless and not noop.

It's really always h[k] =3D h.default -- it's just that the default
default, so to speak, is nil.


David

--=20
* Books:
RAILS ROUTING (new! http://www.awprofessional.com/title/0321509242)
RUBY FOR RAILS (http://www.manning.com/black)
* Ruby/Rails training
& consulting: Ruby Power and Light, LLC (http://www.rubypal.com)
--1926193751-1451961309-1188989512=:1699--
--1926193751-1451961309-1188989512=:1699--

 

[Peña, Botp]

It's really always h[k] =3D h.default -- it's just that the default
default, so to speak, is nil.

arggh, yes.=20
i guess my mind was too focused on x=3Dx

irb(main):027:0> zz
NameError: undefined local variable or method `zz' for main:Object
from (irb):27
irb(main):028:0> zz=3Dzz
=3D> nil

kind regards -botp

 

[Robert Klemme]

Hi --

No, in the case you posited, the assignment happened and C#x= got
called because c.x returned nil.

Right; I got that wrong.

Except that the 'rule' wasn't as you thought. The rule is that

x ||= y

is the same as

x = y unless x

OK, then: What an annoying exception to what should be the rule

Also, in the famous:

h = Hash.new(1)
h[5] ||= 10

case, it definitely isn't doing the "x = x" equivalent, since that
would set the 5 key to 1.

And that's as expected because h[5] returns the default value and
doesn't affect the state of the hash a whit, it doesn't create a 5
key. If you want the default to affect the hash you need something
like

hsh = Hash.new {|h,k| h[k] = 10}

That one I didn't get wrong I didn't say that retrieving the
default value creates a key (which it doesn't, since the default
value, whether nil or what you set it to, is specifically the default
value for keys that don't exist). My point was that this:

h = Hash.new(1)
h[5] ||= 10

does not map to "x = x", assuming that x stands for h[5]. h[5] = h[5]
*does* set a key; as I said, it would set the 5 key to 1. In fact
this whole thread is really about the fact that hash defaults, which
don't set keys, can be true, which short-circuits the ||= thing. I do
think it's the only such case, and probably fairly edge, though
obviously I'd like to see it do otherwise.

I believe it's not the only case. Although x= and []= are different, it
seems in *both* cases assignment is not even invoked for ||=:

irb(main):001:0> class Foo
irb(main):002:1> def x
irb(main):003:2> p "x"
irb(main):004:2> @x
irb(main):005:2> end
irb(main):006:1> def x=(v)
irb(main):007:2> p "x="
irb(main):008:2> @x=v
irb(main):009:2> end
irb(main):010:1> def [](k)
irb(main):011:2> p "[]"
irb(main):012:2> k
irb(main):013:2> end
irb(main):014:1> def []=(k,v)
irb(main):015:2> p "[]="
irb(main):016:2> k
irb(main):017:2> end
irb(main):018:1> end
=> nil
irb(main):019:0> f=Foo.new
=> #<Foo:0x7ff6b5c4>
irb(main):020:0> f.x||=10
"x"
"x="
=> 10
irb(main):021:0> f.x||=20
"x"
=> 10
irb(main):022:0> f.x=30
"x="
=> 30
irb(main):023:0> f[10]||=20
"[]"
=> 10
irb(main):024:0> f[nil]||=30
"[]"
"[]="
=> 30
irb(main):025:0> f[20]=30
"[]="
=> 30
irb(main):026:0>

Kind regards

robert

 

[dblack]

Hi --

That one I didn't get wrong I didn't say that retrieving the
default value creates a key (which it doesn't, since the default
value, whether nil or what you set it to, is specifically the default
value for keys that don't exist). My point was that this:

h = Hash.new(1)
h[5] ||= 10

does not map to "x = x", assuming that x stands for h[5]. h[5] = h[5]
*does* set a key; as I said, it would set the 5 key to 1. In fact
this whole thread is really about the fact that hash defaults, which
don't set keys, can be true, which short-circuits the ||= thing. I do
think it's the only such case, and probably fairly edge, though
obviously I'd like to see it do otherwise.

I believe it's not the only case. Although x= and []= are different, it
seems in *both* cases assignment is not even invoked for ||=:

True, but by "only *such* case" I meant: only case where evaluating
the lhs and getting true doesn't tell the whole story, in terms of the
functionality of the object, because of the defaulting to a value in
the absence of a key.

I suppose one could argue that it isn't ||='s problem if the lhs is
actually a kind of proxy. Nothing has yet convinced me, though, that
||= should not behave like x = x || y, even if the actual assignment
gets optimized away.


David

--
* Books:
RAILS ROUTING (new! http://www.awprofessional.com/title/0321509242)
RUBY FOR RAILS (http://www.manning.com/black)
* Ruby/Rails training
& consulting: Ruby Power and Light, LLC (http://www.rubypal.com)

 

[Robert Klemme]

I believe it's not the only case. Although x= and []= are different, it
seems in *both* cases assignment is not even invoked for ||=:

True, but by "only *such* case" I meant: only case where evaluating
the lhs and getting true doesn't tell the whole story, in terms of the
functionality of the object, because of the defaulting to a value in
the absence of a key.

Ah, ok. Should've read more carefully. I am sorry.

I suppose one could argue that it isn't ||='s problem if the lhs is
actually a kind of proxy. Nothing has yet convinced me, though, that
||= should not behave like x = x || y, even if the actual assignment
gets optimized away.

Umm, now I am confused. I thought the "optimized away" bit is the
critical bit - if you allow for this difference then they do actually
behave the same, don't they?

Btw, here's a possible explanation why the behavior is the way it is:
[]= and x= are usually costly operations (i.e. not just assignments
but methods doing some work) so avoiding that would help overall
performance. Still I believe that's probably better than the
consistency although I usually tend to favor consistency as well.

Kind regards

robert

 

[dblack]

Hi --

I believe it's not the only case. Although x= and []= are different, it
seems in *both* cases assignment is not even invoked for ||=:

True, but by "only *such* case" I meant: only case where evaluating
the lhs and getting true doesn't tell the whole story, in terms of the
functionality of the object, because of the defaulting to a value in
the absence of a key.

Ah, ok. Should've read more carefully. I am sorry.

I suppose one could argue that it isn't ||='s problem if the lhs is
actually a kind of proxy. Nothing has yet convinced me, though, that
||= should not behave like x = x || y, even if the actual assignment
gets optimized away.

Umm, now I am confused. I thought the "optimized away" bit is the
critical bit - if you allow for this difference then they do actually
behave the same, don't they?

That may be right. I guess I'm focusing on the visible behavior, and
not the implementation, so I'm probably saying dumb things about the
implementation.

Btw, here's a possible explanation why the behavior is the way it is:
[]= and x= are usually costly operations (i.e. not just assignments
but methods doing some work) so avoiding that would help overall
performance. Still I believe that's probably better than the
consistency although I usually tend to favor consistency as well.

That was Rick DeNatale's point too: that it could be expensive to call
a =-method. I agree, though if x[1] ||= y were just sugar for x[1] =
x[1] || y, then one could optimize it on the Ruby side with x[1] = y
unless x[1].

Oh well. I don't think this is going to change, and at this point I
should probably stop complaining unless I can come up with a new
implementation


David

--
* Books:
RAILS ROUTING (new! http://www.awprofessional.com/title/0321509242)
RUBY FOR RAILS (http://www.manning.com/black)
* Ruby/Rails training
& consulting: Ruby Power and Light, LLC (http://www.rubypal.com)