Performance of SyncEnumerator

[Matt Mower]

Hi folks.

Thanks to H. Yamamoto I am now able to profile my ART-2 implementation
which I perceive to be running slowly (although i've no competing
implementations to compare it with... maybe it's really greased weasel
fast...?)

The profiler shows the bad boys are:

% cumulative self self total
time seconds seconds calls ms/call ms/call name
33.99 9.92 9.92 37772 0.26 0.26 Kernel.callcc
12.69 13.62 3.70 6118 0.61 9.94 Generator#next
10.91 16.81 3.18 12768 0.25 4.79 Generator#end?
9.97 19.72 2.91 6118 0.48 1.67 Generator#yield
8.68 22.25 2.53 3345 0.76 16.39 Array#map
6.84 24.24 2.00 266 7.50 200.63 Proc#call

Overall runtime is about 30s and everything else is, individually,
less than 2% of run-time. As you can see I'm making use of the
SyncEnumerator from generator.rb. In particular I calculate the
Euclidean distance of two vectors (and I do this a lot) as:

def euclidean_distance( vector )
Math.sqrt( SyncEnumerator.new( self, vector ).collect { |i,j| (
i-j ) ** 2 }.inject( 0 ) { |s,v| s+v } )
end

Because SyncEnumerator cleverly offers me a full Enumerable I was able
to use #collect to calculate the squares to be passed to #inject for
summation. I find this good.

However I'm now wondering if the magic which makes SyncEnumerator work
is of the expensive kind (I'm assuming this is where all the Kernel#cc
calls are coming from as well).

Does anyone have any experience they can share about SyncEnumerator performance?

Regards,

Matt

 

[Matt Mower]

As if often the case, pressing send made me think "why don't i just find out?"

The profiler shows the bad boys are:

% cumulative self self total
time seconds seconds calls ms/call ms/call name
33.99 9.92 9.92 37772 0.26 0.26 Kernel.callcc
12.69 13.62 3.70 6118 0.61 9.94 Generator#next
10.91 16.81 3.18 12768 0.25 4.79 Generator#end?
9.97 19.72 2.91 6118 0.48 1.67 Generator#yield
8.68 22.25 2.53 3345 0.76 16.39 Array#map
6.84 24.24 2.00 266 7.50 200.63 Proc#call

Re-implementing as:

def euclidean_distance2( vector )
sum = 0
each_with_index { |value,index| sum += ( ( value - vector[index]
) ** 2 ) }
Math.sqrt( sum )
end

results in a profile of:

% cumulative self self total
time seconds seconds calls ms/call ms/call name
40.16 1.59 1.59 300 5.31 23.34 Array#each
10.68 2.02 0.42 1978 0.21 0.33 Fixnum#**
5.95 2.25 0.24 2165 0.11 0.14 ART::Cluster#size
4.76 2.44 0.19 4143 0.05 0.05 Fixnum#+
4.71 2.63 0.19 3245 0.06 0.06 Array#[]
3.12 2.75 0.12 1978 0.06 0.06 Fixnum#>=
2.77 2.86 0.11 1978 0.06 0.06 Fixnum#power!

Total run-time is reduced from 29.19s to 3.97s (86% reduction).

I guess the moral of this story (for me) is that SyncEnumerator is
neat but shouldn't be used where performance is important.

Regards,

Matt

 

[Ken Kunz]

Matt,

Take a look a the recent discussion on RedHanded regarding
SyncEnumerator performance issues:
http://redhanded.hobix.com/inspect/enumerateSideBySideWithSyncenumerator.html

It boils down to: Generator uses continuations, which can be dreadfully
slow.

Enumerable#zip(arg) may provide a more elegant alternative than
each_with_index, but only if arg can be coerced into an Array.

Cheers,
Ken

 

[Matt Mower]

Hi Ken,

Take a look a the recent discussion on RedHanded regarding
SyncEnumerator performance issues:
http://redhanded.hobix.com/inspect/enumerateSideBySideWithSyncenumerator.html

It boils down to: Generator uses continuations, which can be dreadfully
slow.

Thanks for the link, I wish I'd read that before I wrote the original
code at least it might have reminded me that Enumerable#zip was an
alternative in my case ;-)

It's a pity about the slowness of continuations since we agree that
SyncEnumerator provides a very neat solution. As you observed maybe
Rite/YARV will improve the situation. We can but hope!

Thanks,

Matt