Gems not working

[Jim Hoskins]

I am currently not able to install any gems. I was using Gem 1.1.1 but
kept receiving the following errors:

ERROR: While executing gem ... (Gem::RemoteSourceException)
Error fetching remote gem cache: Errno::EPIPE: Broken pipe reading
http://gems.rubyonrails.org/yaml

That seemed to be the same problem discussed here:
http://www.ruby-forum.com/topic/149595

That solution does me no good since I am using 32bit and running FreeBSD
not Linux.

I decided to install Rubygem 1.1.0 and tried to install rails and got
this:

gem install rails
Bulk updating Gem source index for: http://gems.rubyonrails.org/
Bulk updating Gem source index for: http://gems.rubyforge.org/
ERROR: While executing gem ... (Gem::RemoteFetcher::FetchError)
Errno::ECONNRESET: Connection reset by peer reading
http://gems.rubyforge.org/gems/rake-0.8.1.gem


Additionally this is stopping me from using the Freebsd rubygem-* ports
due to the same error.

my gem env is:

RubyGems Environment:
- RUBYGEMS VERSION: 1.1.0 (1.1.0)
- RUBY VERSION: 1.8.6 (2007-09-24 patchlevel 111) [i386-freebsd6]
- INSTALLATION DIRECTORY: /usr/local/lib/ruby/gems/1.8
- RUBY EXECUTABLE: /usr/local/bin/ruby18
- RUBYGEMS PLATFORMS:
- ruby
- x86-freebsd-6
- GEM PATHS:
- /usr/local/lib/ruby/gems/1.8
- GEM CONFIGURATION:
- :update_sources => true
- :verbose => true
- :benchmark => false
- :backtrace => false
- :bulk_threshold => 1000
- :sources => ["http://gems.rubyonrails.org",
"http://gems.rubyforge.org"]
- REMOTE SOURCES:
- http://gems.rubyonrails.org
- http://gems.rubyforge.org


Any Ideas? I am at the end of my rope on this.

 

[Leslie Viljoen]

I am currently not able to install any gems. I was using Gem 1.1.1 but
kept receiving the following errors:

ERROR: While executing gem ... (Gem::RemoteSourceException)
Error fetching remote gem cache: Errno::EPIPE: Broken pipe reading
http://gems.rubyonrails.org/yaml

That seemed to be the same problem discussed here:
http://www.ruby-forum.com/topic/149595

That solution does me no good since I am using 32bit and running FreeBSD
not Linux.

I decided to install Rubygem 1.1.0 and tried to install rails and got
this:

gem install rails
Bulk updating Gem source index for: http://gems.rubyonrails.org/
Bulk updating Gem source index for: http://gems.rubyforge.org/
ERROR: While executing gem ... (Gem::RemoteFetcher::FetchError)
Errno::ECONNRESET: Connection reset by peer reading
http://gems.rubyforge.org/gems/rake-0.8.1.gem


Additionally this is stopping me from using the Freebsd rubygem-* ports
due to the same error.

my gem env is:

RubyGems Environment:
- RUBYGEMS VERSION: 1.1.0 (1.1.0)
- RUBY VERSION: 1.8.6 (2007-09-24 patchlevel 111) [i386-freebsd6]
- INSTALLATION DIRECTORY: /usr/local/lib/ruby/gems/1.8
- RUBY EXECUTABLE: /usr/local/bin/ruby18
- RUBYGEMS PLATFORMS:
- ruby
- x86-freebsd-6
- GEM PATHS:
- /usr/local/lib/ruby/gems/1.8
- GEM CONFIGURATION:
- :update_sources => true
- :verbose => true
- :benchmark => false
- :backtrace => false
- :bulk_threshold => 1000
- :sources => ["http://gems.rubyonrails.org",
"http://gems.rubyforge.org"]
- REMOTE SOURCES:
- http://gems.rubyonrails.org
- http://gems.rubyforge.org


Any Ideas? I am at the end of my rope on this.

Have you tried a couple of times? The errors just look like a slow
line that is disconnecting.

I have had strange proxy issues with Rubyforge before, but that
was always when trying to log in, never with getting gems.

Leslie

 

[Jim Hoskins]

That solution does me no good since I am using 32bit and running FreeBSD
http://gems.rubyforge.org/gems/rake-0.8.1.gem
- INSTALLATION DIRECTORY: /usr/local/lib/ruby/gems/1.8
- :backtrace => false
- :bulk_threshold => 1000
- :sources => ["http://gems.rubyonrails.org",
"http://gems.rubyforge.org"]
- REMOTE SOURCES:
- http://gems.rubyonrails.org
- http://gems.rubyforge.org


Any Ideas? I am at the end of my rope on this.

Have you tried a couple of times? The errors just look like a slow
line that is disconnecting.

I have had strange proxy issues with Rubyforge before, but that
was always when trying to log in, never with getting gems.

Leslie

That's what I thought, but other win32 machines on the same subnet work
just fine. It seems contained to the one FreeBSD Box.

A day later, still the same results, even after running a barrage of
network diagnostics and tests.

 

[Hassan Schroeder]

A day later, still the same results, even after running a barrage of
network diagnostics and tests.

Are you running IPV6 on this system? If so, I'd try disabling that -- no
good reason, just a WAG

If that doesn't help, use a packet sniffer to examine the wire traffic.

Good luck,

 

[Tom Copeland]

I am currently not able to install any gems. I was using Gem 1.1.1 but
kept receiving the following errors:

ERROR: While executing gem ... (Gem::RemoteSourceException)
Error fetching remote gem cache: Errno::EPIPE: Broken pipe reading
http://gems.rubyonrails.org/yaml

Hm, that "yaml" file is 20 MB now... is your client timing out, or maybe
refusing to fetch that much data?

Yours,

Tom

 

[Tom Copeland]

Since I've had to reinstall all my gems after the upgrade to 1.8.7,
here's an idea of what's happening with the gem servers right now:

$ for a in actionmailer RedCloth rspec rubygems-update rails fxruby
bindata actionpack gem_plugin rush ruby-debug-ide activeresource
opod ; do echo $a ; until gem install $a ; do echo Retrying $a... ;
done ; done

I think an easier way to do this might be:

sudo gem install actionmailer RedCloth [... etc ...]

actionmailer
Bulk updating Gem source index for: http://gems.rubyforge.org/
Bulk updating Gem source index for: http://gems.rubyforge.org/

I've noticed this double bulk update sometimes as well. You might want
to post a note to the rubygems-developers list about it.

Yours,

tom

 

[Tom Copeland]

I think an easier way to do this might be:

sudo gem install actionmailer RedCloth [... etc ...]

The problem with this is that if -- scratch that, when -- it fails, I
either have to go into the backscroll and figure out which pieces
installed and which didn't, or I wrap that all up in an until and if
(when) any single gem fails, the whole thing gets restarted.

Ah right, because the gem client is timing out, gotcha.

My way, while less efficient overall, at least doesn't reinstall gems
a dozen times over because the next-to-last one failed to install.
Were the gems servers actually reliable, I wouldn't have to do even
the until wrap, not to mention the grotesquely inefficient
one-at-a-time approach I took above.

What surprises me is seeing the gem client fetch the uncompressed index
(that is, the yaml file) rather than fetching the compressed index. The
compressed version is only 1MB, and seems like that would be much less
likely to time out. It might be worth posting to the
rubygems-developers list and seeing if they could help you hunt down
this problem.

Yours,

Tom

 

[Tom Copeland]

There's a lot that surprises me in the recent gem releases (1.0+).
The fact that there seems to be absolutely no caching of metadata is
the first point of surprise. Even if I tell it NOT to go looking for
the metadata it goes looking for the metadata which indicates to me
that it's not caching locally at all and has to. Then the fact, as
you point out, that it needs to download the whole YAML file instead
of the compressed version is suspicious. I'm seriously unimpressed
with the 1.0+ versions of gem, frankly.

And yet another mailing list to sign up for? God DAMN! How about
just a bug tracker somewhere? Is there an URL for that? I really
don't want to increase my incoming mail any more than I already have
it.

Yup, there's the usual complement of RubyForge trackers and such here:

http://rubyforge.org/projects/rubygems/

Yours,

Tom

 

[Leslie Viljoen]

Yup, there's the usual complement of RubyForge trackers and such here:

http://rubyforge.org/projects/rubygems/

Thanks, Tom. Lack of sleep made me not even think about looking at
RubyForge.

Sorry to repost this, but it should probably have been attached to
this thread....


Wondering why the endless "bulk updating" takes so long I looked that
the source a bit.
There seems that if there are more than 50 gems missing from the quick
list, a bulk update
is done:

source_index.rb:
use_incremental = missing_gems.size <= INCREMENTAL_THRESHHOLD

(INCREMENTAL_THRESHOLD is 50)

..yet it seems the quick list is always more than 50 gems out of date (for the
bulk update seems to always be done)

When downloading and extracting both lists, it look like there are 13464 gems
listed in the bulk file and 13432 in the quick list.

(For the curious, they are these files, compressed with zlib:
http://gems.rubyforge.org/yaml.Z
http://gems.rubyforge.org/quick/index.rz)


This is a difference of 32 gems, but then perhaps some on the quick list are out
of date? Or perhaps I got the numbers wrong?


So to make things faster:
--------------------------------

1. Since the bulk index is 854k and expands to 20MB, perhaps there's a way to
keep that quick index more up-to-date?

2. Only 3134 of the 13432 gems are unique gems - 10298 are older versions
of these gems. I think that people rarely search or install old gems, so perhaps
the list can be split into a file for latest versions versus old versions.

3. I often search for gems repeatedly, and the bulk index gets pulled down
repeatedly - why not save this file locally for at least a few hours?
(probably try to implement this myself just now)
-- did this by now, but there seems to be an attempt at this already built-in.

4. Perhaps if the server is taking strain, a mirror or two could be
set up? I doubt
many people would care about such relatively small files on their servers -
I'd be willing to ask some people if they'd do a ZA mirror.



Any comments?

Les

 

[Eric Hodel]

The gem servers at rubyforge and whoever hosts for ruby on rails
seem to be badly overloaded. I get these broken pipe errors all the
time to the point that I wrap all uses of gem in "unless" blocks in
bash to make sure things actually go through.

I do not have this problem, and my usage of RubyGems with remote
sources is excessive. Today I installed 500 gems over the internet,
55MB of gems and 1000 files during the afternoon Pacific Time, which
should be some of the peak usage times for the RubyGems mirrors.

Even on crappy coffee-shop internet, or 40K/s send-my-packets-across-
the-country EVDO internet-on-a-stick I never get Errno::EPIPE, so I
think the problem is on your end of the internet, and not on the
RubyForge or gem mirror end.

What would help is if the --no-update-sources flag actually did
something. Half the time the point of failure lies in updating the
local source cache from the remote site, but whether or not I
specify --no-update-sources the source cache is downloaded at
tremendous expense. Sometimes it's downloaded multiple times in a
single invocation (if I install a lot of gems at the command line at
once).

This code has been replaced in trunk, trunk no longer performs updates
of specs it doesn't need.

 

[Eric Hodel]

Since I've had to reinstall all my gems after the upgrade to 1.8.7,
here's an idea of what's happening with the gem servers right now:

$ for a in actionmailer RedCloth rspec rubygems-update rails fxruby
bindata actionpack gem_plugin rush ruby-debug-ide activeresource
opod ; do echo $a ; until gem install $a ; do echo Retrying $a... ;
done ; done
actionmailer
Bulk updating Gem source index for: http://gems.rubyforge.org/
Bulk updating Gem source index for: http://gems.rubyforge.org/
Bulk updating Gem source index for: http://gems.rubyonrails.org/
ERROR: While executing gem ... (Gem::RemoteSourceException)
Error fetching remote gem cache: Errno::EPIPE: Broken pipe
readinghttp://gems.rubyonrails.org/yaml

No, there is no problem with the rubyforge mirrors.

gem sources -r http://gems.rubyonrails.org/

Will probably fix your problem.

 

[Anthony E.]

No, there is no problem with the rubyforge mirrors.

gem sources -r http://gems.rubyonrails.org/

Will probably fix your problem.

appears to be a 403 if i go directly to

http://gems.rubyonrails.org