Feature Request: Special file '-' denoting stdin/stdout

[Ronald Fischer]

I would like to propose a feature for Ruby which can be
found in Perl, and which is also frequently found in
many Unix utilities and to my knowledge in all of the=20
Gnu stuff:

When a file is opened for reading, and the filename is
"-", the file is assumed to be standard input.

When a file is opened for writing, and the filename is
"-", the file is assumed to be standard output.

This come very handy when writing applications which accept
filenames from, say, the command line or some configuration
file. Without providing any special logic inside the application,
input and output can be switched to stdin and stdout.=20

Ronald
--=20
Ronald Fischer <[email protected]>
Phone: +49-89-452133-162
=20

 

[Bertram Scharpf]

Hi,

Am Freitag, 29. Jun 2007, 18:47:43 +0900 schrieb Ronald Fischer:

I would like to propose a feature for Ruby which can be
found in Perl, [...]

When a file is opened for reading, and the filename is
"-", the file is assumed to be standard input.

When a file is opened for writing, and the filename is
"-", the file is assumed to be standard output.

Something like

class <<File
def open_dashed name, mode = nil
if name = "-" then
if mode =~ /[w+]/ then
yield $>
else
yield $<
end
else
File.open name, mode do |f| yield f end
end
end
end

Untested.

Bertram

 

[Erik Veenstra]

It should be mentioned that both File.new and File.open are
able to receive file descriptors:

$ ruby -e 'File.new(1).write 17' | ruby -e 'p File.new(0).read'
"17"

$ ruby -e 'File.open(1){|f| f.write 17}' | ruby -e 'p File.open(0){|
f| f.read}'
"17"

$ ruby -e 'f=File.new("test", "w") ; File.open(f.fileno){|f| f.write
17}'

$ ruby -e 'f=File.new("test") ; p File.open(f.fileno){|f| f.read}'
"17"

(note: 0==$stdin, 1==$stdout and 2==@stderr)

If you insist on using "-", you might want to have a look on
either of the implementations below.

gegroet,
Erik V. - http://www.erikveen.dds.nl/

----------------------------------------------------------------

class YourFile < File
def self.new(file, mode="r", *perm)
if file == "-"
mode.include?("r") ? $stdin : $stdout
else
super(file, mode, *perm)
end
end

def self.open(file, mode="r", &block)
if file == "-"
super(new(file, mode).fileno, mode, &block)
else
super(file, mode, &block)
end
end
end

----------------------------------------------------------------

class File
@@_non_dashed_new = methodnew)
@@_non_dashed_open = methodopen)

def self.new(file, mode="r", *perm)
if file == "-"
mode.include?("r") ? $stdin : $stdout
else
@@_non_dashed_new.call(file, mode, *perm)
end
end

def self.open(file, mode="r", &block)
if file == "-"
@@_non_dashed_open.call(new(file, mode).fileno, mode, &block)
else
@@_non_dashed_open.call(file, mode, &block)
end
end
end

----------------------------------------------------------------

 

[Ronald Fischer]

It should be mentioned that both File.new and File.open are
able to receive file descriptors:

This does not help much, because still I would then need to decide
at runtime, whether (for example) the number 1 passed via the command
line of my program, denotes the file descriptor 1 (stdout) or a file
name "1".

If you insist on using "-", you might want to have a look on
either of the implementations below.

Both implementations are fine, but I never doubted that we can
implement this explicitly per application. My point was that this
usage of the "-" has become so common - perhaps because it is=20
heavily used in the Gnu world and is built into Perl as well. As
an example for how it works with the Gnu tools, I present a sketch
of a bash script:

#!/usr/bin/bash
# This is file exa1.sh
...
fgrep "$1" /var/mail/log $HOME/mailbox $2
...

I have several ways to call it, for example

exa1.sh foo # search in /var/mail/log and mailbox
exa1.sh bar mailbox.old # search also in mailbox.old
genmail | exa1.sh baz - # also search in the stdout of genmail

Similar in Perl: If you write

open(RCFILE,$ARGV[3]) # This is Perl

the user can supply on the command line a dash, and the program
would automatically read stdin for RCFILE.

Since this is such a common usage of the '-', I suggest that
it should be incorporated into Ruby in a similar way as in Perl.

Ronald
--=20
Ronald Fischer <[email protected]>
Phone: +49-89-452133-162
=20

 

[SonOfLilit]

This creates some awful situations, like having a file called '-' and
doing ./script * . I'm against it - it's a hack.


Aur Saraf

It should be mentioned that both File.new and File.open are
able to receive file descriptors:

This does not help much, because still I would then need to decide
at runtime, whether (for example) the number 1 passed via the command
line of my program, denotes the file descriptor 1 (stdout) or a file
name "1".

If you insist on using "-", you might want to have a look on
either of the implementations below.

Both implementations are fine, but I never doubted that we can
implement this explicitly per application. My point was that this
usage of the "-" has become so common - perhaps because it is
heavily used in the Gnu world and is built into Perl as well. As
an example for how it works with the Gnu tools, I present a sketch
of a bash script:

#!/usr/bin/bash
# This is file exa1.sh
...
fgrep "$1" /var/mail/log $HOME/mailbox $2
...

I have several ways to call it, for example

exa1.sh foo # search in /var/mail/log and mailbox
exa1.sh bar mailbox.old # search also in mailbox.old
genmail | exa1.sh baz - # also search in the stdout of genmail

Similar in Perl: If you write

open(RCFILE,$ARGV[3]) # This is Perl

the user can supply on the command line a dash, and the program
would automatically read stdin for RCFILE.

Since this is such a common usage of the '-', I suggest that
it should be incorporated into Ruby in a similar way as in Perl.

Ronald

 

[Gregory Brown]

Since this is such a common usage of the '-', I suggest that
it should be incorporated into Ruby in a similar way as in Perl.

This reminds me of David Black's post about this kind of logic:
http://www.infoq.com/articles/coming-from-ruby

I don't think that just because something is common in other languages
/ tool chains that it necessarily means it belongs in Ruby.

Personally, I'm happy to see many Rubyists moving away from the
symbolic shortcuts seen in Perl, and I am a former perl programmer.

If it feels useful to you, perhaps release a gem that implements it
and require it in all your projects.

-greg

 

[Ronald Fischer]

Since this is such a common usage of the '-', I suggest that

=20
This reminds me of David Black's post about this kind of logic:
http://www.infoq.com/articles/coming-from-ruby
=20
I don't think that just because something is common in other languages
/ tool chains that it necessarily means it belongs in Ruby.

I agree with the basic idea - just because being in other languages,
does
not mean it should go to Ruby. The crucial point was that it has been
found IMO *useful* in other languages. Actually, Ruby does it that way
in other areas too - when it adopted, for example, the Regular
Expressions
or the interface to the net/telnet library from Perl.=20

So the basic point - and I think that is the essence of a feature
request
- is to find out to what extent the Ruby community would find it useful
too. If it is just a minority, it certainly should not go into "Basic
Ruby". If we feel that a huge number of people would be glad, we could
consider moving it somehow into the core. One goal of my posting was
to see how other people would like the feature. To my surprise, there
was no positive response, so I guess that not so many would be happy=20
with it.

Ronald

 

[Ronald Fischer]

This creates some awful situations, like having a file called '-' and

doing ./script * . I'm against it - it's a hack.

I think that's why people try to avoid filenames like "-" at all. You
run into
the same problem if you do for example one of

grep foo *
tail -f - # Reads from stdin. Use tail -f <- to read from file '-'

At least this is true if you are working in a "Gnu environment" (Linux,
Cygwin,...);
I don't know about Solaris or HP-UX.

For similar reasons, we usually avoid filenames containing spaces,
non-printable
characters and so on, even if they are perfectly valid.

Of course the "Magic '-'" feature, if it would ever go into Ruby, must
not be
there unconditionally; the programmer still needs to be in control on
whether
or not she wants to have it implemented (this could maybe be done by
providing a
separate "open" call, say: File.open_magic_dash).

Ronald

 

[micathom]

File.open_magic_dash

If you're willing to use an extra name, why not something like this:

def handle_magic_dash(filename, mode)
[filename == '-' ? (mode =~ /r/ ? 2 : 1) : filename, mode]
end

File.new(*handle_magic_dash('-', 'w')) ...
File.new(*handle_magic_dash('-', 'r')) ...
File.new(*handle_magic_dash('foo', 'w')) ...
File.new(*handle_magic_dash('foo', 'r')) ...

 

[Robert Klemme]

I think that's why people try to avoid filenames like "-" at all. You
run into
the same problem if you do for example one of

grep foo *
tail -f - # Reads from stdin. Use tail -f <- to read from file '-'

It's not exactly equivalent. The equivalent command would be

tail -f ./-

At least this is true if you are working in a "Gnu environment" (Linux,
Cygwin,...);
I don't know about Solaris or HP-UX.

For similar reasons, we usually avoid filenames containing spaces,
non-printable
characters and so on, even if they are perfectly valid.

Of course the "Magic '-'" feature, if it would ever go into Ruby, must
not be
there unconditionally; the programmer still needs to be in control on
whether
or not she wants to have it implemented (this could maybe be done by
providing a
separate "open" call, say: File.open_magic_dash).

IMHO the question is whether a string is the proper interface to a
method. Consider this:

def write_stuff_file(out)
File.open(out, "w") do |io|
io << ...
end
end

vs.

def write_stuff_stream(out)
out << ...
end

The second form is more flexible than the first one because you can
pass not only file names but IO objects, Strings, StringIOs etc. And
the second method does not need the "-" hack because you can just pass
$stdout.

Also, when using ARGF for reading then you automatically get input
from stdin if there are no arguments in ARGV. This is another
situation where you do not need "-" as representative for stdout or
stdin.

Kind regards

robert

 

[Gregory Brown]

So the basic point - and I think that is the essence of a feature
request
- is to find out to what extent the Ruby community would find it useful
too. If it is just a minority, it certainly should not go into "Basic
Ruby". If we feel that a huge number of people would be glad, we could
consider moving it somehow into the core. One goal of my posting was
to see how other people would like the feature. To my surprise, there
was no positive response, so I guess that not so many would be happy
with it.

Sure, it"s a good idea to ask questions like these. I guess I was
just cautioning against over-emphasis of utility in other languages.
I think that perl probably inherited the '-' from UNIX based tools,
and probably a few years ago, people would have been more receptive.

Nowadays, we're looking to avoid making Ruby unreadable, even at the
cost of some symbolic hacky fun.

 

[John Joyce]

Sure, it"s a good idea to ask questions like these. I guess I was
just cautioning against over-emphasis of utility in other languages.
I think that perl probably inherited the '-' from UNIX based tools,
and probably a few years ago, people would have been more receptive.

Nowadays, we're looking to avoid making Ruby unreadable, even at the
cost of some symbolic hacky fun.

You can always write your own module or library for it and make it
available as a gem.
If it is popular and useful enough, it might become standard Ruby.

 

[Ronald Fischer]

IMHO the question is whether a string is the proper interface to a

method. =20

Agreed. I think the problem originates from the the Unix/Linux/Solaris
world, where it is often not possibile to specify stdin/stdout/stderr
in the same way as a filename (some however *do*, AFIK, by allowing
something
like "/dev/fd0" to be used as a "filename" denoting stdin).

Consider this:
=20
def write_stuff_file(out)
File.open(out, "w") do |io|
io << ...
end
end
=20
vs.
=20
def write_stuff_stream(out)
out << ...
end
=20
The second form is more flexible than the first one because you can
pass not only file names but IO objects, Strings, StringIOs etc. And
the second method does not need the "-" hack because you can just pass
$stdout.

I would rarely use the "-" convention within my program; only as a way
to
receive this uniformly from the outside. And, yes, I'm not that much
satisfied with using just a dash to denote it. It only happened to be
used that way so often, so people are likely familiar with it.=20

=20
Also, when using ARGF for reading then you automatically get input
from stdin if there are no arguments in ARGV. This is another
situation where you do not need "-" as representative for stdout or
stdin.

If it were just for this case, we would for sure not need a "-". But=20
consider the Linux cat command, which takes an arbitrary number of
filenames, concatenates their content and writes it to stdout. Here
an example invocation of cat:

middle|cat foo - bar >baz

This first copies the content of file foo to baz, then appends the
output
of the process "middle" to baz, and finally appends file bar to baz.
From
the user interface point of view, you need some way here to represent
stdin on the command line, and by convention this is done by cat.

If you implement this in Ruby, you certainly *have* to deal with "-"
explicitly,
for example along the way Erik Veenstra proposed. I use for instance in
my
code calls like this:

f =3D ARGV[n]=3D=3D"-" ? $stdin : File.open(ARGV[n])

to implement this behaviour.

This for sure can be done explicitly, but if things are done over and
over
in the same way by different teams of programmers, it makes sense to
make
a general solution from it. And, as some people in this list pointed
out,
maybe it would be really best to provide such a feature as gem and see=20
whether it becomes popular. I will see whether I can do it (as I have
never
implemented a Gem so far).

Ronald

 

[Robert Klemme]

Agreed. I think the problem originates from the the Unix/Linux/Solaris
world, where it is often not possibile to specify stdin/stdout/stderr
in the same way as a filename (some however *do*, AFIK, by allowing
something
like "/dev/fd0" to be used as a "filename" denoting stdin).


I would rarely use the "-" convention within my program; only as a way
to
receive this uniformly from the outside. And, yes, I'm not that much
satisfied with using just a dash to denote it. It only happened to be
used that way so often, so people are likely familiar with it.


If it were just for this case, we would for sure not need a "-". But
consider the Linux cat command, which takes an arbitrary number of
filenames, concatenates their content and writes it to stdout. Here
an example invocation of cat:

middle|cat foo - bar >baz

This first copies the content of file foo to baz, then appends the
output
of the process "middle" to baz, and finally appends file bar to baz.
From
the user interface point of view, you need some way here to represent
stdin on the command line, and by convention this is done by cat.

This does work in Ruby, try it out:

ls -la | ruby -ne 'puts $_' some_file - some_other_file

If you implement this in Ruby, you certainly *have* to deal with "-"
explicitly,
for example along the way Erik Veenstra proposed. I use for instance in
my
code calls like this:

f = ARGV[n]=="-" ? $stdin : File.open(ARGV[n])

to implement this behaviour.

This for sure can be done explicitly, but if things are done over and
over
in the same way by different teams of programmers, it makes sense to
make
a general solution from it. And, as some people in this list pointed
out,
maybe it would be really best to provide such a feature as gem and see
whether it becomes popular. I will see whether I can do it (as I have
never
implemented a Gem so far).

For a certain subset there *is* a solution (see above). As of now I
think that is sufficient but maybe I'm wrong here.

Kind regards

robert