One-liner removing duplicate lines

[Robert Klemme]

Many thanks to everyone who responded, the answers are very
interesting and enlightening !

As far as I can see noone used the Hash with block feature. So here it
is:

ruby -e 'h=Hash.new(){|ha,l| puts l;ha[l.freeze]=1};ARGF.each {|l| h[l]}'

(David, I'm sorry again this is no #inject solution.)

Kind regards

robert

 

[Louis J Scoras]

------=_Part_4899_23305975.1128601893165
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

$ cat -n input | sort -k2 | uniq -f1 | sort | sed -e 's/^ *[0-9]*\t//' >
output

Woops, that second sort needs an `-n` flag to sort numeric. Above will only
work if you have <10 lines.

------=_Part_4899_23305975.1128601893165--

 

[Sean O'Halpin]

(David, I'm sorry again this is no #inject solution.)

I couldn't resist

ruby -e "ARGF.inject({}) {|h, x| h.key?(x) or h[x] =3D puts(x); h}"

To handle missing newline at EOF:

ARGF.inject({}) {|h, x| h.key?(x.chomp) or h[x.chomp] =3D puts(x); h}

Regards,

Sean

 

[Devin Mullins]

(David, I'm sorry again this is no #inject solution.)

I couldn't resist

ruby -e "ARGF.inject({}) {|h, x| h.key?(x) or h[x] = puts(x); h}"

I think he was looking for a functional (non-side-effecty) version:
ruby -e"ARGF.inject(['']) {|s,l| s.include?(l)?s:s<<l}.display"

Devin

 

[Mark Hubbart]

Hello,

Converting from Perl to Ruby, I am trying to find an equivalent to this
Perl one-liner removing duplicate lines in a file (without sorting it a= t
first) :

perl -ne'$s{$_}++||print' infile >outfile

I guess uniq method could be used, but I can't find how.

I tried creating a version that mimics the Perl one (because Ruby also
has the -n option), but in the end this seemed easier (and much more
readable):

ruby -e "puts IO.readlines(ARGV[0]).uniq" infile > outfile

So you are right about using uniq.

Here's a derived version (is this really Ruby?):

ruby -e'$><<[*$<].uniq' infile > outfile

cheers,
Mark

 

[Mark Hubbart]

Hello,

Converting from Perl to Ruby, I am trying to find an equivalent to th= is
Perl one-liner removing duplicate lines in a file (without sorting it= at
first) :

perl -ne'$s{$_}++||print' infile >outfile

I guess uniq method could be used, but I can't find how.

I tried creating a version that mimics the Perl one (because Ruby also
has the -n option), but in the end this seemed easier (and much more
readable):

ruby -e "puts IO.readlines(ARGV[0]).uniq" infile > outfile

So you are right about using uniq.

Here's a derived version (is this really Ruby?):

ruby -e'$><<[*$<].uniq' infile > outfile

As long as we're mostly obfuscated anyway, we might as well make the
whole thing punctuation:

ruby -e'$><<([*$<]|[])' infile > outfile

cheers,
Mark
...who got into the golf round a little late

 

[Ryan Leavengood]

As long as we're mostly obfuscated anyway, we might as well make the
whole thing punctuation:

ruby -e'$><<([*$<]|[])' infile > outfile

That's awesome. I was never a Perl programmer, but I can see the
appeal of golf. Sure the above is pretty incomprehensible, but the
time it takes to understand it improves your understanding of Ruby as
a whole.

Ryan

 

[Sean O'Halpin]

ruby -e'$><<([*$<]|[])' infile > outfile

That has a kind of evil beauty...

Sean

 

[David A. Black]

Hi --

As long as we're mostly obfuscated anyway, we might as well make the
whole thing punctuation:

ruby -e'$><<([*$<]|[])' infile > outfile

That's awesome. I was never a Perl programmer, but I can see the
appeal of golf. Sure the above is pretty incomprehensible, but the
time it takes to understand it improves your understanding of Ruby as
a whole.

I've found it useful to link this into golf discussions:

http://www.rubygarden.org/ruby?GolfDisclaimer




David

 

[Joel VanderWerf]

ruby -e'$><<([*$<]|[])' infile > outfile

That has a kind of evil beauty...

Sean

If it were a smiley, what emotion would it represent?

 

[Devin Mullins]

ruby -e'$><<([*$<]|[])' infile > outfile

If it were a smiley, what emotion would it represent?

"I am being eaten by a grue."

 

[David A. Black]

Hi --

ruby -e'$><<([*$<]|[])' infile > outfile

That has a kind of evil beauty...

Sean

If it were a smiley, what emotion would it represent?

The feeling you get when you're holding up a barbell while standing on
a cut-away cross-section of a partially occupied mausoleum.


David

 

[Ryan Leavengood]

ruby -e'$><<([*$<]|[])' infile > outfile

That has a kind of evil beauty...

Sean

If it were a smiley, what emotion would it represent?

It looks like a totem pole to me. The totem of Ruby Golf.

Ryan

 

[Hal Fulton]

The feeling you get when you're holding up a barbell while standing on
a cut-away cross-section of a partially occupied mausoleum.

That has *got* to be the strangest thing I've ever heard from you.

Working a little too hard on the conf?? Need a break??


Hal

 

[Christophe Grandsire]

En r=E9ponse =E0 Hal Fulton :

David A. Black wrote:
=20
=20
That has *got* to be the strangest thing I've ever heard from you.
=20
Working a little too hard on the conf?? Need a break??
=20

I just think he's been channeling why the lucky stiff .
--=20
Christophe Grandsire.

http://rainbow.conlang.free.fr

You need a straight mind to invent a twisted conlang.

 

[ToRA]

ruby -e 'require "set" ; s = Set.new ; ARGF.each_line {|z| s.add?(z)
and puts(z) }' input > output

or even more verbose

ruby -e 'require "set" ; s = Set.new ; ARGF.each_line {|z| if s.add?(z)
then puts(z) end }' input > output

Tris