LWP::UserAgent question

[Mumia W.]

I do not believe it is a ToS violation; GigaNews' terms may be found
here:
http://www.giganews.com/legal/aup.html


Prehaps you will be kind enough to explain your objections. I have, on
numerious occasions (along with many others) flagged multiposted
messages (manually). I don't recall that anyone has ever been critical
of this practice, and such notifications are common courtesy in
professional newsgroups.

Now I write a bot to do this and several people complain (but don't
really say why). I thought folks like us write Perl scripts to automate
repetitive manual tasks.

Is it the fact that it's a bot that bothers you? That would seem an
odd objection for a programmer. Or do you not like the content of the
auto-messages? I'm not an English major - I would be happy to consider
content edits. But I don't want to simply furnish a link - I've seen
lots of links given to lots of OP's (Posting Guidelines, etc) which
seem to be ignored. Prehaps a direct reply such as this would be more
effective.

Or do you think my code is crap? I would be happy to post it for peer
review.

Regarding the length of the message: There is a lot of "introductory"
info there which was designed to be temporary (as the message says). I
have removed this text, and the message is now down to a much more
resonable size (about 60 lines).

If you don't like seeing the messages, you may killfile the scanner
(just as you may killfile the Faq-O-Matic bot or the Posting Guidelines
bot). If you think I'm a jerk for doing this then you may killfile me
as well. But, first, I hope you would at least give me a chance to
understand and address any concerns you have.

Thank you, Mr. Filmer, for being willing to consider our
opinions. Here is my input:

1) Your message misstates that multi-posting is universally
considered rude; that's false; I don't consider it rude;
instead please say this, "The author of this 'bot, David
Filmer <usenet [at] DavidFilmer.com>, believes multi-posting
to be rude."

2) Anonymously running a 'bot that posts to usenet is 100
times ruder that multi-posting. Please put that in the message.

3) You refer to this 'bot as a "service to the Perl
community," but instead, it is offensive to the Perl
community, so you should put that in the message.

Thank you for your consideration.

 

[Ben Morrow]

Quoth (e-mail address removed):

But it's easy to detect a multipost with a program (without AI). It is
fully possible to eliminate false positives.

Umm... not a good track record so far.

A few false-negatives may
slip through (where the OP tweaks the content) and that's to be
expected. The majority of multiposts CAN be identified (and, in fact, I
cannot recall a single multipost which was not a simple cut-and-paste -
I did go back and research the multiposts that I'vemanually flagged and
every single one of them would have been flagged by my bot).

And did you also go through all *other* posts, and check if there were
any you had the sense not to reply to (say, because the poster was a
known troll, so there'd be no point), that your bot would have hit?

Where are you seeing such volume??? The scanner was deployed last
Thursday, and as of now (wee hours of Monday morning in my $TZ), I've
only observed two messages get flagged (and my tracking database agrees
with my observation). And the first message was an improperly flagged
crosspost because I had not un-commented a line during testing (grrr).

So not only are you creating more junk in this group with a bot, you're
creating *even* *more* by not bothering to test it properly? Gee,
thanks.

This is an argument against bots that try to do anything 'clever' in
general. Usenet is a volatile place: it's very easy to offend people,
so a bot will almost certainly slip up at some point, because it doesn't
ever understand the full ramifications of what it's replying to.

But I am not opposed to the idea of a vote, as you suggest.

Count mine against. (And, in general, against (Perl-)content-free
'correction' replies altogether.)

I'm a reasonable guy;

By my lights a 'reasonable guy' would have asked what the group thought
*FIRST*.

Ben

 

[usenet]

Ok, a short reply (hah).

Thanks for your reply - I respect and appreciate your input.

From what I have seen so far, your bot replies to each
multiposted message (correct me if I am wrong), in each group.

That's correct. Of course, that's not behavior specific to a bot -
users often do that (manually), if they notice. I always flag
multiposts (when I notice).

Meaning if someone multipost in 3 groups what we might get is:
1 post per group (3 in total)

Right. Of course, my bot does nothing to contribute to this count.

1 reply by your bot per post (3 in total)

Right. My bot does not add to this count any more than if I (or someone
else) manually flagged each post (which I (and others) commonly try to
do). The bot only increases the count to the extent that humans did
not notice or bother/desire to flag the multipost. I believe most
multiposts get flagged, so the bot really isn't having much impact
here.

0 or more possible replies per post by people not being aware of the
multipost.

My bot does nothing to increase message traffic from people who don't
know about my bot. Folks are always free to reply to a multiposted
question, so if someone is determined to reply, my bot has no effect on
this count (especially if the person has killfiled the bot).

0 or more people replying to your bot replies (complaints, asking what it is.)

There's nothing in the reply text (now) to indicate the message is
coming from a bot. Folks who are not already familiar with the bot
(newbie OP's, for example) have no way to know it's not coming from a
vigilant user with a canned reply. The original couple messages had a
lot of background info that I've now removed.

As far as complaints go - some people feel that usenet should be
laissez faire anarchy and object to any standards of conduct and will
flame anybody (human or bot) that suggests otherwise (we've seen
hotheads like that in this group complaining that Tad would DARE
maintain posting guidelines). I don't think we should be discouraged
(or even influenced) by hotheads like that...

0 or more angry replies by people who discovered the multipost. [1]

The only such replies should come from folks who killfile the bot (as
your [1] footnote suggests is the likely scenario for that). People
who are motivated to flag multiposts manually are unlikely, IMHO, to
killfile a bot that does it for them (usenet anarchists might killfile
the bot, but those folks aren't ever gonna gripe about a multipost). I
think this category of message would be very rare.

Does it stop multiposting: no

I would think that someone who lurks before posting might notice that
multiposts are quickly flagged and thus be discouraged from ever doing
so. Most newbie GG posters who don't bother to lurk will not be
discouraged, of course - the first time. But I doubt they would
consider it in their best interest to do it again. So a bot won't
eleminate multiposting, but it should reduce it (especially from users
who do this more than once - which is something we've all seen happen).

Does it add a lot of noise to the group: I am afraid that the answer is yes.

If it works as it should, I believe it should reduce both multiposting
and related noise (for the reasons I've already stated). Of course,
there's a lot of traffic at the moment with you and I (and others)
discussing the issue, but I don't think such traffic is representative
of what would be expected in the future.

If you just wanted to teach people not to multipost, carefully reread what
your bot spits out and try to wear the shoes of the offender. Do you
really think that 140+ (IIRC)
something anyone really is going to read without being offended?

As I have noted, the initial message contained a lot of background
info, which was intended for the benefit of the group-at-large (not the
OP). I've now removed that (but no multipost has since been made). The
new message is down to 60 lines.

lines with big black boxes (####) is

I thought the black boxes added to readability by clearly marking
sections. But maybe it makes the message look "heavy" (which might
contribute to making it look flame-y). That's a good point, and one
that I hadn't considered. I'll fix that.... And I'll review the text
to see if I think it can be reworded more nicely (though I think it's
not unkind now - direct, prehaps, but not unkind).

What was wrong with:
Don't post the same message to several groups (multipost), see
http://www.cs.tut.fi/~jkorpela/usenet/xpost.html.

In my experience, newbie types don't bother to follow links. How many
times have we given a new poster a link to the CLPMisc posting
guidelines only to see no evidence that s/he ever followed or read the
linked page? But I've seen instances where an issue has been pointed
out in the message body and the OP reads and understands. I think
posters are more likely to read replies than links in replies.

But I'm open to that idea of a note and a link (and someone else
suggested that as well). I'm trying to balance what will be effective
with what will be overboard (and the original message was clearly
overboard with all the intro text, which I have now removed).

 

[usenet]

Umm... not a good track record so far.

True, the first post the bot flagged was a crosspost. I had
commented-out a line which distinguishes crossposts from multiposts
(for testing) and forgot to uncomment it (the line checks if a
uniq-ified list of message ID's is greater than one). That's truly
embarrassing.

So not only are you creating more junk in this group with a bot, you're
creating *even* *more* by not bothering to test it properly?

Ironically, the problem was introduced during testing (and I did test
rather extensively - see alt.test.testing, alt.test.test2, and
alt.test.test).

This is an argument against bots that try to do anything 'clever' in general.

I'm not sure this bot is so 'clever.' I agree that 'clever' programs
are risky (especially when they run in the wild), but all I'm doing is
calculating md5sum()s on the message body (and, just to address that
one-in-a-zillion chance that two bodies hash to the same value, I also
calculate the md5sum of the reversed message). That's a pretty
straightforward way to identify identical messages; no cleverness is
required here.

I welcome peer review of my code by any interested individuals.

And did you also go through all *other* posts, and check if there were
any you had the sense not to reply to (say, because the poster was a
known troll, so there'd be no point), that your bot would have hit?

I always reply to multiposts, even from known trolls. Because not
everybody knows who the trolls are. Heck, I might even answer a
properly posted question from a known troll (it might encourage the
troll to post properly).

Count [my vote] against.

Noted, thanks.

By my lights a 'reasonable guy' would have asked what the group thought
*FIRST*.

You're right - I was wrong not to. I really didn't anticipate that most
folks would even take an interest in my little bot. I was apparently
mistaken about that. I should have opened a discussion before
deploying the bot.

 

[usenet]

1) Your message misstates that multi-posting is universally
considered rude; that's false

Although I detect a faint hint of sarcasm in most of your message, you
are absolutely correct in this regard. I will reword it to say "widely
considered," which I think is an accurate statement.

 

[John Bokma]

John Bokma wrote:

0 or more angry replies by people who discovered the multipost. [1]

The only such replies should come from folks who killfile the bot

No. Due to how Usenet works people might not see your bots reply for other
reasons.

If it works as it should, I believe it should reduce both multiposting
and related noise (for the reasons I've already stated). Of course,
there's a lot of traffic at the moment with you and I (and others)
discussing the issue, but I don't think such traffic is representative
of what would be expected in the future.

My experience, not with running a bot, but with reporting spam which has
the advantage that a lot of abusers lose their accout, is that the noise
keeps coming.

I thought the black boxes added to readability by clearly marking
sections.

That's why we all use them in our posts. I consider the layout of your
original message and the length abusive (no, I am not joking).

In my experience, newbie types don't bother to follow links.

Ah, and they are going to read 140+ lines with big black boxes?

But I'm open to that idea of a note and a link (and someone else
suggested that as well). I'm trying to balance what will be effective
with what will be overboard (and the original message was clearly
overboard with all the intro text, which I have now removed).

IMO, the bot should go overboard. It's a waste of bandwidth, and it's not
going to prevent people from multiposting.

 

[John Bokma]

I always reply to multiposts, even from known trolls. Because not
everybody knows who the trolls are. Heck, I might even answer a
properly posted question from a known troll (it might encourage the
troll to post properly).

So what's next? A "This is a known troll" bot which replies to each and
every known troll with "Don't feed the trolls"?

I think the most important rule IMO on Usenet most people have to study
very well is:

*You don't have to reply*

Your bot breaks that one.

 

[usenet]

So what's next? A "This is a known troll" bot which replies to each and
every known troll with "Don't feed the trolls"?

Hmmmm.... Naah. Trolls can reform (though I don't know if that theory
has ever been proven).

But I get the point you are making. And, as I've agreed before, Perl
is a blunt instrument when it comes to parsing natural language for
meaning and intent (such as troll posts), so there's good reason not to
try to bot-flag most messages. But Perl can be very precise at flagging
multiposts. I've always thought it's good to do with Perl what Perl
does well.

I think the most important rule IMO on Usenet most people have to study
very well is:
*You don't have to reply*
Your bot breaks that one.

You are absolutely right - the bot is much too heavy-handed, and goes
too far in that regard. I'll fix that...

 

[DJ Stunks]

So what's next? A "This is a known troll" bot which replies to each and
every known troll with "Don't feed the trolls"?

I think the most important rule IMO on Usenet most people have to study
very well is:

*You don't have to reply*

Your bot breaks that one.

And how many replies have you had on this subject? And on the Xah Lee
subject (no, I wasn't joking)

I'm not a fan of Xah's rants or David's bot, but I don't mind ignoring
and I certainly wouldn't resort to reporting either of them. I also
find it odd that you would make anti-Bush comments when my experience
is that you're not very libertarian yourself...

-jp

 

[axel]

**********************************************************************
********** PLEASE DO NOT RESPOND TO THIS THREAD ***********
**********************************************************************

This message has been multiposted as indicated by these message IDs:
<JoPDg.386093$Mn5.194189@pd7tw3no>
<GbPDg.382930$iF6.158066@pd7tw2no>

It might be more informative if the actual groups to which the messages
were multiposted were named.

[ Excessive verbage snipped ]

# NAQ (the Never Asked Questions - TEMPORARY SECTION) #
# Q-What ruleset defines a multipost? A--Two MD5 digests are #
# calculated (one forwards, one backwards) on the message body #
# and stored in a small database. If a new post (new message ID) #
# matches these digests, it's a multipost. Author and subject #
# lines are not taken into account. Reply messages are ignored. #
# Q-Are crossposts flagged? A--No. A message must have an identical #
# body but a unique message ID to be flagged. A crosspost which is #

Presumably this means that a single extra space or line distinguishing
two messages (easy enough to do even if not deliberately using
cut-and-paste) would not cause them to be flagged as multiposted. So the
bot would be only partically effective.

# Q-What groups are scanned? A-The main Perl Groups in Google Groups,#
# namely: comp.lang.perl.misc, perl.beginners, perl.dbi.users, #
# comp.lang.perl.modules , perl.beginners.cgi, and alt.perl #

I can see a problem here... perl.beginners exists as a mailing list
which is gatewayed to Usenet. Someone might mail to the list and also
post to a Usenet group, not realising that perl.beginners also appears
on Usenet, although possibly not on his server.

Axel

 

[Matt Garrish]

How is it not working? We know that guidelines alone don't work...

It's not working in that it wasn't well conceived. As I wrote in
another thread on this topic, if it can be toned down from the rape and
pillage bot to a simple informative bot it would be very helpful.
There's nothing more annoying than answering a question in one group
only to find it in another. And a gentler approach with fewer #s
probably would have found greater acceptance. But as pointed out by Ben
Morrow, he really should have asked the group first before unleashing
this monstrosity.

Matt

 

[Matt Garrish]

(e-mail address removed) wrote:
[ snip and ignore MULTIPOSTED message ]

I think this multiposting bot is a great idea.

Me no, and I have reported it for what I see it is right now: Usenet
abuse.

What happened to your "last warning", Bokma? Itchy reporting finger?

Come on, he's fighting the good fight so we can all have a better
Bokmanet experience...

Matt

 

[Matt Garrish]

(e-mail address removed) wrote:
[ snip and ignore MULTIPOSTED message ]

I think this multiposting bot is a great idea.

Me no, and I have reported it for what I see it is right now: Usenet
abuse.

What happened to your "last warning", Bokma? Itchy reporting finger?

And it's good to see that Bokma still believes democracy should reign.
I have no doubt that David would remove the bot if it comes to that,
but why wait for the discussion to end when we can all report abuse.
I've taken the time in this instance to report what a nut Bokma is and
hope others will too.

Matt

 

[John Bokma]

And how many replies have you had on this subject? And on the Xah Lee
subject (no, I wasn't joking)

They are far from the 600+ replies Xah seems to attract after a single
cross post to 5 groups.

I'm not a fan of Xah's rants or David's bot, but I don't mind ignoring
and I certainly wouldn't resort to reporting either of them. I also
find it odd that you would make anti-Bush comments when my experience
is that you're not very libertarian yourself...

Instead of posting brain farts, educate yourself. For one, I don't live in
the USA.

 

[John Bokma]

I've taken the time in this instance to report what a nut Bokma is and
hope others will too.

Reporting as in what all those Xah fans claimed to do? Which was fun,
since one actually did and my hosting provider contacted me for a
translation since o boy had this sockpuppet mixed up things.

What he is or was doing in a technical oriented news group is still beyond
me since if one can't grasps some basics...

 

[DJ Stunks]

They are far from the 600+ replies Xah seems to attract after a single
cross post to 5 groups.

wait, (I'm so confused). It's ok to reply as long as you're not the
600th?

Instead of posting brain farts, educate yourself. For one, I don't live in
the USA.

If ever there was a brain fart, I think your response was it. Are
those sentences connected in some way to each other, nevermind to my
point? Are you somehow 'super educated' simply by _not_ living in the
US?

In the future do us all a favor and do the following any time you see
something you don't like on Usenet:

1) look up 'libertarian' in a dictionary; and
2) re-read (and study if necessary) your advice (reprinted here
verbatim for your convenience):

"I think the most important rule IMO on Usenet most
people have to study very well is: *You don't have to reply*"


We definitely have a Usenet Cop in our midst, only it isn't actually
David...

-jp

 

[DJ Stunks]

wait, (I'm so confused). It's ok to reply as long as you're not the
600th?

oh, hang on, I get it now. I should have been able to see this sooner:
it's only ok to reply so long as your initials are John Bokma.

my bad.

-jp

 

[usenet]

I have opened a new thread in CLPMisc to consolidate continued
discussion on the multipost bot: http://tinyurl.com/nkg7y (this
discussion is clearly OT to this original post).

If you wish to reply regarding this topic, I respectfully ask that you
do so in the new thread (and quote context so folks can follow along).

 

[John Bokma]

my bad.

Glad you discovered it finally.

 

[DJ Stunks]

Glad you discovered it finally.

uhhh, ok, based on this evidence could everyone please cross "super
educated" off on your lists?

for those of you keeping track, "thug" is now the frontrunner.

-jp