SPAM from Usenet

[James Gray]

On Usenet, there are some options:

1. Complain to the ISP of the offender.

Yes and this is fine if Usenet folks complain about the message before
it was gated. Again, please do not complain about a message on Ruby
Talk after it was gated as you are then complaining about my email
address.

James Edward Gray II

 

[James Gray]

What will happen if some mad person reads this and tries to spoil
the list?

The gateway will be shut down. If my host shuts it down, that's
that. There's nothing I can do. I pleaded our case to get it this far.

I'm not worried about this though. I believe we can discuss this
reasonably as we are now.

My suggestion:
Mark spams by replies from some responsible persons which add
[SPAMTHREAD] in topic.
Most newsreaders should be able to hide a thread marked this way.

I think we need to find a solution that doesn't involve adding to the
noise level.

James Edward Gray II

 

[Michael Bruschkewitz]

My suggestion:
Mark spams by replies from some responsible persons which add
[SPAMTHREAD] in topic.
Most newsreaders should be able to hide a thread marked this way.

I think we need to find a solution that doesn't involve adding to the
noise level.

I was aware that it would about double the number of unnecessary messages.
But it would only bring more message headers.
I thought about a solution which could be handled by readers on NNTP and on
the mailing list too.
I don't think it is possible to implement a solution without human
assistance. I also think it is not practicable to convert the NG to a
moderated one, because this would need positive acknowledgement of every
single message. Not only the amount of work would be huge, it would bring
unnecessary delays.
So, every message must pass first. Because of the need of human
intervention, it would be only possible to remove the spam belatedly.
Because I think it is nearly impossible to remove a message from ML/NG/GG
altogether, the only possibility is an additional mark. Either, this mark is
brought in inband by using the same medium as the original message, or by
using an different medium. But a different medium would need extension of
the features of the newsreaders/mailreaders.

2ct.

(Maybe "Shoes" should be renamed as this subject possibly is magnetic to all
these sellers of Nike scrap...

 

[Charles Calvert]

Yes and this is fine if Usenet folks complain about the message before
it was gated. Again, please do not complain about a message on Ruby
Talk after it was gated as you are then complaining about my email
address.

Noted.

Thank you for your efforts in maintaining the gateway. Spam issues
aside, I think that your work benefits everyone by creating a larger
community while allowing everyone to use their interface of choice.

 

[Eric Hodel]

Yes and this is fine if Usenet folks complain about the message
before it was gated. Again, please do not complain about a message
on Ruby Talk after it was gated as you are then complaining about my
email address.

Leaving the NNTP Received headers in will help us trace spam to its
origin.

 

[Charles Calvert]

No. If you look above, your post via NNTP came as sent by
"(e-mail address removed)"- I'm assuming your actual email address (if not,
should be easy to change to a valid one). The list can thereby use
that to check to see if you're "subscribed".

In other words, your user experience wouldn't change one bit, other
then having to do a one-time subscription (with delivery turned off)
in order to post.

Huh. That's an interesting suggestion. I'd personally be okay with
making the effort to maintain a subscription from the same address
that I use to post in comp.lang.ruby if it would help the folks on the
mailing list get less spam.

One downside to your suggestion is that folks who find the newsgroup
and start posting would be seen only by those reading the newsfeed
until and unless they subscribed. It's a small barrier, but a barrier
nonetheless.

 

[Charles Calvert]

[snip]

I thought about a solution which could be handled by readers on NNTP and on
the mailing list too.
I don't think it is possible to implement a solution without human
assistance. I also think it is not practicable to convert the NG to a
moderated one, because this would need positive acknowledgement of every
single message. Not only the amount of work would be huge, it would bring
unnecessary delays.

Not necessarily. As a Usenet moderator, I can tell you that there are
a number of possible solutions.

1. First, one can take the whitelist approach, using a 'bot:

a. Whitelist anything coming from ruby-talk.

b. Whitelist posters with a history of posting on-topic.

Anything on the whitelist gets automatically approved by the 'bot. If
a whitelisted poster goes rogue, you'll have to remove them from the
whitelist post facto, but that shouldn't happen often, if ever.
Whitelisting can even take paths into account, to deal with forgeries
from net.kooks.

Anything not on the whitelist gets queued for human moderation. That
should be a fairly small percentage.

2. The moderation submission address can employ spam filters to help
deal with spam that gets sent via NNTP or directly to the address.
This will prevent most of the NNTP-originated spam from being passed
to ruby-talk.

3. PGPMoose could be used by the moderation 'bot to sign approved
messages. Anything that isn't signed gets dropped by the gateway.

[snip rest]

This would take some work, but it is possible.

 

[James Gray]

Leaving the NNTP Received headers in will help us trace spam to its =20=

origin.

Fred, do you happen to know if I could legally do that? Are they =20
speced the same? I will look when I have a less crazy day, but was =20
just curious if you would know.

I wonder why they felt the need to fake them with the original gateway=85

James Edward Gray II=

 

[James Gray]

As a Usenet moderator, I can tell you that there are
a number of possible solutions.

1. First, one can take the whitelist approach, using a 'bot:

a. Whitelist anything coming from ruby-talk.

b. Whitelist posters with a history of posting on-topic.

Anything on the whitelist gets automatically approved by the 'bot. If
a whitelisted poster goes rogue, you'll have to remove them from the
whitelist post facto, but that shouldn't happen often, if ever.
Whitelisting can even take paths into account, to deal with forgeries
from net.kooks.

I'm kind of liking this approach. It does add some maintenance I =20
realize, but it's flawless and I don't think it would be too bad. =20
Food for thought=85

James Edward Gray II=

 

[F. Senault]

Le 7 juin 2009 à 03:36, Charles Calvert a écrit :

One downside to your suggestion is that folks who find the newsgroup
and start posting would be seen only by those reading the newsfeed
until and unless they subscribed. It's a small barrier, but a barrier
nonetheless.

It's far from a small barrier. If the poster uses an invalid address,
he will never know he's supposed to subscribe. Ditto if his email
server or spam filter eats the mailing-list warning message, of if he
doesn't understand it.

Not to forget that, even if he receives the warning, he'll have to
repost his message a second time, while some people on Usenet could
already respond to the first, adding to the confusion...

Fred

 

[F. Senault]

Le 6 juin 2009 à 17:22, trans a écrit :

Exactly.

Ah. I see.

That's just ironic. So we have:

Ruby-Talk <--> Usenet comp.lang.ruby <--> Google Group
comp.lang.ruby

Via Jame's gateway.

and

Ruby-Talk <--> Google Group ruby-talk-google

Yep.

Fred

 

[F. Senault]

Le 7 juin 2009 à 04:31, James Gray a écrit :

On Jun 6, 2009, at 8:35 PM, Eric Hodel wrote:

Heh. I answered to that in a previous message... which wasn't accepted
by the mailing-list software and correctly propagated...

So, I said that there's a path header instead, and a bunch of relevant
headers, notably for the web-news gateways (i.e. google groups) ; here's
a sample of the headers usenet-side for one of the last spams :

(I'll try pastie instead of pasting the raw headers...)

http://www.pastie.org/503343

I just broke down the long headers ; the path is always on one (longà)
line.

Fred, do you happen to know if I could legally do that? Are they
speced the same? I will look when I have a less crazy day, but was
just curious if you would know.

The question is moot : you don't have the same headers on one side or
the other, meaning you could put the path in the mail messages and the
received's in the usenet messages (which wouldn't be the worse idea,
IMHO).

I wonder why they felt the need to fake them with the original gateway?

James Edward Gray II

Fred

 

[F. Senault]

Le 7 juin 2009 à 03:48, Charles Calvert a écrit :

This would take some work, but it is possible.

With the sad state of most usenet servers world wide, a status change
will not be propagated correctly everywhere.

Fred

 

[Robert Klemme]

What does suck about gmail is I can't match against arbitrary headers.

What header do you want to look for? I managed to find your message via
the message id (but not with the header name included). Maybe there is
a way to do the match even in absence of general header indexing at GMail.

And if you read via POP or IMAP you can still do the filtering with your
preferred email client. But I do agree, a general header indexing and
search would certainly be good.

Kind regards

robert

 

[Aaron Turner]

What header do you want to look for? =A0I managed to find your message vi= a the
message id (but not with the header name included). =A0Maybe there is a w= ay to
do the match even in absence of general header indexing at GMail.

And if you read via POP or IMAP you can still do the filtering with your
preferred email client. =A0But I do agree, a general header indexing and
search would certainly be good.

I'd like to match:
Received: from Usenet via a Usenet to mail gateway

Yes, my IMAP mail client can do that (hell, procmail could do that in
1995), but the gmail rules can't seem to do that.



--=20
Aaron Turner
http://synfin.net/
http://tcpreplay.synfin.net/ - Pcap editing and replay tools for Unix & Win=
dows
Those who would give up essential Liberty, to purchase a little temporary
Safety, deserve neither Liberty nor Safety.
-- Benjamin Franklin

 

[James Gray]

Le 7 juin 2009 =E0 04:31, James Gray a =E9crit :


Heh. I answered to that in a previous message... which wasn't =20
accepted
by the mailing-list software and correctly propagated...

So, I said that there's a path header instead, and a bunch of relevant
headers, notably for the web-news gateways (i.e. google groups) ; =20
here's
a sample of the headers usenet-side for one of the last spams :

(I'll try pastie instead of pasting the raw headers...)

http://www.pastie.org/503343

I just broke down the long headers ; the path is always on one (long=E0)=

line.

Thanks for the suggestions.

The question is moot : you don't have the same headers on one side or
the other, meaning you could put the path in the mail messages and the
received's in the usenet messages (which wouldn't be the worse idea,
IMHO).

OK, I can do that.

I need to rewrite the news_to_mail.rb side of the gateway first =20
though. I've converted the mail_to_news.rb code to be tmail based =20
which makes it super easy to do stuff like move over a bunch of =20
headers. However, the news_to_mail.rb side is still the old Regexp =20
based code I inherited. If I update it first, this will be a lot =20
easier.

So, request received. It's a little work though and my summer is =20
pretty insanely busy. Please be patient with me if it takes me a bit =20=

to get to it.

Thanks for all the suggestion everyone.

James Edward Gray II

 

[Aaron Turner]

Le 7 juin 2009 =E0 03:36, Charles Calvert a =E9crit :


It's far from a small barrier. =A0If the poster uses an invalid address,
he will never know he's supposed to subscribe. =A0Ditto if his email
server or spam filter eats the mailing-list warning message, of if he
doesn't understand it.

Not to forget that, even if he receives the warning, he'll have to
repost his message a second time, while some people on Usenet could
already respond to the first, adding to the confusion...

Frank,

First, let me say I appreciate the contributions that people who post
via NNTP. I honestly don't want this to become an "us vs them"
situation where mailing list users like myself call for tearing down
the gateway. At the same time, it's clear that the spam to the list
is coming via NNTP. Perhaps people who are used to reading many
newsgroups have just become accustomed to the spam and/or have great
anti-spam tools, but many email users have found well run mailing
lists to be one of the few spam-free havens on the internet.
Honestly, if it wasn't for ruby's rapid development which makes lists
like this necessary for staying on top of what's new I probably would
of unsubscribed long ago in disgust.

Simply put, if you don't find my proposed solution workable, then
please provide a better one which has the impact of stopping the spam
(keeping the headers while nice, won't accomplish that).

Another option, is do what the svn-users list does- have moderators
approve messages from email addresses which are not subscribed. Yes,
this means if you're not subscribed, then your post gets delayed.
However, if you have enough mods across different timezones then it's
usually an acceptable delay. If not, you can always subscribe as I
mentioned above. I'll take it one step further and offer up to be a
moderator for a minimum of 1 year.

--=20
Aaron Turner
http://synfin.net/
http://tcpreplay.synfin.net/ - Pcap editing and replay tools for Unix & Win=
dows
Those who would give up essential Liberty, to purchase a little temporary
Safety, deserve neither Liberty nor Safety.
-- Benjamin Franklin

 

[Michael Bruschkewitz]

Anything not on the whitelist gets queued for human moderation. That
should be a fairly small percentage.

Whitelisting bots would help, but...

There are much more of first time posters than spammers.
One of the strengths of Ruby is friendly support provided by community to
noobs.
Although spam is annoying, it is not a big amount of work just to skip those
messages. Clean up your mind and you will not even take notice of spam as
inconvenience.
(Take spam messages as possibility to train your brain...
On the other side, delaying responses to noobs and employing moderators
would waste real resources.
It would be really a bad trade.

My approach was to waste as less resources as possible.

Regards,
Michael B.

 

[Aaron Turner]

Whitelisting bots would help, but...

There are much more of first time posters than spammers.
One of the strengths of Ruby is friendly support provided by community to
noobs.
Although spam is annoying, it is not a big amount of work just to skip th= ose
messages. Clean up your mind and you will not even take notice of spam as
inconvenience.
(Take spam messages as possibility to train your brain...
On the other side, delaying responses to noobs and employing moderators
would waste real resources.
It would be really a bad trade.

My approach was to waste as less resources as possible.

You're ignoring the mailing list- most mail clients can't delete an
entire thread based on a single message via rules. I know Gmail and
Mail.app can't. It also creates a situation ripe for abuse since now
anyone can delete entire threads.

--=20
Aaron Turner
http://synfin.net/
http://tcpreplay.synfin.net/ - Pcap editing and replay tools for Unix & Win=
dows
Those who would give up essential Liberty, to purchase a little temporary
Safety, deserve neither Liberty nor Safety.
-- Benjamin Franklin

 

[Michael Bruschkewitz]

You're ignoring the mailing list- most mail clients can't delete an
entire thread based on a single message via rules. I know Gmail and
Mail.app can't. It also creates a situation ripe for abuse since now
anyone can delete entire threads.

To be honest, I don't know much about mail clients today. Some years ago, I
was eager to check out every mail client, but these were the times when 1
hour of internet access did cost 6 Deutschmark.
Because I don't want to fiddle about installing mail clients on every
machine I use - I now use just the default client for the current OS or just
the first which comes across.
IMHO, Gravity would have had absolutely no problem setting up such a rule
but probably would not be abel to handle mailing list. Forte Agent too. So I
was expecting current mail clients would provide this for mailing lists too.
At least ML-clients should provide the possibility to hide a message based
on topic, so it would be easy for users of such clients to drop the
additional messages.

The abuse issue may be resolved easily, although I don't expect it would
occur frequently. I wrote "some responsible persons could mark the thread" -
the thread-muting rule just needs to depend on sender _and_ topic, or those
persons use a particular name for kill-posts, for example "Aaron Spamkilla
Turner".
So spamkillers could be easily whitelisted.
How to do this could be an one-liner in the content of each
spamkiller-message.
Additional, an [THATS NOT SPAM] message could be added to deal with wrong
spamkiller-messages, maybe somebody marks a thread by mistake.

I wouldn't expect spam problem to be resolved by some free-wheeling ideas in
the night, so I dont expect my idea was perfect - it was just an idea. If it
would be so easy to kill all spam w/o flaws, it would be already done years
before.

Regards,
Michael B.