Gateway is malfunctioning

[F. Senault]

Le 2 décembre 2006 à 17:37, Ross Bamford a écrit :

I have a feeling this is what is, or at least was, happening - I wasn't
going to say anything (since it was prior to December first) but I
noticed that posts with attachments never made it across.

That's not systematic.

ruby-talk 226884, for example, doesn't appear to have made it to the
newsgroup.

If you have an e-mail version of the message, could you send it to me
with all headers as a (g)zip file ? I'll run it manually through the
filters and look at what could cause the rejection. (Maybe the base64
encoding in this one, passing for a binary post.)

Fred

 

[James Edward Gray II]

Le 2 d=E9cembre 2006 =E0 17:10, James Edward Gray II a =E9crit :


Yes, but it won't be multipart/alternative (which means different
representations of the same data, nearly universally HTML + plain =20
text),
more likely multipart/mixed (simply different parts).

For instance, with a quick search, I found this one :

http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/227642

Which has been gated correctly, as it has :

Content-Type: multipart/mixed; boundary=3D"sm4nu43k4a2Rpi4c"

Am I understanding right Fred that your filters do allow multipart/=20
mixed?

I ask because I cannot find that the following message was moved to =20
Usenet:

http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/226884

But here is the Content-Type header from that message:

Content-Type: multipart/mixed; boundary=3D"=3D-nN7cXnGaqHsTLxonszcP"

James Edward Gray II

 

[F. Senault]

Le 2 décembre 2006 à 18:05, James Edward Gray II a écrit :

On Dec 2, 2006, at 10:55 AM, F. Senault wrote:

Am I understanding right Fred that your filters do allow multipart/
mixed?

They should.

I ask because I cannot find that the following message was moved to
Usenet:

http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/226884

But here is the Content-Type header from that message:

Content-Type: multipart/mixed; boundary="=-nN7cXnGaqHsTLxonszcP"

If you have the complete message, please forward it to me. I'll be able
to give more precise answers if I can run it manually through the
filters.

Fred

 

[F. Senault]

Le 2 décembre 2006 à 18:05, James Edward Gray II a écrit :

Am I understanding right Fred that your filters do allow multipart/
mixed?

I ask because I cannot find that the following message was moved to
Usenet:

Okay, it was interpreted as binary (probably the base 64 part). I've
lifted that condition (I moved clr to the binaries allowed groups) but
messages like this will probably be plagued by a bad distribution, maybe
raising even more difficult to troubleshoot problems.

Fred

 

[James Edward Gray II]

Le 2 d=E9cembre 2006 =E0 18:05, James Edward Gray II a =E9crit :


Okay, it was interpreted as binary (probably the base 64 part). I've
lifted that condition (I moved clr to the binaries allowed groups) but
messages like this will probably be plagued by a bad distribution, =20
maybe
raising even more difficult to troubleshoot problems.

We appreciate all your efforts. Thank you Fred!

James Edward Gray II=

 

[James Edward Gray II]

Did the old gateway address this somehow?

No. The new code does everything the old one did and more. The main
difference is that I can understand the code now.

Before I took over the Gateway though, it did operate through a
different NNTP host. I don't know what that host allowed, but the
evidence suggests it may have been pretty accepting.

James Edward Gray II

 

[James Edward Gray II]

Assuming that the base64 encoding was done to cope with UTF8 in the
source, is there no way we can improve the distribution situation?

Here's what I said to Fred on the subject:

"I'm guessing we could get radical and read incoming emails with an
email library, then use that to compose a sensical Usenet post. That
way we could pull the text section of a multipart/alternative
message. Attachments are trickier and I guess we would have to
inline them. That would work for simple Ruby scripts, but it gets
more complicated when someone posts something like a zipped archive
file.

There would still be situations it couldn't handle, but maybe we
could reduce them. This sure sounds like work though.

I'm considering making the Gateway code public now that I have
rewritten it. Maybe this will encourage enterprising souls to hack
on it a bit for features like this."

We just need to remember that we are joining two worlds with
different rules here.

James Edward Gray II

 

[F. Senault]

Le 2 décembre 2006 à 22:11, James Edward Gray II a écrit :

Here's what I said to Fred on the subject:

"I'm guessing we could get radical and read incoming emails with an
email library, then use that to compose a sensical Usenet post. That
way we could pull the text section of a multipart/alternative
message. Attachments are trickier and I guess we would have to
inline them. That would work for simple Ruby scripts, but it gets
more complicated when someone posts something like a zipped archive
file.

This path leads to the Dark Side (tm) : content mangling of a message
(even if it's transcoding). It's usually considered very bad mojo to
touch anything else than the headers...

(At least, I'll have someone to blame if I typo something in a post...
)

There would still be situations it couldn't handle, but maybe we
could reduce them. This sure sounds like work though.

Yep. I'm not sure there are any good fast libraries to treat MIME, and
transfer-encodings out there ?

I'm considering making the Gateway code public now that I have
rewritten it. Maybe this will encourage enterprising souls to hack
on it a bit for features like this."

That's definitely a good idea.

We just need to remember that we are joining two worlds with
different rules here.

What he said.

James Edward Gray II

Fred

 

[James Edward Gray II]

Le 2 d=E9cembre 2006 =E0 22:11, James Edward Gray II a =E9crit :


That's definitely a good idea.

I've published half of the code now:

http://blog.grayproductions.net/articles/2006/12/05/mail_to_news-rb

James Edward Gray II