[OT] Map of email origins to Python list

[Claire McLister]

Sure, please do make it available, or at least the geolocation
component
anyway. I'm sure you'll get lots of useful comments from the many
clever
and experienced folk who frequent this group.

I've made the script available on our downloads page at:

http://www.zeesource.net/downloads/e2i

Let me know if you have any trouble accessing it. Sorry to disappoint,
but we actually use a commercial service to convert from IP to
location. There are several of them available on the net, and we picked
this one after some testing. I think it has some location problems
(like putting Mountain View south of San Jose in California), but
otherwise seemed to be one of the better ones available.

Don't be aggrieved at the negative comment you've received: I think
what
you're doing is fascinating.
Thanks.

I look forward to the map with updated precision

Me too. Please let me know how we should modify the script.

 

[Alan Kennedy]

[Claire McLister]

I've made the script available on our downloads page at:

http://www.zeesource.net/downloads/e2i

[Alan Kennedy]
[Claire McLister]

Me too. Please let me know how we should modify the script.

Having examined your script, I'm not entirely sure what your input
source is, so I'm assuming it's an mbox file of the archives from
python-list, e.g. as appears on this page

http://mail.python.org/pipermail/python-list/

or at this URL

http://mail.python.org/pipermail/python-list/2005-November.txt

Those messages are the email versions, so all of the NNTP headers, e.g.
NNTP-Posting-Host, will have been dropped. You will need these in order
to get the geographic location of posts that have been made through NNTP.

In order to be able to get those headers, you need somehow to get the
NNTP originals of messages that originated on UseNet. You can see an
example of the format, i.e. your message to which I am replying, at this URL

http://groups.google.com/group/comp.lang.python/msg/56e3baabcd4498f2?dmode=source

The NNTP-Posting-Host for that message is '194.109.207.14', which
reverses to 'bag.python.org', which is presumably the machine that
gatewayed the message from python-list onto comp.lang.python.

So there are a couple of different approaches

1. Get an archive of the UseNet postings to comp.lang.python (anybody
know where?)
A: messages sent through email will have the NNTP-Posting-Host as
a machine at python.org, so fall back to your original algorithm for
those messages
B: messages sent through UseNet, or a web gateway to same, will have an
NNTP-Posting-Host elsewhere than python.org, so do your geo-lookup
on that IP address.

2. Get the python-list archive
A: Figure out which messages came through the python.org NNTP gateway
(not sure offhand if this is possible). Automate a query to Google
groups to find the NNTP-Posting-Host (using a URL like the one
above). Requires being able to map the python-list message-id to the
google groups message-id. Do your geo-lookup on that
NNTP-Posting-Host value
B: Use your original algorithm for messages sent through email.

2A message-id lookup should be achievable through the advanced google
groups search, at this URL

http://groups.google.com/advanced_search?q=&

See the "Lookup the message with message ID" at the bottom.

Sorry I don't have time to supply code for any of this. Perhaps some one
can add more details, or better still some code?

 

[Michael]

We've been working with Google Maps, and have created a web service to
map origins of emails to a group. As a trial, we've developed a map of
emails to this group at:

http://www.zeesource.net/maps/map.do?group=668

This represents emails sent to the group since October 27.

Would like to hear what you think of it.
------------------------------

<sigh>
Another sleepless camera pointed at the fishbowl that is my online life.

I guess it's a great way to find where there might be Python jobs to be
found, or at least kindred souls (or dissident Python posters in countries
where Internet activity is closely monitored...)

To me, it's either cool in a creepy sort of way, or creepy in a cool sort
of way.

As long as it gets my location WRONG, I'm happy.

:-|


Michael.

 

[Tom Anderson]

We've been working with Google Maps, and have created a web service to map
origins of emails to a group.

Top stuff! The misses are, if anything, more interesting than the hits!

I, apparently, am in Norwich. I have been to Norwich a few times, and, in
fact, i think i've walked along the very street where i'm supposedly
located, but i don't think i've ever posted news from there. I read this
group via an SSH connection from my office (in north central London) or
home (in north-east inner London), or elsewhere, to a shell account on
urchin.earth.li, a machine colocated at an ISP (in Docklands, London),
which peers at three POPs (probably also in Docklands, London).

The domain earth.li, in which the machine lives, however, was registered
by someone who gives their address as being in Norwich, which i guess is
where that comes from.

What it doesn't explain is why Sion Arrowsmith is also down as being in
Norwich - i don't know Sion from Eve, but based on the fact that she's a
chiark.greenend.org.uk user, i'd guess she's in Cambridge. Now, chiark has
no links to Norwich that i can see, but it is also colocated at the same
ISP as urchin (chiark and urchin are sort of mirror images of each other
in many ways) - is this a case of 'Norwich by association'?

tom