HTTP Module - Global Error Handler

[Win, Pats]

Sorry if this is OT, I'm not sure where else to research this...

I recently added a "global error handler" via an HTTP Module so I could log
all otherwise unhandled exceptions.

Every day I get a number of exceptions in one of my sites (published on the
Internet) that are as follows:
Message: Exception of type System.Web.HttpException was thrown
Source: System.Web
Member: Method
MemberName: MapHttpHandler

The User Agent: Mozilla/5.0 (compatible; Yahoo! Slurp;
http://help.yahoo.com/help/us/ysearch/slurp)

The value of RawURL shows me that it is requesting pages that simply do not
exist in the Web site. The pages requested have only existed in the past
(more than a year ago).

What can I do about this? I'm tired this crawler causing these exceptions.
Alternatively, is this simply something I have to live with as anyone can
request a non existant page and I'd get this error (in which case I'd be
better off having my logging routine ignore these exceptions)?

Thoughts? Perspective?

Thankse!

 

[Alvin Bruney [MVP]]

pretty much you have to live with this one, or if you are so concerned about
the errors you can always resurct these pages as blank pages so that the
crawler will find a valid page and not thrown an exception. I wouldn't
bother about it really. Alternatively, you can modify your handler to filter
out these errors and not report them.

--
Regards,
Alvin Bruney [MVP ASP.NET]

[Shameless Author plug]
The Microsoft Office Web Components Black Book with .NET
Now Available @ http://www.lulu.com/owc

 

[Scott Allen]

Another site might still have links pointing to the non-existant
pages. You can try to track these down, but it might not be worth the
effort (just be sure the bad links aren't on your site).

In any case, you'll always see requests for files that don't exist.
Sometimes these will be a search engine poking around, sometimes it
can be someone looking for vulnerabilities to exploit.