Googlebot searching for ".../bin/en.jsp"

[shimi]

On Tue, May 20, 2014 at 10:15 AM, Rabin Yasharzadehe <rabin at rabin.io> wrote:

> I have installed fail2ban on one of my servers, and created a set of rules
> to block some request the (from my point of view) looks like probing
> attempts.
>
> One of the rules is to block on site, any request to *.jsp which i don't
> have on this server.
>
> Today i got a mail about a blocked IP which belong to Google (based on
> whois).
> # whois 66.249.79.57
>
> can any one tell me, why Googlebot will search for something i don't have
> any reference to in my site?
>
>
The ".." does look strange, I think Googlebot always use Canonical URLs in
general...

Just a note: The fact that there's no reference in your site (if that is
indeed a fact...) - does NOT say that there isn't such a reference in any
other site on the Internet...

Note that Google also has GCE - I would assume the netblocks for GCE would
also say "Google"... maybe it's a crawler which is not really Googlebot,
rather than an impersonator running through GCE...

-- Shimi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.huji.ac.il/pipermail/linux-il/attachments/20140520/e23624b4/attachment.html>