Googlebot searching for ".../bin/en.jsp"

[Rabin Yasharzadehe]

Good point, thank you.


*-- Rabin*


On Tue, May 20, 2014 at 10:23 AM, shimi <linux-il at shimi.net> wrote:

> On Tue, May 20, 2014 at 10:15 AM, Rabin Yasharzadehe <rabin at rabin.io>wrote:
>
>> I have installed fail2ban on one of my servers, and created a set of
>> rules to block some request the (from my point of view) looks like probing
>> attempts.
>>
>> One of the rules is to block on site, any request to *.jsp which i don't
>> have on this server.
>>
>> Today i got a mail about a blocked IP which belong to Google (based on
>> whois).
>> # whois 66.249.79.57
>>
>> can any one tell me, why Googlebot will search for something i don't have
>> any reference to in my site?
>>
>>
> The ".." does look strange, I think Googlebot always use Canonical URLs in
> general...
>
> Just a note: The fact that there's no reference in your site (if that is
> indeed a fact...) - does NOT say that there isn't such a reference in any
> other site on the Internet...
>
> Note that Google also has GCE - I would assume the netblocks for GCE would
> also say "Google"... maybe it's a crawler which is not really Googlebot,
> rather than an impersonator running through GCE...
>
> -- Shimi
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.huji.ac.il/pipermail/linux-il/attachments/20140520/8e919904/attachment.html>