Looking at our website stats, I noticed that our traffic has increased remarkably, from around 2000 to 3000 hits per month to around 2000 hits per day, with 20,000 in this month so far. They mainly appear to come from Brazil, Vietnam or unknown.
Any ideas what might be going on? Has our disused Victorian graveyard become an internet sensation without my knowing?
Thanks
Norman
Alas it's the stupid badly behaved web robots that are doing it.
If you look in the stats for your site you'll see that /calendar/view has been viewed 23,697 times in October (so far!).
What happens is that the bad robots ignore the robots.txt and the "nofollow" rules set in the links, and basically browse every possible day in the calendar backwards and forwards ad infinitum. This is of no value to them of course, and yet they do it anyway.
(well behaved robots that follow the rules end up just indexing the events in your calendar but not going to any of the many thousands of possible calendar pages)
I've tried to put things in place to block this behaviour, but ultimately since they look essentially like normal browsers I can't always block it all, because that would also block real users.
Unfortunately this sort of makes the web stats rather less useful than they would otherwise be. :-(
I'll have a think to see if there are any other tricks I'm missing for blocking these robots, but to some extent it seems that this is just the "new normal". Sometimes the Internet really is just a bit of a pain...
Thanks
Joe