Noting that our server is suffering a denial of service attack - 44 requests PER SECOND.   Dealing with it as best I can from where I am (I'm on a train - in Lithuania!)

 

OK - dealt with ... the heavy black line is today's worker server load.  Looks dramatic - there was (perhaps) a handful of proper requests that may not have been answered but the server did not fall over.



If you wonder how I find out there's a problem like this, the server sends me a message 

Now in Vilnius

 
"Server slow" but perhaps different reasons.  The number of spiders crawling our site has been creeping up - to some extent as expected once we moved from "http" to "https".   As I see individual spiders adding a significant load, I at taking a look at those that come onto my radar and asking (or forcing) some of the more noisy ones not to index our content. 

In terms of web searches, 19 out of 20 in the UK use Google and the Googlebot is allowed Coffee Shop access just as any human guest might be (so no indexing of things like Frequent posters) but others such as Yandex and Petal which are not widely used in the UK are turned away.  Ironically, Google does not put a big load at all on when indexing - I see it, but it's not effecting performance like (for example) the Petal crawler was earlier today.

It's not just web search indexers that crawl sites ... there are a variety of other companies indexing and selling data to their customers or making use of it within products - from plagiarism identifiers to AI feeds and Search Engine Optimisation tool and link reporters.  In some cases they may do us a bit of good, but as we're not driven by sales volume and income here, and we're very much UK based when they tend to index worldwide, that good if it's there tends to be tangential. 

It's a bit of an ongoing game - others play it too and have things like "Captcha" - "are you really human" test which we could do if we need to.   We do have some logic that spots aggressively or characteristically automated visiting without the need for users to tell us which boxes have motorcycles or stairs (or GWR class 158s!) in them from time to time.   Please let me know if you get a "you are not really human" type message more than very rarely!

 
An update on this one - still being well and truly spidered and no single nasty culprit.  I have made a few ongoing adjustements while static here in Mosjoen - but on the road (or should I say rail?) again from tomorrow and I expect to be mobile for another week or so.

Receptionist and worker loads:





Views from just up the road - one looking forward and one looking back





And even here there are things that remind me of England, and of my home town

 
Our server load is going "through the roof".   But - it's a bit of a "careful what you wish for".  One of the hoped for consequences of going to https (six months ago!) was to make the pages more secure-looking and encourage crawlers ... and they are crawling.   I note Google and Alexa spiders have been especially active over the last couple of days and I am keeping an eye in others.  Here are the browsers that called for over 1000 Coffee Shop responses yesterday