Site slow loading

[bluechris]

Yes, every 5-10 seconds its melting and some times it produces 502 error but the strange part is that the 502 error page is coming from cloudlare where in the bottom it says that the site has the problem.
The algorithm that enables cloudflare needs refining for sure but i don't know what to suggest here Steven, sorry because in all the sites i control i am full beens under cloudflare.

You know a trick that i do is to set in cloudflare control panel not to show the catchpa thing when the requests come for a specific country. In my case in my sites it's Greece. You can try to set the UK in there if the attack's in this site case are not from UK. Maybe you can put some more countries there and you take off the script, you go full CL just with some countries passthrough.

[bluechris]

Here is the error that i got just now. A refresh gave me after the 500 nginx error meaning the transition from cloudflare to direct connection has something wrong.

[Steven]

You can thank the ByteSpider and ByteDance crawlers.
At least for the problems in the last hours.

These are aggressive bots by the company that owns Tiktok.
Problem is: they change IPs and don't identify as those bots anymore (visits look like regular chrome users).

Implemented some extra protection against the sources that have recently caused trouble.
Seems better now. I'm holding my breath...

[bluechris]

It's much better now, good job m8.

[DJ Downforce]

What these companies do should be illegal. It is killing forums all over the internet.

[Steven]

What these companies do should be illegal. It is killing forums all over the internet.

Yeah... I couldn't agree more
Further analysis on the problem shows that these bots were firing 80 page requests/s. It has faded off a bit since then, but still ongoing.

When than happens for couple of minutes, I probably don't need to draw you a picture to explain why our server pops up some 500 errors.

[dialtone]

An extreme-ish strategy would be to allow a certain qps for authenticated traffic and much less for non-authed traffic.

If you can get any stats on what distribution you have today, you could set a proper threshold that should still allow forum to be open but not abused by open.

[dialtone]

Also a protection against IPs changing while not having cookies is to use CSRF token for them.

This would allow you to keep track of the qps of the given CSRF or cookie so you can filter when they go too high. And if they have neither cookie nor CSRF you can just redirect them until they persist either sending them in infinite loop.

Other strategies involve honeypots where once you detect it’s a bot you send them to a fake site.

[bluechris]

What these companies do should be illegal. It is killing forums all over the internet.

Yeah... I couldn't agree more
Further analysis on the problem shows that these bots were firing 80 page requests/s. It has faded off a bit since then, but still ongoing.

When than happens for couple of minutes, I probably don't need to draw you a picture to explain why our server pops up some 500 errors.

The best solution is to block anything for an amount of time that does more than 5-10 requests per second (counting in mind many people that are connected with CG-Nat and the ip's are the same). But this is a bit difficult to implement.

[AR3-GP]

It has been intermittent today.

[AR3-GP]

It's happening again.