Access Blocked

[MikeB]

When I click the Home button on the navigation bar, I somet9imes get Access Blocked, Then I click the refresh button on the Browser and the page loads again:

 

[revhtree]

We've been getting hammered my AI Bots and we've enacted some rules that you got caught up in it looks like! I just made more adjustments to hopefully help.

 

[revhtree]

Please let me know if you get this blocked page again!

 

[revhtree]

Also let me know if you get a verification page.

 

[MikeB]

We've been getting hammered my AI Bots and we've enacted some rules that you got caught up in it looks like! I just made more adjustments to hopefully help.

I will update this thread if or when I see it again.

 

[MikeB]

Also let me know if you get a verification page.

I got the human verification page like 20 times yesterday, none today.

 

[MikeB]

Just got Access Blocked again, it seems like I can recreate it by going idle for a few minutes, then clicking the home button. If I click the refresh button, I do not get it.

 

[MikeB]

I am getting it consistently still.

 

[MikeB]

Still getting.

 

[MikeB]

Still getting today.

 

[Crabs McJones]

@revhtree

 

[BeanAnimal]

My guess is that the user is not human. Maybe some kind of weird bot trying to infiltrate the fish hobby. The dead giveaway is “Mike”. Way too common and clearly the avatar photo is an AI generated family. Dead giveaway, nobody has smiling kids AND a smiling wife and is named just “Mike”. I bet the B stand for Bot. As in. Mike Bot. The same hacker maybe has a BobB and a TedB. Bob Bot and Ted Bot. Maybe even a JudyB.

 

[MikeB]

My guess is that the user is not human. Maybe some kind of weird bot trying to infiltrate the fish hobby. The dead giveaway is “Mike”. Way to common and clearly the avatar photo is an AI generated family. Dead giveaway, nobody has smiling kids AND a smiling wife.

HAHAHA This was great, made me laugh. Thank you!

 

[MikeB]

@revhtree - You know Software Engineering is what I do, have been for 20 years now. I couldn't help myself and researched last night based on what I can see is running the forum (Community platform by XenForo) and this is what I have found and could maybe help troubleshoot the issue and mitigate it.

What’s likely happening​

1) Bot/Integrity checks + missing cookies/JS on the first hit​

Many WAF/CDNs (especially Cloudflare) will do a quick “is this a browser?” check. If the first request looks “odd” (no cookies yet, privacy extensions, prefetch request, unusual headers), it can trip a block/challenge. After refresh, the browser now has the right cookies/headers and it passes.

Fixes to try

• If Cloudflare: turn off or reduce Browser Integrity Check and/or lower Bot Fight Mode sensitivity.
• Make sure you’re not blocking based on cf.client.bot, “Known Bots only”, or overly strict “Super Bot Fight Mode” settings.
  

2) Rate limiting or “burst” detection on the homepage​

Home (/) often gets hit by:

• browser prefetch
• multiple tabs opening
• redirect chains (/ → /forums/ → /…)
• addons doing extra AJAX calls immediately

One request triggers a rate rule, then the next squeaks through because the burst window resets or the second request hits a cached route.

Fixes to try

• If you have rate limits, loosen them for normal GET traffic to / and XenForo “bootstrap” endpoints.
• Ensure rate limiting is not counting static assets + HTML together.
  

3) A false positive WAF rule triggered by the first request only​

Example: the first request includes a query string (from the menu link, tracking params, or a redirect) like:

• /?_xfToken=...
• /?utm_source=...
• /?something= that matches a rule pattern
  

Refresh loads clean / without the suspicious query string, so it works.

Fixes to try

• Inspect what URL the “Home” link actually points to in the HTML (is it truly / or does it include parameters?).
• Check if any JS adds tracking params on click.
• If Cloudflare/WAF logs show a managed rule hit, add an exception for that specific rule on / (or strip/ignore the offending query params).

4) CDN edge inconsistency
Sometimes one edge node blocks due to a temporary rule/cache state, and refresh routes to a different edge that allows it. This happens more with:

• Anycast networks
• mobile carriers
• users behind large NATs

Fix

• Again: check the WAF/CDN security logs with the event ID/Ray ID.

The fastest way to pinpoint it​

Next time someone gets blocked, have them grab the bottom of the block page. If it’s Cloudflare it will show a Ray ID.

Then you can do:

• Cloudflare Dashboard → Security → Events
• Filter by Ray ID or their IP
• It will tell you exactly: WAF Managed Rule, Rate Limit, Bot, or Custom Rule and the rule name/ID.

Without that, you’re guessing.

Practical, safe mitigations (even before you identify the exact rule)​

If this is happening to legit users:

• Prefer “challenge” over “block” for suspicious traffic (Cloudflare lets you do this per rule).
• Add an exception so GET requests to / don’t get outright blocked (but keep protections on POST/login endpoints).
• Make sure your WAF isn’t blocking requests just because of common forum patterns (cookies, headers, long URLs, etc.).
  

One very specific XenForo angle to check​

If your Home tab routes to something like “Latest posts” or an add-on landing page that makes extra background requests, the WAF may see it as “automation.” Try temporarily pointing Home to a dead-simple route like /forums/ and see if the problem drops.

 

[MikeB]

Here is the latest Cloudflare ID for Access Blocked: Cloudflare Ray ID: 9be629d7281196df

 

[BeanAnimal]

I am not sure how much help ChatGPT is here. This is also what I do for a living and I don't envy the mess or trying to find a balance.

The real issue here does not appear to be configuration as much as the it is simply that the bot traffic is insanely high and as they try to tighten the noose it is creating inconvenient side affects fort actual users. Loosening the filters will certainly help the side effects, but also allows the bots back in.

The "bots" are becoming insanely hard to stop. They are using headless browsers and sophisticated scripting that generate real human traffic looking headers, mouse behaviors, session variables, IP spaces, etc. The entire site is likely being scraped countless times per day with hundreds, if not thousands of separate bot sessions to bypass rate limits, etc. So, "hey bot engine, create 5,000 agents and go to R2R and have each agent scan as many pages as you can before rate limits kick you, when that happens, create 5,000 more agents and pick up where you left off". No imagine 10 different "companies" or "bad actors" doing this at the same time, 24/7

Each of those agents is a different IP, IP block or geo, etc. Each agent has unique session variables and headers and looks to be human. So the challenge is trying to block this crap and not humans. I don't envy the battle.

IOS and Microsoft (etc) secure browsing makes things even worse, as those IPs are now being leveraged by the bots as well, adding a layer of obfuscation that makes filtering even harder because they are obfuscating or fully hiding the original traffic IP and headers.

I would assume at this point Rev and team have engaged both CF and XenForo for help.

 

[MikeB]

I am not sure how much help ChatGPT is here. This is also what I do for a living and I don't envy the mess or trying to find a balance.

The real issue here does not appear to be configuration as much as the it is simply that the bot traffic is insanely high and as they try to tighten the noose it is creating inconvenient side affects fort actual users. Loosening the filters will certainly help the side effects, but also allows the bots back in.

The "bots" are becoming insanely hard to stop. They are using headless browsers and sophisticated scripting that generate real human traffic looking headers, mouse behaviors, session variables, IP spaces, etc. The entire site is likely being scraped countless times per day with hundreds, if not thousands of separate bot sessions to bypass rate limits, etc. So, "hey bot engine, create 5,000 agents and go to R2R and have each agent scan as many pages as you can before rate limits kick you, when that happens, create 5,000 more agents and pick up where you left off". No imagine 10 different "companies" or "bad actors" doing this at the same time, 24/7

Each of those agents is a different IP, IP block or geo, etc. Each agent has unique session variables and headers and looks to be human. So the challenge is trying to block this crap and not humans. I don't envy the battle.

IOS and Microsoft (etc) secure browsing makes things even worse, as those IPs are now being leveraged by the bots as well, adding a layer of obfuscation that makes filtering even harder because they are obfuscating or fully hiding the original traffic IP and headers.

I would assume at this point Rev and team have engaged both CF and XenForo for help.

That all makes sense, and I appreciate the detailed explanation. I definitely wasn’t assuming this was a simple config issue. I know the bot situation has gotten completely out of control. The tradeoff between tightening and usability is tough, especially for a site as large and visible as R2R. The part about headless browsers, residential and mobile IPs, and secure browsing from Apple and Microsoft being used by bots matches what users are experiencing. The fact that a refresh usually lets them back in feels less like a hard block and more like a case where a legitimate user briefly falls on the wrong side of a rule. I understand that loosening filters isn’t a real solution and would just let the bots in again. From my side, the main goal was to give possible clues about what could be happening, no matter what is used to research the issue. With all that being said, there is a solution and I am just trying to help find it

 

[BeanAnimal]

That all makes sense, and I appreciate the detailed explanation. I definitely wasn’t assuming this was a simple config issue. I know the bot situation has gotten completely out of control.

It is insane really. As fast as we detect a pattern and figure out how to block it, a new one emerges. In the end, it is a fight that can't be won I suppose. It is not so much that bots are learning to be "high tech" -- but rather that they do the bare minimum to pass current countermeasures. They just keep upping the game, which adds cost/complexity to the bots. In the end, we will not be able to tell a human from a bot and some type of digital signature for all endpoints and traffic will be required. Say hello to global tracking -- and even then the bad actors will find a way.

Whole situation is unfortunate.

And here I am talking to a bot named Mike!

 

[areefer01]

It is insane really. As fast as we detect a pattern and figure out how to block it, a new one emerges. In the end, it is a fight that can't be won I suppose. It is not so much that bots are learning to be "high tech" -- but rather that they do the bare minimum to pass current countermeasures. They just keep upping the game, which adds cost/complexity to the bots. In the end, we will not be able to tell a human from a bot and some type of digital signature for all endpoints and traffic will be required. Say hello to global tracking -- and even then the bad actors will find a way.

Whole situation is unfortunate.

And here I am talking to a bot named Mike!

We have been dealing with this on another forum, not the bot named Mike, but random 502 errors and increased bot / AI traffic and intermittent navigation slowness. Lots of time spent looking at the logs, making slight changes, one at a time, rechecking logs, looking at patterns, time of day, with server infra mixed in.

Fun times. Hope your day is well.

 

[Daniel@R2R]

Hello and sorry for the delayed response. We changed a few settings in our software that we hope may have fixed this issue