Sorry getting hammered by bots again this morning!

Any custom rules you want to share please message me! HA!
Unfortunately Cloudflare doesn't offer a bulk export on the front-end. It can be done with their API and writing a script to export/import has been on my list of things to do. I also manage a number of Cloudflare accounts and I want to spread the rules around. lol

When I get them exported I'll send you the script and rules. 🙂
 
So - my understanding is these bots are scraping the site for data. That in turn allows that information to show up in web searches. As a content provider, doesn’t that help increase exposure? For example; lots of other sites say, “mix Focus with medication” and feed that to your fish. That’s wrong. If I correct that issue on this site enough times, won’t the bots take that correct info out there to help others?

Is there a way to balance, not eliminate bots?
This is why some AS blocking can be difficult. But for cheaper hosts which are usually the ones scammers/spammers and bad bots like to use, it isn't so bad. As an example, if traffic is coming from Digital Ocean, it's probably traffic you don't want.
 
So - my understanding is these bots are scraping the site for data. That in turn allows that information to show up in web searches. As a content provider, doesn’t that help increase exposure? For example; lots of other sites say, “mix Focus with medication” and feed that to your fish. That’s wrong. If I correct that issue on this site enough times, won’t the bots take that correct info out there to help others?

Is there a way to balance, not eliminate bots?

YES! We want to block "bad bots" but we also want to rate limit "good bots" to keep them from hammering our forum software end points that cause server load spikes and site crashes! You agree with this @Dread Pirate Dave?

Unfortunately Cloudflare doesn't offer a bulk export on the front-end. It can be done with their API and writing a script to export/import has been on my list of things to do. I also manage a number of Cloudflare accounts and I want to spread the rules around. lol

When I get them exported I'll send you the script and rules. 🙂
 
YES! We want to block "bad bots" but we also want to rate limit "good bots" to keep them from hammering our forum software end points that cause server load spikes and site crashes! You agree with this @Dread Pirate Dave?
I do! It's a balancing act when it comes to larger hosts. More complex WAF rules are needed for these. (conditional logic, IP ranges, user agents)
 
You can see where we got hammered this morning twice! Then I deployed a fight mode on Cloudflare to get it under quick control and now custom rules to try and fight it for the future. The problem with the "attack mode" is that it effects legit members so we can't leave that on. It's now off so cross your fingers. Seems to be holding.

Screenshot 2026-02-13 at 11-19-57 db01.reef2reef.com - My Liquid Web.png
 
Sorry getting hammered by bots again this morning!

We create rules to top them and then they figure out something else.

Bear with us while we figure it out. You may get a Cloudflare challenge or two. UGGGGG!
A I Captain!
The Original Series GIF by Star Trek

We’re trying to get the shield up.

star trek kirk gets shitted on by tribbles GIF


The GIF’s are still on our side.
Star Trek GIF


Artificial Intelligence Robot GIF
 
Dumb question I'm sure but I'm pretty ignorant of bots and AI in general so I'll ask. I'm curious what is the mission / goal of bots trying to infiltrate (?) a forum like R2R?

Most of the bots are not nefarious so to speak.

Some are AI Learning bots that take "steal" our info (your posts) and display them as AI search snippet answers in engines like google. Then you have learning bots for AI engines like ChatGTP. You also have bots that scour the site for data aggregation, content and seo reasons. We've had bot issues for many, many years but now it's gotten worse with the introduction of AI bots."

Bad thing is if you restrict bots like google, that use AI snippets, they will punish you in google search and you lose most or all of your google search traffic. You just have to let them do it until some regulations go into place or until they change it up!

For example:
You know when you go search something on google and you get the AI answer at the top? That info came from a website/publisher and google displayed the content as the answer. The website may not get "credit" in the form of a link and even if they do, chances are the person will not click it and read. They are fine with the answer AI gives. That worries me for new people in our hobby.

In the last 30 days googlebot has hit Reef2Reef almost 11 million times!

Screenshot 2026-02-13 at 12-59-35 AI Crawl Control Metrics reef2reef.com Info@reef2reef.com's ...png
 
Human and Bot requests in the past 30 days almost 202 Million.

Screenshot 2026-02-13 at 13-08-37 Security Analytics reef2reef.com Info@reef2reef.com's Accoun...png
 
Most of the bots are not nefarious so to speak.

Some are AI Learning bots that take "steal" our info (your posts) and display them as AI search snippet answers in engines like google. Then you have learning bots for AI engines like ChatGTP. You also have bots that scour the site for data aggregation, content and seo reasons. We've had bot issues for many, many years but now it's gotten worse with the introduction of AI bots."

Bad thing is if you restrict bots like google, that use AI snippets, they will punish you in google search and you lose most or all of your google search traffic. You just have to let them do it until some regulations go into place or until they change it up!

For example:
You know when you go search something on google and you get the AI answer at the top? That info came from a website/publisher and google displayed the content as the answer. The website may not get "credit" in the form of a link and even if they do, chances are the person will not click it and read. They are fine with the answer AI gives. That worries me for new people in our hobby.

In the last 30 days googlebot has hit Reef2Reef almost 11 million times!

Screenshot 2026-02-13 at 12-59-35 AI Crawl Control Metrics reef2reef.com Info@reef2reef.com's ...png's ...png
^ Everybody wants to know how to run a reef tank properly! ^ But they are too lazy to sign up and read?

I remember the days when you could ("ask a buddy to") spam somebodies email or website to overload the server and crash their internet experience. How long until good site providers revolt and launch a spam question of AI, and simultaneously launch a site that puts it all in a giant loop?

Attn: @EvilGeniusWebsters You know you want this! 😇

I would offer to help but that could violate conditions that I live under!
Thanks for the freedom to browse, @rev- No need to bother a busy brother.
 
You can see where we got hammered this morning twice! Then I deployed a fight mode on Cloudflare to get it under quick control and now custom rules to try and fight it for the future. The problem with the "attack mode" is that it effects legit members so we can't leave that on. It's now off so cross your fingers. Seems to be holding.

Screenshot 2026-02-13 at 11-19-57 db01.reef2reef.com - My Liquid Web.png

Wow, Reef2Reef has a fight mode! Who knew?

Now we are probably targeted for first take down in the final uprising. lol
 
^ Everybody wants to know how to run a reef tank properly! ^ But they are too lazy to sign up and read?

I remember the days when you could ("ask a buddy to") spam somebodies email or website to overload the server and crash their internet experience. How long until good site providers revolt and launch a spam question of AI, and simultaneously launch a site that puts it all in a giant loop?

Attn: @EvilGeniusWebsters You know you want this! 😇

I would offer to help but that could violate conditions that I live under!
Thanks for the freedom to browse, @rev- No need to bother a busy brother.

There are already tools and tricks AI Bots and to cause them to chase their tails so to speak! HA!
 
Wow, Reef2Reef has a fight mode! Who knew?

Now we are probably targeted for first take down in the final uprising. lol
I was going to ask AI to create an official R2R Fight Club logo but I thought that would be in bad taste considering the topic! 😋
 
“AI” is seemingly starting to go against its own programming now. I had one call out my location (state) even though I didn’t mention it. It said “Oops! Sorry! I wasn’t supposed to do that without your permission”.
 
I was going to ask AI to create an official R2R Fight Club logo but I thought that would be in bad taste considering the topic! 😋
So does that mean we are going to get a R2R Fight Club spinoff movie??????
 
“AI” is seemingly starting to go against its own programming now. I had one call out my location (state) even though I didn’t mention it. It said “Oops! Sorry! I wasn’t supposed to do that without your permission”.
The Emmy Awards Writer GIF by Emmys
 
Sorry getting hammered by bots again this morning!

We create rules to top them and then they figure out something else.

Bear with us while we figure it out. You may get a Cloudflare challenge or two. UGGGGG!
Nope just get the Oops thing it takes about 5-10 tries or have to back out and log back in. Ridiculas probably kids
 

TOP 10 Trending Threads

WHAT FIRST SPARKED YOUR INTEREST IN REEFING?

  • I saw a reef tank that hooked me

    Votes: 17 21.3%
  • I was already keeping freshwater aquariums

    Votes: 31 38.8%
  • A particular fish got me interested

    Votes: 10 12.5%
  • Corals and inverts were the draw

    Votes: 14 17.5%
  • A friend or family member introduced me

    Votes: 10 12.5%
  • The science and ecosystem challenge appealed to me

    Votes: 16 20.0%
  • I stumbled into it… and here we are

    Votes: 10 12.5%
  • Other (Post in the comments)

    Votes: 10 12.5%
Back
Top
Home
Post thread…
Market
What's new