You probably shouldn't block AI bots from your website. You should serve them garbage instead.
Today is one of those rare days when I not only remembered I have a blog, but I also turned a fedi thread into a blog post.
A tiny mouse, a hacker, a luddite, a malicious actor. Might be a little unhinged. "Little" is up for interpretation.
Formerly (2017), formerly (2017), formerly (~2013).
There are no hashtags in my bio: bots, bridges, scraping, and all that stuff should be opt in, not opt out. Misbehavers will be served an infinite remix of the Bee Movie script.
You probably shouldn't block AI bots from your website. You should serve them garbage instead.
Today is one of those rare days when I not only remembered I have a blog, but I also turned a fedi thread into a blog post.
Today, my VPS served over 51.5 million requests. Well over 99% of that was AI crawlers and other obnoxious shits.
This is not normal. This is complete and utter bullshit. This is also happening all over the place.
It can be caught, it's not even hard. But we shouldn't need to. This is about three orders of magnitude more requests I'd normally receive, and it's almost entirely useless garbage.
Every single one of you who use GenAI tools, you personally, are complicit in this. You are responsible for these bots hammering the entire internet, you are enabling it.
If you think this price is acceptable, that every single person who hosts anything outside of BigTech walled gardens deserves this relentless assault of thieving robots, then you are a garbage human being.
But it is not too late to change course. You too can look back at the carnage you enabled, and feel remorse. It's okay. We'll forgive you.
You don't need to look at the environment damage LLMs cause - we can have an educated guess (it's very bad). You don't need to look at the unsustainability of it all. All of those are things that we don't directly feel right now.
But look at the damage these things cause to everyone outside of the BigTech walled gardens. That is measurable. These attacks are fact. You can't debate it. You can't justify it.
You, dear enabler of GenAI bullshit, you are responsible for enabling this carnage. Think about that. Feel bad about it, and stop. Today is a great day to do that.