Hacker Newsnew | past | comments | ask | show | jobs | submit | professorsnep's commentslogin

I run a Mediawiki instance for an online community on a fairly cheap box (not a ton of traffic) but had a few instances of AI bots like Amazon's crawling a lot of expensive API pages thousands of times an hour (despite robots.txt preventing those). Turned on Cloudflare's bot blocking and 50% of total traffic instantly went away. Even now, blocked bot requests make up 25% of total requests to the site. Without blocking I would have needed to upgrade quite a bit or play a tiring game of whack a mole blocking any new IP ranges for the dozens of bots.


AI bots are a huge issue for a lot of sites. Just putting intentional DDoS attacks aside, AI scrapers can frequently tip over a site because many of them don't know how to back off. Google is an exception really, their experience with creating GoogleBot as ensured that they are never a problem.

Many of the AI scrapers don't identify themselves, they live on AWS, Azure, Alibaba Cloud, and Tencent Cloud, so you can't really block them and rate limiting also have limited effect as they just jump to new IPs. As a site owner, you can't really contact AWS and ask them to terminate their customers service in order for you to recover.


How do you feel, knowing that some portion of the 25% “detected bot traffic” are actually people in this comment thread?


You don't need buttflare's mistery juice to rate-limit or block bad users.


For personal projects I usually use Astro[1] solely because 90% of the stuff I am making doesn't require anything more than basic HTML/CSS and maybe a couple static components, but I also have the flexibility to add SSR rendering or even more dynamic components like Svelte without making an entirely new project.

[1]: https://docs.astro.build/


Astro is an excellent choice! I've never used it, but always wanted an excuse to. I think it blends the best of both worlds, and the fact that you mix various view libraries is powerful!


Discord has one of the better APIs out there for social media, and is definitely free of charge.


If you click on the three-dot menu, there's a small caret button you have to click to show more of the pill buttons at the top, and one of them is "Cached". It feels like every time I need to use that button it's moved somewhere new.


Thanks. This caret doesn't exist on mobile (at least for me). Hopefully it doesn't get removed elsewhere!


GCE allows one e2.micro instance in their free tier


On the contrary, I'm a user of social media for exactly that purpose. When I use Twitter or Instagram, I'm specifically using it for the natural discoverability it offers. Almost half of the people I follow on Twitter are people I never would have known existed if I didn't see them pop up in my feed one day.

It's why I can never get into stuff like Mastodon or Misskey with how they are today, it's extremely difficult to just naturally find people to follow.


I tend to use the argument, "if we stopped developing technology because it threatened some people's livelihoods, a 'calculator' would still refer to a person."


Seconded - you might even say a 'computer': https://en.wikipedia.org/wiki/Computer_(occupation)


Bit late but:

To The Moon (and the subsequent A Bird Story, Finding Paradise, and Imposter Factory): Wonderful series of games that focuses on narrative. Amazing soundtrack and good recurring cast of characters.

Firewatch: Beautiful game set in the wilderness with pretty much your only interactions being over the radio with your boss who you never meet face to face. Focuses on the relationship with your boss and the theme of escapism.

Heaven's Vault: Great game that focuses on exploration and discovery. It shines with the small, focused stories you'll run into on different planets while piecing together an ancient language and the mysteries of the universe.


ffmpeg too! Been doing a lot with it lately and I barely even feel like I have scratched the service of the ffmpeg argument science.


I really don't like any argument for how to pronounce ".gif", so I pronounce it in as many ways as possible to try to annoy people.

Here's a wonderful video on the subject: https://www.youtube.com/watch?v=bmqy-Sp0txY


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: