Is it ethical to scrape when a site has explicitly blocked bots? I know a fair number of people who run small sites who are already considering closing them down because the bots are relentlessly hammering their sites and driving up hosting costs.

Use cases range from sending 100s of requests per second just to bring a website down to doing a montly request to a municipality's endpoint to get a local dashboard of when trash is picked up. I don't think you can pass a single judgement of automating web requests in general.

> Is it ethical to scrape when a site has explicitly blocked bots

Ethical? ppl get jailed in China for this. "Breaching computer systems" is a felony.

I know for a fact companies in China do this. One paid me directly for work in this space, and in another, I found my own OSS project in their artifact cache while working there.

I've only ever gotten in trouble in the UK

I suppose it's like trying to enforce no picture rule in public.

Why do you feel entitled to navigate to a website you neither own nor pay for via a method the owner expressly forbids?

Why do you feel entitled to dictate what user agent I use, if it’s well-behaved?

Why do your opinions about what a "well-behaved" agent is override the wishes of a site's owner?

Site owners should not have full control over what users do. Their wishes do not apply past certain boundaries.

For a non-AI example I run into occasionally, any wish to stop time-shifting should not be relevant to what I choose to do as a media consumer.

There actually is a well defined demarcation point where the site owners wishes do apply. It's called... A demarc point! Everything that happens past this point is by definition entirely in the control of the admin. Which includes closing the door on whoever they wish. This is a technical reality, not something you can effect with opinions or wanting things more than other people want other things. It might seem contradictory but this is actually the main thing that makes the internet as free as it is.

When we're talking about ethics, I think we should give the owner more than that. If they don't want to be DDoSed we shouldn't say "too bad, the attackers are outside your network, your wishes don't apply, get good".

A request is a request. None of your business what I do with it after that.

If it's just impersonating me to help me better consume the content as if I were the one driving, then it's perfectly ethical, and not even related to a bot ban. Shades of gray and all...

Accessibility, saving time, personal preference. Any number of reasonable things that don’t put undue stress on the host

Because I should be able to choose whatever user agent I want. The owner gets to "forbid" things on his computer, not on mine.

This page is only viewable in Internet Explorer 5.5. Please switch to a supported browser.

Blogging is cheap if the writer values neither their nor your time.

Blogging is also cheap only for writers who don't spend time and money to gain experience, validate information, and maintain relationships in order to writing interesting words.

Blogging only costs a few euro per year if we ignore the resources that go into servers, electricity, IT folks, laptops, and the luxury of time and safety to write.

I maybe shouldn't judge their ideas about film by their choice of illegible background and text colors, but I do.

Definitely keep in mind that right or wrong, these hosts are unusual as far as most commercial services are concerned and it can reveal annoying edge cases in their software.

eBay still in 2026 can't send to subdomains.

Use something like ebay@shipping.example.com and they send to ebay@example.com

I had to check the server logs to find why I wasn't receiving any mail and now need a top level alias just for eBay to handle their broken mail infrastructure.

I'm constantly annoyed how much trouble I have using a .health domain (looking at you T-Mobile). I can't imagine using this many subdomains off a .us.

True. I struggled to get signed up for my COVID vaccine back in 2021 because Walgreens wouldn't accept that my totally valid .rodeo email address could possibly exist.

I still use that domain for most corporate accounts. Currently, my wireless carrier refuses to believe I exist in some of their systems (but not others) because of it.

Fortunately, escalating complaints with large corporations with shitty practices is a hobby of mine.

Every effort doesn't need to address every problem, I think. You're right that individual effort isn't sufficient and many of the permacomputing folks are also activists of varying sorts but I think it's ok to separate concerns as most people understand that there's no single and complete solution.

I think you're missing one of the major reasons people are against "AI": the jerks at the top. When obviously nefarious people are lining their pockets and not bothering to even pretend to care about the people around them, it's no surprise they're hated.

This is a vibe coded app and isn't what I'd want but still, it's interesting to consider what a good implementation of "Figma for TUIs" could be, especially if it avoids the trap of simply treating the console as a crude raster instead of taking advantage of text and keyboards. IMO we don't need WIMP GUI shoved into terminal emulators.

The correct title is "Don't host your own transactional business email, use a transactional email provider."

Hosting your own personal mail isn't for everyone but it's not impossible. It is, however, something you'll need to maintain so be ready to pick up a new hobby.

I saw this headline and excitedly clicked through expecting to find an attempt to make a modal drawing canvas made using lessons from and the ethos of vim. It isn't that. It's a vim plugin for writers. Which is fine, just not what I hoped for.