Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

same here.

there are 2 approaches they use that make developing bots very difficult.

1. they detect device input. if there is no mouse movement, while the website is being loaded, they will consider it's a bot.

2. they detect the order of page visiting. A human visitor will not enumerate all paths, instead, they follow certain patterns. This is detectable with their machine learning model.

I really don't have a solution for #2



I think the solution is "hybrid" scraping with a human driving the clicks and the scraper passively collecting the data.

If you record, you can probably teach AI to emulate.


I love this. I might try it. It doesn't scale, but that's okay for my project.




Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: