As an open source maintainer, I can chime in on this.
With TxtAI, I've seen a large uptick in PRs (https://github.com/neuml/txtai/pulls?q=is%3Apr+is%3Aclosed+s...). While the extreme verbosity of Claude messages and commits is very annoying (plus the constant defending itself on why it's a bug), I do think it's a positive that more people are enabled.
It does require reviewing the PRs. Some can be tricky just like a human. For example I did merge this PR (https://github.com/neuml/txtai/pull/1136) and it would have completely broke search. But a human could also do that.
From an open source standpoint, I say the more the better. You just have to be willing to do the work to review and no not just having AI agents to review what the AI agents are submitting. There still needs to be a human in the loop, if you care about quality.
Numbers are for the Gigatoken API, but compatibility mode just means eating a bunch of Python overhead (creating lists, reading strings to bytes). You can expect a modest ~200-300x speedup with compatibility mode depending on how you use it.
I can add some benchmarks for compatibility mode in the future. I have a little more juice to squeeze out of the Python interop though, so not quite ready for it yet.
Interesting project. Happy to see someone who shares an interest in tiny vector embeddings models. I've worked on tiny (1MB - 4MB, 250K - 950K parameters) embeddings models called BERT Hash https://huggingface.co/blog/NeuML/bert-hash-embeddings
Local models are great for a lot of things past just software development. We need to move towards solving other real world problems vs just building software. I've been focused on that with TxtAI (https://github.com/neuml/txtai) for 6 years now.
With TxtAI, I've seen a large uptick in PRs (https://github.com/neuml/txtai/pulls?q=is%3Apr+is%3Aclosed+s...). While the extreme verbosity of Claude messages and commits is very annoying (plus the constant defending itself on why it's a bug), I do think it's a positive that more people are enabled.
It does require reviewing the PRs. Some can be tricky just like a human. For example I did merge this PR (https://github.com/neuml/txtai/pull/1136) and it would have completely broke search. But a human could also do that.
From an open source standpoint, I say the more the better. You just have to be willing to do the work to review and no not just having AI agents to review what the AI agents are submitting. There still needs to be a human in the loop, if you care about quality.