Alignment just means "this machine operates in ways that align with the intent of its users". It doesn't imply anything about the inscrutability of the machine in question. A gun with a misaligned scope would likewise fail to operate in accord with its user's intent, and likewise with potentially deadly consequences.
Learning ML, there was a high emphasis on the error part of things as most of the course was on minimizing errors. After ChatGPT, there is a weird anthropomorphization going on, where it's all about hallucinations, alignment and what not.
We have something that is statistical in nature so there should never been any expectation of error-free results/actions. The value has always been about discerning trends or the cost of errors being way lower than any good result.
If there is guillotine in the weights of the model, it needs to be properly labeled so you can look it up by name using a database index (or a graph-vector index).
It helps both the bad guys and good guys. Like responsible disclosure in cyber security, we need to have a conversation around it.
> We need to insist on building tech that's explainable by design.
You realize this means insisting on terrible tech that humans can understand right? It essentially caps human progress at some point about 4 years ago.
If you are old and happy with the way things are this might sound like a good idea. It does not to me.
Oh you want tech that helps discover new science instead of parroting existing wisdom?
There is little evidence that the RSI we are discussing is capable of inventing the theory of relativity (or the more advanced equivalent). All we have seen is pattern matching in a much larger space than humans can, with some human provided verification tech.
I would argue that human-AI collaboration with explainable tech has a better chance. Continuous learning can be done in a way that doesn't violate IP or privacy.
All I'm saying is: if you have a choice between two systems with equal power of discovery and one is more understandable than the other, we choose the more understandable one.
Limits of Human comprehension and quest for power are two different motivations that could lead to black box systems that are marketed as semi-explainable.
We need to verify that human comprehension is actually limiting progress before allowing such things and even when we do, do it responsibly on an explainable foundation.
Agreed. I would vote for the "when there is a choice" version of your argument.
I'm not sure it is always possible to verify when human comprehension is the limit though. Most of research is done an the frontier of knowledge where we don't know what we don't know.
There would need to be a great deal of nuance in any law, and nuance in practical terms tend to just mean "loophole." Still, you're right that we should try to build explainable systems where it is possible/reasonable to do so first.
> The problem is that you're not offered a choice. No one is making the "err towards explainability" choice.
I'm not sure that's entirely fair. OpenAI recently discussed this at length in a blog post after some accusations around Astra and the trade-offs. The grown-ups are definitely thinking about it, and making tough choices about the trade-offs.
It's reasonable to debate whether ENOUGH is being done here, and I doubt that even the most rabid AI advocate would argue that more couldn't be done, but everyone in the industry is very much actively thinking about it.
They discuss recent choices they made specifically for that reason.
> The ones who do discuss these ideas are confrontational about LLMs and not effective spokespeople.
Very much this. I'm very open to reasonable debate on the subject, but 8/10 times when I try someone who is rabidly pro/anti jumps in. It turns from a debate amongst reasonable people who reasonably disagree into some kind of political/religious battle of belief systems.
I think part of my problem is that a lot of peoples careers very much depend on them not understanding it and spreading misinformation intentionally.
Ahh yes I remember the bad old days of 4 years ago when everyone decided human progress had enough, and we would have been stuck there forever if it hadn't been for LLMs ... we didn't know how good we had it
So many words. But missing the one that matters the most: Explainability.
AI slowdown is worth it only if it can be made more explainable.
Changing the language we use to discuss it is a good first step.
We need to stop using inside baseball terms like alignment and mechanistic interpretability. Replace them with explainable tech. Graph Databases, Causality, Shared semantic spaces.
Previous writings on the topic (also on LinkedIn, but can't find urls):
You can have a transformer based index built on top of a graph database. Much of the explainability tech (people who prefer "alignment" also prefer "mechanistic interpretability") is reverse engineering the real world graph that's hiding in the weights.
I'm encouraged by this result. It's the primary hypothesis behind latentpedia.org.
Instead of distilling the geometry of a model into a huge knowledge graph, we start from the largest known open source graphs and build it up towards something that resembles this geometry.
Come and join us. Discuss on github.com/latentpedia. We have the basic tech covered. Need more compute, storage and enough business to cover the cost of serving.
Simplicity. It’s a bunch of files, trivial to edit, version and develop against. This is important for things sometimes intended to stay relevant for a long time.
I’d flip the question towards the benefits more complicated solutions give you. One conclusion might be “not enough”.
Surprised that there isn't more discussion of DuckDB source distributions.
Why do we need a source distribution for a well regarded MIT licensed project? Because it's not easy to contribute code to DuckDB if you don't work at DuckLabs. The CI used to take 5 hours for a simple bug fix last I looked (may have improved since I flagged it on social media).
Pygmy-Goose is focused on making agentic workflows faster by splitting the repo, making git worktrees cheap and 5 minute cached CI on 3 major platforms.
Several "Graph on DuckDB" efforts started on DuckDB and ended up reinventing a basic columnar codebase to innovate on because of these reasons. Even though people didn't document why, lack of outside contributor friendly flows is likely one of the reasons.
But circa 2023 decided to write their own. Work continues as LadybugDB. One of my long term goals is to find an integration point with DuckDB's table implementation as the "node table". Conversely at some point DuckDB could implement all the join algorithms and LadybugDB's REL table in their code base.
For now, any talk of Graph on DuckDB is limited to DuckPGQ and the more recent entrant DuckGQL (both of which don't touch the storage layer which is the main reason why LadybugDB exists).
Very different from the comparison on github and the website.
Given that on-disk data structures are similar to SQLite, I expect the competition from other "graph on sqlite" projects when they co-opt the techniques in LatticeDB.
But connect them all together...
reply