Aren't RLVR signals typically based off formal systems not natural phenomena? The formal sciences are certainly useful for producing tools used in natural science, but it's not entirely clear they alone are sufficient to associate text to natural (real-world) phenomena. Honestly, the pretraining and RLHF are probably more tied to the world than RLVR, for all that RLVR might be useful (maybe even more useful) for making the model perform better in certain tasks (such as working with formally defined systems).
Yeah, I should have said RL, not RLVR. The point is RL interacts with external world, while autoregressive pretraining is limited to the passive ingestion, and RLHF relies on a model of human preferences that has no access to truth sources besides the limited training data it was built upon.
If you want a more concrete example, then LLMs are also trained on visual data these days, which means they do have access to the world in an important way. This directly contradicts the blogpost's claim that LLMs have
> no way to associate the text vectors they manipulate with real-world phenomena.
Historically, that LLMs were text-only used to be a major argument for why they "lack access to meaning", see the Stochastic Parrot paper and the Octopus paper that it references. But even the authors of those papers have (grudgingly) conceded that the argument no longer holds due to multimodality.
"Repeating what they've seen" could be, depending on their generalization abilities, something like low-n-gram Markov chain, repeating of surface-level speech patterns, repeating reasoning patterns, repeating mind-theory-level patterns, repeating self-correction patterns and so on.
And they fall somewhere between "surface-level speech patterns" and "repeating reasoning patterns" imo, yeah. They do indeed learn higher-order patterns than a Markov chain... but they'll blatantly misapply reasoning patterns and self-contradict confidently, just like you see with human bullshitters who don't understand one bit of what they're doing.
They sound fairly human, until you notice the patterns. They sound like they're thinking, until you pay attention.
> I did crack up seeing a martini glass presented to the camera, then float in place on it's own
It's just a cheap video-generation model doing its thing. I wonder if the orchestrator model noticed this and tried "Create a scene with a martini glass. Make no mistakes".
It works in limited ways (but in the realest-real world nevetheless). Waymo, Wayve, Baidu Apollo, Tesla and others seem to rely on VLAs, VLMs or transformer models in general to do autonomous driving.
Also, none of those companies' cars are really autonomous. Waymo, for example, relies on remote workers that are ready to intervene and suggest a course of action when the AI driver gets stuck:
Non-trivial percentage of people work in the real world until they do something stupid and work no more. It's not real autonomy, not yet.
It's a matter of degree. Sorry, I don't want to watch an hour long video to be told how something that works 99.99% of the time doesn't really work for some contrived definition of "really".
BTW, as is typical with people, remote operators occasionally cause problems.
Fair if you don't want to watch the entire video. The VLA stuff is early in the video but I can summarise it for you (if you trust me to do so): basically none of the current deep learning based approaches to autonomy generalise. Not RL, not transformers, not anything else. They all tend to work fine in environments and tasks within their training set but outside of it, pffft, performance evaporates. That's my summary.
It's really not about 99,9% correct. Rather that's the error you can expect when you deploy such systems to the real world. You can start building an intuition about this if you consider the combinatorial space that these systems must search to find the right action in a given situation. Most of those systems are trained on images i.e. sets of pixels (some also have more sensors like lidar which just blows up the combinatorics even more). Specifically what these systems learn is a function mapping a set of pixels to actuator commands, essentially mapping input images to actions. For an RBG image of 128 x 128, there's (256^3)^(127^2) = 16,777,216 ^ 16,384 = 6.468074e+118369 unique combinations of pixels. Each of those has to be mapped to one of k actuator commands where k is sometimes an integer, sometimes a real. As you can see, that's an insanely high number and there is no way to make a dent in it even with millions of examples of images-to-actions.
Now of course deep learning approaches have shown remarkable, stunning ability in powering through gigantic combinatorial spaces - but all those successful applications are in domains where it is not necessary to map the entire space of inputs to outputs, not even a big chunk of it. Consider the success in board games like chess and Go both of which have huge search spaces, as often pointed out with cardinalities larger than the number of atoms in the universe etc. Yes, but an automated system doesn't have to search that entire space to beat a human: like the joke with the two guys trying to outrun a bear, all the deep learning system need do is search further than a human can search. Or, consider image classification or language generation: even 20% error (which is what you get in real world situations, as opposed to "in the lab" testing on specific datasets like ImageNet) is OK a lot of the time. So what if your image classifier thinks a cat is an impala? Nobody dies. Mostly.
That's not the case in the real world where a system can be expected to continuously find itself in situations that are either subtly or wildly different than the ones in its training set and the stakes are very high. 20% error in that setting doesn't work. If the robot can't deal with a previously unseen situation it can very well destroy itself, and cause an unknown amount of damage besides. The real world is as unforgiving as it is unpredictable; and non-ergodic (i.e. trying stuff at random until you find the thing that works, doesn't work).
Self-driving cars are ultimately made to work with very careful, good, old-fashioned manual engineering, including detailed mapping of an area where the system is restricted etc. That is why e.g. you see Waymo rolling out its cars slowly from one city to the next: if their systems were really autonomous they could just deploy them anywhere a human driver can drive. From New York to Mumbai and from Athens to London. The fact they can't is the strongest evidence that whatever proprietary secret sauce moat they have... doesn't really work.
The "really" I keep dropping in there all the time is suspicious, I know, but that's not entirely my fault. Waymo and all the others claim that their systems are "autonomous". I have to bat that back by saying, no, they're not autonomous. Then there's going to be a lengthy, pedantic discussion about what is "autonomous". So I try to avoid that, in my opinion unproductive, exchange with the "really" hedge. Maybe not such a good idea. I'm open to suggestions to improve the clarity of my language.
> So what if your image classifier thinks a cat is an impala?
For starters, the artificial neural network is the only instrument we have for sufficiently robust image classification. If it doesn't work nothing else will. We can only validate and retrain.
Getting back to self-driving. What if an occupancy network misclassify an impassable surface patch as passable? It would mean a possible accident, so you bias the network towards impassable classification. In this way misclassification is overwhelmingly more likely to cause spurious braking, unneeded swerve, and the like rather that a collision ("brake stabbing" was a recurring problem in Tesla FSD. Today it is largely mitigated.)
> The fact they can't is the strongest evidence that whatever proprietary secret sauce moat they have... doesn't really work.
Well, Waymo has publicly embraced world models, VLMs and end-to-end backpropagation relatively recently (December 9, 2025)[1]. There's not enough data yet to judge how this has influenced their expansion rate. Waymo representatives tout improved generalization (sorry, I can't find the interview. I think it was Dmitri Dolgov.)
> if their systems were really autonomous they could just deploy them anywhere a human driver can drive
As I've said it's a matter of degree. Human-caused road casualties are a familiar and largely inevitable evil. An autonomous vehicle killing a person will (and already has) cause a media storm. They can't ignore a potentially problematic distribution shift caused by deployment in a new area.
You are talking about the perfect system. Real systems are computationally constrained. Distillation necessarily degrades performance and might require retraining for particular areas.
I largely agree with what you say, except the bit where you say I'm talking about the perfect system. I tried hard not to give that impression because that's not what I'm on about. Autonomous systems trained with deep learning (including Deep RL from human demonstrations and so on) are very far from "perfect". They can perform very well in a broad range of scenarios until they encounter an unforeseen situation and their performance falls off a cliff. The problem is that the real world is full of unforeseen situations, full of long-tailed distributions that are near impossible to learn just by training on data, or anyway a reasonable amount of data (even very big data).
Sure, you can "mitigate" whatever edge case you become aware of during operation but when the edge cases are the entire distribution there's always more of them where the ones you mitigated for came from and that's a big problem, it's a very real problem, and we don't yet know how to solve it.
>> Well, Waymo has publicly embraced world models, VLMs and end-to-end backpropagation relatively recently (December 9, 2025)[1]. There's not enough data yet to judge how this has influenced their expansion rate. Waymo representatives tout improved generalization (sorry, I can't find the interview. I think it was Dmitri Dolgov.)
Well, let's wait and see. But my intuition is that Waymo is simply trying the latest trend to see if it will change something. It might even move the needle a bit, who knows.
"VLAs fine-tuned on human demonstrations overfit. Here's how to mitigate it." Er, OK, I guess. But I think that Waymo uses mitigations or a different approach (RL, for example).
Unfortunately the mitigations in the video don't work either.
The bottom line is that none of the techniques we have have been shown to work, to "really" really work, in the real world. If they did, you'd know instantly because suddenly robot cars and robot maids would be absolutely everywhere, not just in a few cities in the States or China. The Chinese in particular are chomping at the bit to flood the market with cheap robot labour. They can't because there's no such thing, yet. Not because there aren't cheap robots: because there's not the AI to drive them.
I know it's hard to get a good feel for where the state of the art is currently, but try this exercise. Back in 2017, if you were watching the video of Atlas doing backflips [1], how far would you think we are from robotic autonomy? Most people would have said "very close" and indeed many did. Yet, 9 years later we still don't have robot maids and robot butlers.
Atlas, of course, was hand-coded, but that doesn't make any difference. Its programming was good enough for backflips (and many other really impressive feats besides; e.g. much more impressive was seeing it balance itself after stepping on a length of pipe, but I can't find the video now). Just because there's a trend now to do the same coding by machine learning rather than by hand, doesn't change anything. The point is we don't know how to create the right program for a robot to act autonomously, independently, freely, and safely, in the real world.
Autonomy is hard and it goes way beyond anything anyone has demonstrated in the last... I'd say 50 years. Robotics has a long history of very impressive results that never turn out as useful as people imagine.
Protecting forests is good and all, but it will not reverse climate change. We'll have to stop all fossil fuel burning and, additionally, sequester around 600 billion tonnes of CO2 (in addition to all the natural carbon sinks) to get back to +1.5C by 2100.
> there is no reason to believe that the way to build it would be found before, say, the sun runs out of hydrogen.
A lot of computational power can be thrown at the problem in this time. So, the problem should admits no shortcuts, no decomposition into simpler problems, no alternative ways to get similar functionality that allow shotrcuts or decomposition. The result should look like a jumble of atoms that somehow produces the required functionality.
I wonder which problems can admit only this kind of a solution.
> A lot of computational power can be thrown at the problem in this time.
We know plenty of actually very simple problems that we would not be able to solve from now till the Sun explodes even if we all of the computers on earth were only doing this. Say, solving a worse case instance of the Knapsack problem for n > 250 items easily fits into this.
If you want to be precise, the complexity class EXP best fits my description (though NP-complete would also be enough). That is, if discovering the required technology to build a self-replicating interstellar probe requires solving an instance of an EXP problem that is not ridiculously tiny, then we would likely never be find out how to build this probe. And no amount of AGI would change this, in this case.
I'm not suggesting that I have any reason for this ridiculously precise scenario to be correct. Just that some technology being allowed by the laws of physics should not be taken to mean that it is actually implementable in any imaginable amount of time, even if we were to assume singularity-style AGI.
> if discovering the required technology to build a self-replicating interstellar probe requires solving an instance of an EXP problem[...]
Ah, I think I know why I felt it was very unlikely. Engineering constraints should be very "unlucky" so that only the solution works. That is, there's no wiggle room that allows an approximate solution (which is usually simpler to find) to work, even if it would be slightly heavier, costlier, or something of the sort.
For such a tight constraint not to be a coincidence, nature itself would have to be able to solve EXP problems. Which might mean that not all is lost.
ETA: ChatGPT has pointed out that many interesting instances of computationally hard problems are still solvable. For example, see SAT-solvers (SAT is NP-complete).
That is in addition to tight constraints the problem instance should have no exploitable structure.
> They're word-association mechanisms with no embodiment and no way to associate the text vectors they manipulate with real-world phenomena.
This is wrong too. RLVR grounds foundational models in reality.
reply