"Latent" means it (the space) is there all the time but usually not exposed or apparent. Latent space is a high-dimensional space that is not legible to humans, so normally reasoning gets constrained to progress in tokens instead so that it can be evaluated.
Models already reason in latent space, but they have to keep encoding-decoding their "thoughts" from/to human interpretable tokens, and notably those tokens are sampled from a distribution. The model cannot output a vector and have that same vector fed back in at the next step, it only sees what token the sampler collapsed its vector into.
It's as if the only way you could think was by writing down a word, erasing all the thoughts from your head, then reading the word you just wrote down and deciding on the next word, etc.
Reasoning purely in latent space means that the model would still produce an output equivalent to tokens but unconstrained e.g. the output could be raw and opaque vectors. A significant downside is that you lose the ability to inspect the reasoning trace. It would also make the reasoning trace potentially larger which has operational issues.
> The model cannot output a vector and have that same vector fed back in at the next step, it only sees what token the sampler collapsed its vector into.
Not completely true: KV is a projection of the activation at each layer's input, so attention heads see (a representation of) all previous tokens' activations at that layer. The hard decision at the LM head doesn't change that.
I think https://arxiv.org/abs/2412.06769 is a good description of the premise of reasoning in latent space, although https://arxiv.org/abs/2604.15726 argues it's already what really happens.