somnial's comments

somnial · 2026-08-04T16:43:18

throughput scales superlinearly with number of GPUs when networked well and deployed with wideEP, so 1x won't compare.

also it would be interesting to figure from the DSpark paper whether their numbers are consistent with the GPUs still being H800s, since they never actually say...

somnial · 2026-06-27T16:55:02

this is a blog post from a company that hosts open weights LLMs (https://www.doubleword.ai/). I think its possible it might have been tongue in cheek

somnial · 2026-06-11T15:27:17

https://fergusfinn.com/blog/economics-of-speculative-decodin...

good point tho - plus for Deepseek the shared expert increases the overlap slightly

somnial · 2026-06-08T09:51:18

true, but no reason the predictor model couldn't use linear attention (i.e. mamba, GDN etc) to predict KV caches