I think people might prefer the lower idle power consumption from lpddr over the better bandwidth in hbm in battery powered stuff. That said right now the price is definitely preventing us from finding out.
That hasn't been true since early HBM2 days, before the controllers standardized on power/voltage management and did things like leave them in P0 to ship on time
LPDDR/DDR/GDDR generally still win over HBM when there’s no data being transferred. HBM is primarily better in watts/byte transferred. Consumer electronics spend most of their time idle.
Right, but HBM idle is significantly worse than LPDDR idle or even DDR idle for that matter. That matters a lot precisely because the device is idle most of the time. Your idle power draw dominates.
This is literally a thread where people are saying “but why laptops no HBM”. It’s a perfectly cromulent response that meets the question where it is. It might be mildly a more defensible question for normal plugged in desktops but that would require a massive ecosystem change since people who build those traditionally really like their DIMMs and less buying a CPU that has non-swappable RAM built in, not to mention there’s no sustainable market for the higher cost and low volume product.
If LPDDR6 PIM ever becomes a thing then the advantage of HBM will shrink. I'm not saying LPDDR6 PIM will make HBM obsolete in the datacenter or where it is currently shining, I'm saying that large volumes have their own charm and it is more likely for LPDDR6 PIM to be in your laptop or smartphone or SBC than HBM.
LPDDR6 PIM would primarily help the low end and mid range accelerator market. E.g. embedded models running on SBCs can be up to 1 GiB in size with acceptable performance, small models at 8 GiB become really easy to run at reasonable speeds on a smartphone and PIM enabled laptops or mini PCs make it possible to run 32 GiB models locally without compromise.
Of course this also assumes that the associated accelerators (NPUs) will catch up too, but the general point is that you won't need a 5090 or a 4090 anymore.
The real issue is that housing is heavily underweighted in the cpi basket. How many people do you know that are only spending 12.9% of their after tax take home on housing, water and fuel? Only people with paid off mortgages.
if you bought a nvidia h100 at wholesale prices (around $25k) and ran it 24/7 at commercial electric rates (lets say $0.1 per kwh), then it would take you over 40 years to spend the purchase price of the gpu in electricity. Maybe bump it down to 20 for data center cooling.
I don't think the cost of the ai is close to converging to the price of power yet. Right now its mostly the price of hardware and data center space minus subsidies.
Actually since the qe to fix 2008 foreign investors/governments cut down on bond buying. Treasury has foreign bond holdings up about 2T since 2014 and current account deficits sum to around 7T.
Nowadays we're mostly selling companies and real estate. (though there was a nice jump in bond buying a few years when rates went up)
Or maybe a lot of it can't deliver returns in line with the current price. The world realizes that and stops sending us stuff for those investments. Then we're stuck with limited manufacturing, high inflation and relatively low ownership of our existing high value exports.
If the pie gets bigger too slowly then I think we still lose.
1800 on the h100s is with 2/4 sparsity, it’s half of that without. Not sure if the tpu number is doing that too, but I don’t think 2/4 is used that heavily so I probably would compare without it.
reply