More than that, running hundreds of conversation streams at once is essentially the same cost as running a single conversation. And then you add on the secondary benefit of having the GPUs running nearly all the time rather than mostly idle...
Local inference makes sense for speciality needs, or very small models. But if your model is bug enough to span GPUs its excessively wasteful to hoard those GPUs for yourself without piggybacking hundreds of other conversations on top of all that memory bandwidth and matrix multiplies.
The alternative is to switch to far more efficient models and have aggressive optimization of efficiency as part of the process of improvement.
There are at least two resources here that have a Pareto optimal front: time and energy. And effort spent to change the shape of that may pay off more than efforts spent purely on improving intelligence and agency.
I'm sure this will stop all the public opposition to data centers! /s
This type of thing will just enrage the opposition more, give them far more fuel, and attract far more people into opposing data centers.
The ruling regime is used to controlling a subservient political group that always, 100% falls in line to marching orders. Now that they control the entire country, they either need to have oppressed tho populace enough that all objection gets arrested or is subject to immediate political violence. But that's not going to be possible with how unpopular they have made themselves.
One thing to note about the first graph: nobody is doing as well as the first part on tool calling, and it's not close.
This might be the fault of the other providers, but it's probably just something slightly different that the first party does with the model inference program than anybody else, and that's not sure to weights it's due to vLLM twiddling (or whatever) and probably becuase the first part actually uses their own customized inference program rather than the standard methods that all the third party providers use. This isn't nefarious, it's just the challenge of these sorts of stochastic systems.
Having been in science for decades now, and seen benchmarking across many different fields, these results are completely expected for me. LLM serving is not mechanical, it's hard to get right and has lots of unknown footguns. Even something as extreme as scrambling a matrix will still likely get results that are nearly as good as normal, and if there's a bug deep in vLLM or the tensors metadata that results in that, then it's going to be pretty hard to find unless you're an active researcher with knowledge of the particular model you're running inference on. I kind of doubt that's happening here, but maybe!
In the scientific literature, when benchmarking methods, everybody's own method performs best in their own hands. Some attribute it to researchers gaming benchmarking for publication purposes, but I think it's just what we see here: the people who made a method are just the best at using it because they know all the quirks and use it best.
Programmers are not used to thinking with that nuance, and jump to conclusions about lying about quantizations, etc., but this is really just an unavoidable part of AI/ML methods: when things aren't perfect they're still pretty good and it's going to take the model creator to truly debug it. At least until the open weights ecosystem gets a lot better at ensuring reproducibility, and model cards are nowhere detailed enough for that to happen yet.
This matches what I found experimentally with gpt-oss-20b during OpenAI's red-teaming challenge. After moving from hosted inference to running the model myself on rented H100s via vast.ai, I saw the model refuse the same kinds of prompts at noticeably different rates depending on the inference stack — differences of roughly 5–10 percentage points with otherwise identical experimental parameters and seeds.
So I very much agree that this isn't necessarily about providers secretly changing the weights. For reproducible work, the serving stack - engine, version, hardware, configuration, and probably more - really belongs in the methodology alongside the model itself.
I wrote up the results here: “In AI Sweet Harmony” (arXiv:2510.01259).
Of course, it's impossible to know for sure what was LLM processed or not, but some of your posts (like this one) have been getting classified that way. (And if this was a false positive, I apologize!)
LLM ops is not trivial. The systems for running inference are very complex, and running across multiple GPUs and nodes adds tons more complexity. And when LLMs are run incorrectly, they still work, just not at optimal performance. Even noticing that something is wrong is not trivial, and finding the problem is far far harder.
So I guess I shouldn't be surprised at all to see these benchmarks, but still I am!
There are such huge economies of scale with batched inference that it's clear this sort of service will continue, but it has a lot of growing up to do. Even AWS Bedrock has a Claude that feels different to me, but I haven't had a chance to do actual benchmarks that would show that.
I hope the data center's waste heat can be used for some form of district heating! Finland has pretty great municipal infrastructure, but I don't know enough about its heating systems.
The new planned datacenters aren't near major population centers, so potential for district heating might be limited. Eyeballing from the map one of them is about has about 25 km from Oulu (pop 200k), and another is 30 km from Kajaani (30k). But that's about it.
Edit: I was wrong, the locations do have small (5-10 MW) district heating networks.
The city of Kajaani's district heating (northern Finland) is already completely covered by datacenter waste heat from CSC datacenters (and maybe some other ones).
there are some papers about communal gardens heated by data centers... since they don't produce as much there, hope it also paves the way for developing indoor agriculture
For those regions green house and barn heating is a major cost (to the point that Norway for example subsidizes barn heating for dairy farmers), while pretty much all the energy consumed by a datacenter goes straight to heat without any meaningful work (in thermodynamic sense) done, so it sounds like a good match.
Microsoft's data centre in Espoo is used for district heating.
>When the Microsoft datacenter areas are finalized and the waste heat offtake is working fully, the heat from fossil-free fuels will cover 40% of all district heating in Espoo.
One can imagine the saunas the guys will have there :)
On the main point - there are persistent leaks/rumors that one of the main points of the Trump's envoys (Kushner and Witkoff) to Putin is building data centers in Karelia ( the Russian side of Finland border) - the same reasons of cheap electricity and plentiful of cool water.
Nuclear is the most expensive form of clean energy. It is likely to bankrupt at least one company.
When Finland contracted with France to build their latest reactor, they wisely had a fixed price contract where the French company would pay for const overruns. At groundbreaking in 2005, they planned for a 2009 completion. In 2016 it bankrupted Areva, with the French government owned EDG saving parts of the company crucial to the nation. It wasn't completed u tip 2023:
When the US last decided to build nuclear, Westinghouse AP1000s, two in South Carolina and two in Georgia, it resulted in the bankruptcy on Westinghouse, putting all of Toshiba, its owner, at risk:
The two South Carolina reactors were abandoned at the cost of a substantial fraction of the build cost, in the billions, and utility executives went to jail for deception about it. Georgia finished its two reactors, but only by creating some of the most expensive electricity out there today.
Storage and renewables are cheaper, faster to deploy, and work really really really well in Texas and Arizona. Despite nearly free natural gas in Texas due its state as a by-product of oil extraction, most of the new electricity grid assets in Texas are wind, storage, and solar, and that's independent third party investors making those choice in the grid to make money, as Texas has perhaps the "freest" electricity market in the United States when it comes to who is allowed to build grid assets and make money off them.
If the far north places have hydro that's already built and not being used, that makes sense for data centers. But new nuclear is not a serious technology, it exists mostly based on vibes.
It seems to me the main problem is we build so little of it that we don't have the people, skills, supply chains etc to do it well. China generally builds it's reactors on schedule and within budget.
China is only building a token amount of nuclear compared to storage wind and solar. It makes a lot of sense for a large nation like China to waste a bit of money on nuclear power just to provide support for the core national security interest: nuclear weapons.
On time, yes, because China is great at construction. On budget, almost certainly, but we have very little visibility into true costs. China's labor costs are getting far higher as it becomes richer, and there's a good chance that it will reach a state of higher economic development where its costs for nuclear mirror those high costs in present day France.
A one party state where the plants are owned, insured, backstopped and cleaned up for free by the government doesn’t have the same economics as free market builds.
And even with all that their total share of nuclear is declining y/&y and projected to continue to decline. Because it’s still too expensive even for a autocracy
> Unfortunately, the project was cancelled for moronic political reasons.
From Wiki[1], I see:
> In March 2022, in light of the Russian invasion of Ukraine, the Finnish Government declared that a rapid independence from Russian energy production is desired, and that the construction of the plant should not continue. Mika Lintilä, Finland's Minister of Economic Affairs, declared that he would not issue a construction permit.
1. Consistent stable long-term energy
2. Controlled by Finland, to minimize all kinds of risks (war foreign influence, cut sea cables, worsening of relationships with other countries, market conditions and economic situation changing.
So what did Finnish politicians decided to do? To cancel the project for nonsense political reasons (let's see how much Finland will be forced to pay in damages), therefore INCREASING the reliance on foreign sources or unstable local sources.
This is the stupidity on the level of "Germany decided to run down all the nuclear powers for the sake of environment". With disastrous results!
"We don't want to allow Russians to build anything" is a statement full of feelings, but not a lot of sense, if one thinks even a little bit deeper.
Finnish oldest Loviisa nuclear station is Soviet based and this has not caused any problems. The nuclear fuel is also being provided by Russia, despite the war.
I think it is reasonable that the Finnish gov't cancelled the permit to build a Russian design reactor while NATO is deeply involved with the largest conflict with Russia since its founding in 1949. However, you raise some good points in your reply. They should have switched to a different design, such as EU, US, Canada, or Korea. All have modern, well-proven reactor designs.
You should read on the history of Finnish nuclear reactors. The disaster of Olkiluoto showed that French can't be trusted.
USA (Westinghouse) nuclesr development was/is in the gutter as well, plus there are risks giving it to the USA (the biggest hybrid and non-hybrid war wager in the world).
Then read about the history of Pyhäjoki/Hanhikivi. Finns really did try to get anyone else, RosAtom was not in the initial running. Unfortunately one after another each candidate was eliminated/wighdrew. In the end there were zero. It is at that point that RosAtom came in and saved Finnish nuclear future. They wanted to do that because the project in Finland (with a very strict Stuk government nuclear safety agency) would be a really good reference for the rest of Western Europe. Check the number of nuclear stations built by RosAtom. They really are very professional and a leader in the field.
What most people are missing in this conversation is that they think that building a Russian nuclear station in Finland would increase the amount of Finnish energy dependency on Russia. In reality it is the exact opposite, the dependency would decrease. There would be more energy produced in Finland, by Finns, instead of importing it via sea cables (can be cut) from other countries, including Russia (yes, Russia!). In case of Putin's malevolence, what would be easier for Putin, to stop providing energy via a cable, or to disrupt the work of the Finnish nuclear station?
The answer is clear. That's my point, the decision makes sense only if you don't think about it. With a little depth, you realize how incredibly emotional and stupid the decision was.
When your supplier is threatening to invade and kill your people, and has critical control over supply of parts of your reactor, and regularly commits acts of nuclear terrorism such as droning the newly completed sarcophagus around Chornobyl:
Well, avoiding business deals with that neighbor is the only rational action. Building a Russian nuclear reactor as one of their neighbors is suicidally stupid. At least go to South Korea instead.
The US is also finding out that when it threatens its neighbors and allies that it will lose out on crucial industrial deals, such as Canada and Europe moving away from US military contractors.
There would not be a critical control over the reactor, just the opposite. Finland would control everything and could have even bought fuel elsewhere.
Loviisa station is Soviet-built, has been working amazingly well and till this day has been supplied with the fuel from Russia, despite the war and all.
Nuclear station being a big target on its own is the only negative, among a pile of positives. That's why:
1. Mini-reactors should be developed
2. No, even nowadays, Putin has not really attacked nuclear stations for real in Ukraine. It's been a big no-no. All the rest of the energy infra - yes, but not nuclear stations.
3. If the real war starts with nuclear stations being bombed, it will not matter who build the station, Finland is going to be completely fucked, considering how much of the energy is nuclear. Taking this argument into a logical conclusion, Finland (or any other country) should not have any nuclear reactors on the chance that they will be attacked. This is a silly level of risk management, a very shallow and blind view.
This is Finland. A third of the country is above the Arctic circle. In the winter the sun barely gets above the horizon. Nuclear makes a lot of sense there.
Agreed, it makes sense for Finland, one of the very few places it does. But in the future of global energy supply, nuclear will be a far less serious player than it was in the 20th century.
Local inference makes sense for speciality needs, or very small models. But if your model is bug enough to span GPUs its excessively wasteful to hoard those GPUs for yourself without piggybacking hundreds of other conversations on top of all that memory bandwidth and matrix multiplies.
reply