Biology has been trying to grey-goo the world for billions of years, but it turns out the world is not something so trivial.
He is pushing back against the folks with pure CS backgrounds who think that computers are all there is. Its a form of magical thinking unique to programmers who live in a world where speaking the right words to a machine is enough to impart your will on the world. Believe that strongly enough, and you fall into the trap of thinking that a sufficiently smart entity could speak the words "let there be light" and it would be so.
The author is pointing out that speaking the words is insufficient. To end humanity there must be an execution phase. The author is correct to point out that acquiring superhuman intelligence is not some guarantee that you will have or obtain the resources necessary to make that happen, in the same way that genius generals still lose to ordinary ones, and the best-laid plans are oft to go awry.
There certainly are risks, but 10% risk of extinction in 10 years is not one of them.
Spoken like someone with a somewhat limited understanding of what it actually takes to calculate the fitness function of arbitrary organisms in an arbitrary environments.
Yeah it takes long because natural selection is a blind process starting from absolute zero and is only concerned with fitness, AI is a human guided process, has all of human progress as a starting point and is concerned with any number of goals we can create reward signals for, there's almost nothing you can extrapolate from one to the other.
Build a flowsheet based steady state chemical process simulator, then use it to simulate and optimize a full scale oil refinery.
1) Building a solver engine that works at this scale is not a trivial problem, and the successful ones rely more on heuristics than some categorically different solution approach.
2) Defining the engineering equations relevant to this task is relies on understanding what level of fidelity is required to answer the questions people ask of steady state process models.
3) Knowing the thermophysical properties of chemicals and crude oils is possible from the open literature, but the information is diffuse and different correlations are applicable in different situations.
4) Creating a GUI which converts a flowsheet into matrix math is non-trivial, although a sequential modular approach is a bit easier.
5) Defining large scale models in such a way that they solve robustly is as much art as science. For example, completely closed recycle loops like refrigeration systems are a nightmare for solvers, so it is often better to define them in an open-loop way.
6) Optimization involves knowing the relevant commodity prices, but more importantly how to define the constraints on the model so it doesn't just say to produce infinite gasoline.
7) Troubleshooting the inevitable convergence failures is also as much art as science. There are a large number of diagnostic techniques, but fundamentally you need to be able to relate what is happening during the solver iterations with the intent of your model because more often than not the problem is that you've asserted something impossible, redundant, or irrelevant.
Arguments about super-intelligent AI have all the hallmarks of the philosophical "proofs of god's existence": they start from seemingly innocuous premises, and conclude in an apparently airtight way that a god exists.
There is a tendency among rational minded folks to look at such proofs of god and exclaim "you can't DO that" then turn right around and do the same thing about superintelligent AI.
The key message I want to deliver to people is:
A) Philosophy is not such a trivial thing that you can just wander in, "be a smart guy" and find flaws in established philosophical arguments.
B) AI has real theological implications, and everyone is tiptoeing around it. More than one public intellectuals are trying to smuggle their own metaphysical positions into the public consciousness via discussion of AI.
The most obvious one is the "god of the gaps" problem now applied to intelligence. If you look at things like Aquinas' Summa Theologiae [1] you'll see that there are explicit assertions that things like rationality transcend corporeal nature. As such the more examples we have of AI behaving in rational ways, we will be forced to conclude that either rationality does not transcend nature (removing the last remaining gap where we can invoke a supernatural explanation) OR that we have created artificial souls.
I think theologians will hang on for a while saying "what AI is doing isn't really rationality". But eventually the theologians are going to face a reckoning: what AI is doing will look so much like rationality that they will be required to answer the question "what specifically differentiates the two?" and then they will be stuck. Neuroscience does not understand how our brains produce rationality nor do our AI scientists understand what algorithms their neural nets are implementing post-training. Therefore, any specific claim the theologians make runs the risk of immediately getting invalidated by some scientific discovery on either side (theologians these days are generally smart enough to avoid putting themselves in that position.)
Interesting stuff. "Soul" is hard to define. A thought experiment I do is this: Does a human have a soul? Sure. A rock? No. What about a single-cell organism? Nah. An insect? No. But what about a mouse? There is a kind of gradient forming, and it's very hard to say where the soul begins and where it ends. It's a very unweildy word.
Peter Thiel famously went to the Vatican and started giving lectures about AI and the Antichrist. Pretty rich for a guy who built his billionaire fortune on an AI-powered spying company.
People have. That's a common objection to golf courses.
The difference here is that data centers are both economically and environmentally extractive. People in the community, or at least in the surrounding area, will play golf there. Data centers are purely extractive, and that's why they have become so galvanizing. It's transferring probably the only real asset these communities have left, their open spaces, directly to the elite.
What is the exact value proposition of AI as it exists right now for the average person who lives where these things are being built? They don't even get jobs. With golf, they can enjoy the outdoors and escape a little bit.
I'm not entirely certain, but I think a data center might just provide more temporary and permanent jobs than a golf course, and with likely higher mean and median wages.
The golf park isn't even the best natural-adjacent use of the space.
A regular park or a trail will service a significantly larger portion of the population than a private property that transfers wealth from the community to a single person or corporation.
>I'm not entirely certain, but I think a data center might just provide more temporary and permanent jobs than a golf course, and with likely higher mean and median wages.
Well, you're right to be uncertain. After they're built, once the crews that built them move on to the next one, they don't even bother to keep the lights on. They might hire private security guards to patrol the exterior, but that's hardly an infusion of capital into the community.
> The golf park isn't even the best natural-adjacent use of the space.
> A regular park or a trail will service a significantly larger portion of the population than a private property that transfers wealth from the community to a single person or corporation.
Ok. Then they can decide to build a nature trail instead. The local community should make the decision and not be coerced because someone from the coasts thinks that they're a minor annoyance that needs to be lectured on the future of humanity or whatever lessbad rational-du-jore is being peddled.
> going to go out on a limb here and guess that net-net, a datacenter provides more benefits to a broader spectrum of society than a golf course does.
Is that true locally though? Having a golf course nearby your home is better than the golf course just existing somewhere, at least if you play golf, but that is decidedly not true of data centers. You might benefit from at least some of them existing, but derive very little benefit from having one in your backyard and a lot of potential downsides.
> Having a golf course nearby your home is better than the golf course just existing somewhere, at least if you play golf, but that is decidedly not true of data centers.
Isn't that the standard NIMBY argument, which ensures nothing ever gets built? Who wants a power plant, or dump near their house? Even if there's some vague local benefit (eg. a dump that benefits the county), you could always make the argument to put it on the other side of the county or whatever.
> I've not seen a single politician calling for statewide bans on golf courses.
I've not seen a single state where anyone is trying to build a ton of new golf courses. There are around 140 being built in the whole country, and I think the most in any one state is under a dozen in Florida.
There are more than an order of magnitude more data centers under construction with some states having more data centers under constructions than the whole country has golf courses under construction.
I bet if you tried to build 2-300 golf courses at the same time in one state it would become an issue for state level politicians.
I seem to recall that data centers require orders of magnitude less water than a golf course too if you want to compare apples to apples.
Look, just say what you mean. If you think "AI is BAD" just say that, don't try to couch it in some disingenuous argument about water usage or climate change.
I did say what I meant. You overlooked the context.
The person I was replying to was trying to make inferences from the fact that data centers have become a statewide issue in some states but golf courses have not.
My point is that because golf courses are being built at a much lower rate than data centers they remain just local issues in most places. Try to build 200 golf courses at the same time in some state and I bet you'll see that get statewide attention.
No one would give a shit about two hundred golf courses being constructed in Texas.
Even with the massive data center build out, existing golf courses today use more water than all projected data centers globally are expected to use in 2030.
Also, the "pollution" from data centers is just "major construction project temporarily disrupts water table" not "facility is pumping random chemicals into the water".
That is to say, nothing about next door neighbors to a major construction project getting mud out of their well for a little while is unique to datacenters.
I did not say that golf courses are pollution free, but there is many degrees of magnitude difference between the mere possibility of illnesses becoming more common around golf courses and data centres that have to pay millions in fines as they are not following the rules.
I feel like you're constructing straw men arguments here despite the very obvious polluting behaviour of datacentres.
Go look up articles from mid to late 2010s. There's where you will find frustration about golf courses, during the time where water restrictions were being put on residential lawns. Do not conflate the more recent, more successful campaign of anti-data-center sentiment with the non-existence of a past campaign against golf courses. It existed, it just petered out.
Look, you're talking out of both sides of your mouth. Either water consumption is a problem today, or it is not. If it is a problem, then golf courses are today a better target for people who want to solve the water problem than data centers. If water consumption is not a problem, then the data center's water consumption is not a problem.
if we limit the number of data centers so that it cannot exceed number of golf courses in any zipcode in america than this is a solid argument, otherwise it is strawman
I am not doing any such thing. You are pulling whataboutisms left and right, and I'm trying to get across that the lack of a successful campaign does not prove the lack of sentiment. You need to carefully pick which political battles to take and, more importantly, when to take them.
If you think the current data center installations have problems you want to be addressed, you don't also bring up golf course water usage, farmland water rights, residential lawns, factory runoff, and climate change effects on rainwater. You'd just be distracting from the conversation.
No, but someone yelling about data center water usage while clearly not giving a shit about the things that use way more water quickly make it clear that they’re either very ignorant, or this isn’t actually about water.
The left has long been opposed to excessively watered golf courses too. I could probably find another piece of The Economist calling them foolish for that as well, if I searched.
The "leftist" take is "we should seize the means of computation" not "we should protect intellectual property rights!"
The hate for data centers is a liberal thing, not a "leftist" take. Although, being against golf courses should be supported by everyone. Golf courses suck.
True but not campaigning against golf courses make anti data center campaigns look hypocritical.
My personal criticism of data centers is mostly regulatory failure. The tech companies are choosing random ass states and stealing land from poor farmers instead of the traditional data center states like Virginia mostly because of tax breaks. The local government is just as if not more responsible for the issues with data centers. It's easy however to just point fingers at big tech instead of he underlying market failures in a lot of the states. The traditional data center states almost always have a supply of clean energy and water with a proper regulatory framework.
Might as well say you're a hypocrite for calling these people hypocrites but not others, and unless you criticize all things at the same time, which is impossible, it's all magically invalid.
If you really cared about people calling things out, you would be focusing on other topics than datacenters because people call all kinds of things out all the time. That fact you only care about data centers is telling.
No, it’s everyone watched the More Perfect Union video, saw sludge coming out of their faucets and low decibel hums ruining sleep, and decided to fight when peasants aren’t allowed to fight. Between this and Flock Cameras, tech isn’t making themselves the noble savior on a white horse and being a good neighbor when moving in.
Clearly, it's both. Surely tech has a bad image and Flock does whatever they've done, but just because "one data center caused sludge" nowhere near equals "all data centres cause sludge", which is the implication from a lot of the anti-tech influencers. It's like saying, "data centers consume vast amounts of water" which, of course, is sometimes true but also sometimes not true at all. Data centres don't create many ongoing jobs, true but so what? Don't build anything that doesn't provide jobs?
People like my dad with no technical knowledge at all read an article in some rage-bait newspaper and it must be true (like foreigners took all your jobs, and the left-wing government wants to tax all your money and give it to scroungers).
If people are not taught how to ask "how often is this true" or "how much is this true" or "will this always be true", then they will fear whatever they are told to fear.
> Many to many are born. For the superfluous ones the state was devised... The state, I call it, where all are poison-drinkers, the good and the bad: the state, where all lose themselves, the good and the bad: the state, where the slow suicide of all—is called “life.”
But why would we poison our society by trying to protect these superfluous ones? They may not deserve death, but the more we eliminate the consequences of irresponsible actions, the more irresponsibly people can be and survive, thrive, and even rise to the throne.
This is an unhinged, eugenics-adjacent take that only reflects on your lack of imagination/undeserved faith in your own intellect.
Everyone thinks they are the exception. You are not. You have eaten things you did not vet. You haven’t been poisoned by bathroom products (yet). You are alive because of protections like this.
No. These aren’t two sides of the same coin. People by default deserve to live even if they make bad calls.
The only reasonable way to live your life is under the assumption that people don’t deserve to die for being wrong about something they buy and eat. This is not something that should have to be spelled out man. The “darwin awards” argument is cruel and sociopathic if you take it beyond a morbid joke and instead actually believe people deserve to die over something as simple as not having perfect knowledge over what they consume.
But the article explicitly says that people did know what they were consuming. They bought poison honey, they got poison honey, they ate poison honey, and they got poisoned.
Now you can certainly argue we should make that impossible, but I am explicitly arguing that doing so makes society worse. I have made no claims one way or the other about what they deserve, because that is a religious claim, not an objective one.
reply