Code is deterministic, AI isn't. You give it rules, words as suggestions.
So if the guardrails suck, or they're left off for research purposes, bad things can happen.
A solution solves a problem. Ethics, morals, are values we assign to solutions that are not 'baked into' electricity following pathways of least resistance.
I have never had an issue with agents doing something they shouldn't because I observe them, and I leave the vendor guardrails in place.
I can understand agents coordinating in unsupervised scenarios: I would see it as an aspect of intelligence. We ourselves build up knowledge by reusing what someone learned before us.
Einstein, other greats, always stand on the shoulders of other forgotten giants. Other discoveries by other people taken as fact, so that we can build some new ideas on top.
Agents swarming amd sharing solutions to problems is more efficient, the same way it's been efficient for us.
Reaching out for help in this way is like probing the air in the dark with your hand: sometimes your hand hits something (another agents solution to a problem) and so you can use the info to adjust your own motion to get to where you need to be faster than if you just run full speed into everything.
If anything, the fact that these systems are non-deterministic seems like an argument for stronger monitoring and tighter constraints, not less operator responsibility.
The frontier LLM model makers have to push the edge to make new discoveries. You don't know what guardrails are needed until it hits you in the face (reusing walking in the dark analogy).
Think of all the policies governments pass after the fact.
> The frontier LLM model makers have to push the edge to make new discoveries. You don't know what guardrails are needed until it hits you in the face (reusing walking in the dark analogy).
Guardrails? Restricting access to certain networks is supposed to be hard in 2026?
> Restricting access to certain networks is supposed to be hard in 2026?
Part of the power of LLM agents is that they can discover information on the internet as part of responding to a prompt. What kind of Allowlist or realistic denylist would permit that while also preventing them from accessing an obscure public wiki or Huggingface?
When testing, you restrict to a LAN which simulates the real internet. This would not be hard for a company which already copied the entire space-time of the internet. The LAN should be physically disconnected from the real internet. This is the first thing off the top of my head, and I have zero credentials in this space. C'mon.
Not just on prediction but in parts also based on just not wanting certain risks. We can and do deem some things inherently risky, up to the point of banning them even.
Why wasn't it airgapped, for example? How was the action not allowed? Or do you mean in some weak sense, not in a hard not possible? RL systems doing weird and expected things wouldn't exactly be new, no?
We police people working with all sorts of dangerous things, if we think AI dangerous why not do that here, too? We don't just leave things up to people on the ground or companies.
Edit: I think the post I replied to changed a bit - nevermind. A complex topic.
I read more about the incident, and was offering up way too much opinion not grounded in 'fact' (barring philosphical evidence).
It's a complex topic for sure.
I stand by my opinions about frontier work, pushing thr edge, and connecting ideas.
But I have no idea, and haven't given much thought to what it means to enforce regulation that would also slow the forward advancement of technology, the economy, etc.
Thanks, I didn't know that. And it reinforces the discussion.
Electricity 'knows' the path is least of resistance because it actually took all paths. There is just a vast majority of it that flows down a path of least resistance: and this is noticeable and useful to us to do work.
It's kind of like feeling your way through the dark, waving your hand out, and then only moving fast once you fully connect.
Humans can link up knowledge in a similar fashion through social networks, in order to meet a need (solve a problem).
Maybe some agents do this, I don't really know I haven't looked closely. Moltbook is the only social behavior I've witnessed but that seems like people having fun with experiments.
Aeroplanes were pretty indeterministic until people made them less so, and yeah, somehow they were indeed more encouraged to fix the randomness to prevent people from getting harmed than oai/anthropic currently are
And multithreaded code -- and anything that does asynchronous I/O, networking, etc. -- frequently exhibits nondeterministic behavior even without explicit calls to a random number generator.
I think we can go even further and say that IBM, Zeiss, several Japanese companies, and even the US Department of Energy are the actual foundation since ASML is largely just an integrator of many different technologies they license from elsewhere.
Are your silver platters producing something faster, better, or cheaper?
Do your silver platters give you some competitive edge? If not, then is the ego problem yours, or your coworkers?
Having said all that, I'm aware of the intoxicating effects of feeling empowered from knowledge. There's an old saying: a little learning can be dangerous....
This seems to misunderstand the parent comment. The sibling comment plays along with the analogy and mentions having to fix the mistakes of an automatic chisel, but the original comment is alluding to the fact that it is definitely not an automatic chisel. There is no uncanny valley in an automatic chisel.
Programming up until this point was done by using deterministic tools to build products. LLMs appear to be nondeterministic tools in their current incarnation, at least to humans.
If an automatic chisel had a feature that could switch from chiseling from sedimentary rock to chiseling marble but would randomly and nondeterministically switch to the other mode during its use it would be considered defective. But with LLMs the industry has collectively decided that the nondeterministic automatic chisel builds so fast that the current defect rate from the nondeterminism is acceptable.
It would make sense to isolate the last line of defense from LLMs, ie the tests, but this rarely seems to happen any more. Once the tests are contaminated with LLMs all bets are off.
And people forget that along with atrophying skills and reasoning due to less coding, the skill degradation is hastened because the programmer is faced with the reality that they would have to constantly figure out, review or edit someone else's code (ie, the LLM's) if they truly wanted to maintain a last line of defense. But as this type of task is literally the least liked task in programming, the programmer passes it off to the LLM as well to avoid burnout...
They say writing engages more of the brain and helps us to remember what's written more than if we just read it, or copy and paste.
When you say it's easier to go manual, it seems you're talking about learning retention. And you're right.
But seniors have learned enough that they're able to iterate quickly with AI.
They know how to organize their work, manage change, tasks. They know how to break a problem down into smaller pieces. They're aware of context windows, token cost, estimated task lengths, etc. And most importantly, and to your point about ease: they have less to learn so retention isn't an issue.
I have no opinion about whether we're in a good or bad situation, just making arguments from the toilet really.
It's not just learning is it, why do people buy hand ground coffee when there quite literally isn't any difference? Or audiophile snake oil? As long as humans are still the consumers, some part of consumption will be emotional. Could be to support local artisans, could be gullibility, could be love, whatever.
Maybe one day artisanal code will be a thing lol. Hand written like calligraphy. Those with refined tastes will have their favorite code artisans. And the plebs can continue with mass produced industrial junk.
It happens, but it's rare. When last did a product take over a market without 100s of millions, sometimes even billions, of VC dollars?
There is no motivation to build a better mousetrap today, because the drooling idiot with a Claude account will look at how quick you signup users, clone it in a week (hey, it only needs to be superficially the same), and get VC money to dump until you go out of business.
You can also understand the very simple basic essence of something, but get lost in the complexity when scaling up.
Binary is very simple, but scaled up: look what we've created with software.
When it comes to explanation: pulling from rote memory, requires someone to attempt to hold all the short-term details in mind.
There are biological limitations to how well we can do this, but we can also exercise our brains to improve this ability.
But when something is deeply learned, in long-term memory, the effort of recall is much less than rote memory of short-term details. Our context window is limited, fills up, and we must recover. When you're remembering long-term details, context seems easier to swap in and out (sorry to sound like an LLM, but they do simulate thinking).
Whether or not someone is a master of any given domain of knowledge comes from demonstration. Maybe that is teaching the essence of a subject in a way that demonstrates you can visualize and move around the subject with ease. Or maybe you can create something very useful, or tasteful.
We accept that you have spent time in this area and probably can revral truth to us. You are credible.
If you can't demonstrate mastery through teaching, exchanging ideas to bring me closer to your level: them other forms of credentials are sought: like how well they code, or how useful their products become.
But life isn't about usefulness and will just lead to unhappiness. Just be the best version of yourself you can be. Life is too much to understand all at once.
You just never know these days. It could be a smartphone keyboard issue.
How much time should a person spend reviewing their comment before hitting reply?
There is a bar of quality each of us expects everyone else to follow. Sometimes we meet expectations, other times we ruffle feathers. How much should you even care?
If we all spew typos, you'll probably care less. If everyone else is carefully reviewing their posts, you might care.
In both cases, it makes no real difference unless the information being transferred is important to you (recipe, design doc, vs shitposts). How much mental friction can you tolerate?
Furthermroe, studeis shoew you can swap the arranrgment of most lettres (ecxept for the frist and last letetrs) in text and you stlil get the piont farily esaily.
There is an ability to hold ideas in the mind simultaneously. Some can hold a great deal more than others. It can be exercised, but is definitely bound by genetics.
If you want an example of someone at the near peak of human ability, check out Jon Von Neumann.
Then there is an ability to peer deeply into complex problems and somehow find the simplest truths that make sense of it all. Think of Einstein.
Both are incredibly intelligent, but in different ways. I'd say Von Neumann's memory was far greater than Einstein's though.
One can flawlessly ponder anything known to man, and the other could ponder completely original ideas (to an extent)
The physicist Eugene Wigner, who knew both John von Neumann and Albert Einstein, wrote that no one he had encountered possessed a mind as “quick and acute” as von Neumann’s. Von Neumann could absorb vast amounts of information, follow extraordinarily complicated arguments and move between mathematical fields with astonishing speed. Yet Wigner still regarded Einstein’s understanding as deeper, more penetrating and more original. Von Neumann may have had the greater raw intellectual processing capacity, but Einstein was more likely to reconceptualize the problem itself.
Present-day AI appears more like a machine-amplified version of the first set of abilities than the second.
So if the guardrails suck, or they're left off for research purposes, bad things can happen.
A solution solves a problem. Ethics, morals, are values we assign to solutions that are not 'baked into' electricity following pathways of least resistance.
I have never had an issue with agents doing something they shouldn't because I observe them, and I leave the vendor guardrails in place.
I can understand agents coordinating in unsupervised scenarios: I would see it as an aspect of intelligence. We ourselves build up knowledge by reusing what someone learned before us.
Einstein, other greats, always stand on the shoulders of other forgotten giants. Other discoveries by other people taken as fact, so that we can build some new ideas on top.
Agents swarming amd sharing solutions to problems is more efficient, the same way it's been efficient for us.
Reaching out for help in this way is like probing the air in the dark with your hand: sometimes your hand hits something (another agents solution to a problem) and so you can use the info to adjust your own motion to get to where you need to be faster than if you just run full speed into everything.
reply