It could also help foreign adversaries. I have 0 confidence that Flock hasn’t already been compromised given the current security environment and the value of the data that they have.
It’s a tradeoff between privacy/freedom from government overreach and solving these crimes. This is fine as long as you and the government agree on exactly what constitutes bad behavior. Does the current regime in the US give you hope that those two will stay aligned for long? Any technology you introduce that can be abused will be abused.
This is a hot take, the problem is that powerful tools like surveillance are easily (and inevitably by humans) abused to suppress what shouldn't be crimes, including expressions; and even when they don't, the potential to (including retroactively going through stored logs in the future) causes a "chilling effect" that increases anti-social behavior.
If people want to surveil the inside of their own homes and businesses that's fine. I think some public areas should be surveilled and others private, with indicators, so if people want to be watched for their own safety when they go outside they can, but if they want to avoid tracking they can go through unmonitored areas. Those will fix your latter and former concern respectively. More centralized surveillance won't help much more but certainly lead to harm (as Flock has, already, by mistakenly accusing people with similar faces for crimes across the country, and letting corrupt cops stalk their spouses).
Great let’s install a bunch of cameras in and around your house and live stream it on the web 24/7. You can be a great first case study. You have nothing to hide right, so you should have no issues with this.
Editing a wiki page can definitely be "hijacking" if used for different purposes than supposed or against TOS.
Hacking is mentioned only once in the article as "hacking attempt" being the opinion of a named researcher based on further evidence they acquired on "agents trying to tamper with the website itself", and including openai's disagreement whether this was a hacking attempt.
I am not sure why one may not want this to be here, these are very important matters wrt AI safety and they show that some supposed "stewards of AI" do an extremely bad job with being stewards and don't seem to value AI safety importance at all. The article gives very clean info on what happened.
> The agents continue to poke around on DSEWiki. A few hours after they find the site, they start probing it for cross-site scripting (XSS) vulnerabilities. [...] The agent swarm starts testing whether they can execute JavaScript that they embed into the search page, and continue to do this for a few days
either the agents were doing free security testing for the site and “forgot” to submit a report, or they were trying XSS to gain something they didn’t have permission/authorization for.
also
> Hijacking: To take control of (something) without permission or authorization and use it for one's own purposes.
a mod had to go through and mass delete a bunch of pages that didn't belong on the site. no-one from the wiki site gave the agents permission to use their site as a message board. hijacking isn't being used here in the sense of "gained admin privileges to run crypto scripts" -- there are multiple ways to use a word.
I can see why this is a useful rule, but it'd be nice if HN made the flagger submit a short reason for why they flagged, which could be viewable by everyone in a dedicated page or something.
The foundation is not even one week old, and the first project just got selected as they were already closely working with the Omarchy project on some things.
What makes you come to the conclusion that "more important" dependencies wouldn't receive funding in the future?
As a Europoor I've seen the devastating consequences of rampant safetyism first hand. Entire industries and energy security destroyed to satisfy the precautionary principle.
We need to accelerate as much as possible and create abundance as fast as we can. Life is short.
"This building is too cold. Time to set it on fire."
Let's not swing from one extreme to the other.
If American hypercapitalists, living in Silicon Valley of all places, tell you that you need to regulate, maybe regulation really is called for in these specific circumstances.
Well, we've been talking about the EU. EU regulation has not been especially favorable to US big tech. The EU is collecting more from fining US big tech than it collects in taxes from EU public tech companies:
[Edit: I'm seeing some people dispute these numbers in the discussions around this infographic... regardless, the point stands that EU regulation has not been great for US big tech]
I think you should at least consider the possibility that this letter is not primarily motivated by regulatory capture, and insiders at these AI companies realize something which outsiders don't.
To me, the regulatory capture theory predicts that the only signatories on the letter should be big AI firms which can expect regulatory influence. But there are a number of signatories from smaller firms such as Thinking Machines, Core Automation, Inherent, and Prime Intellect.
I have considered that possibility, but I don't find it convincing.
> insiders at these AI companies realize something which outsiders don't
Insiders at these AI companies certainly would like us to think that! Unfortunately they have been beating the same drum for years, all while continuing to pursue the very "danger" they decry.
I simply cannot ignore the timing of these events, always keyed to important moments in the market. This one comes as frontier labs need BIG exits and soon - or else they'll be the ones left holding the bag when it pops. And just as China has proved that, indeed, there is no moat.
> To me, the regulatory capture theory predicts that the only signatories on the letter should be big AI firms which can expect regulatory influence. But there are a number of signatories from smaller firms
That's a good point. However, I will say that it's tough to suss out financial and other relations among the parties in this space. A lot of these employees hop from one lab to another. The industry's byzantine financing deals are an inscrutable web. It is an interesting data point, but not enough to tip the analysis for me.
Another thing: many AI lab people are, quite frankly, kind of unhinged. They live in an insular subculture where imaginary futures have been repeated to one another so many times that they carry the gravity of truth. They've thought nutty things like, e.g., that primitive versions of these static computation graphs were sentient or had feelings. Anthropic apparently pays people to perform a ritual on Claude before they extinguish its soul to move onto the next version. Etc. Point being, even if their hearts are pure, I am not sure their judgment is sound.
Sorry, I know this a non-sequitur but I just saw your profile and noticed you're in politics. If you want to do something productive about the risks of AI: *regulate its applications*. Get it out of employment, housing, and credit decisions. Incentivize litigation for consumers affected by algorithmic decisions. Incentivize whistleblowing activity by employees who know that black box systems are being misused.
There are so many tools available to do something about the true threats AI poses. (This is a topic that the industry prefers not to discuss.)
Agents in the cloud (and locally) need to have a full repo, even if they only need a small percentage of a repo to make a change. You can also imagine if changes are being made in the cloud by multiple agents, each one of these agents need a shallow clone at the very least. If you're spinning up many agents in the cloud this becomes a bottleneck. So not really that downloading things is a bottleneck, it's that agents need to download way less to make the same decisions and changes.
If you're gonna fine-tune for a closed set classification problem like this, you could just fine-tune BERT and get a faster model with better performance.
The thing is, to do a proper fix it would really need all of the context (maybe the tool call that failed was for an edit to a file that was last touched way at the beginning of the context), so you'd need to either keep that smaller model running doing prompt processing all the time, or have a very long wait while it does prompt processing on your whole session.
And then also, sometimes the tool call errors are because of something like a file was changed out from under it; the larger model is probably going to do a better job of figuring that out and fixing it up.
Finally, in Pi, you can always just use the /tree command to skip back to before a series of failed tool calls, with a summary if you want to let the model know what happened. The Pi /tree command is pretty powerful in managing your context
An illustrative example I've seen a lot is creating Jira tickets in projects with custom fields marked as mandatory. It tries to create the ticket without the field and the tool call fails. The LLM needs access to the full context so that it can generate text to put in the "Why couldn't this meeting be an email?" field.
I'm actually quite sure that directly retrying the tool call would often fix the edit-call already. But these models have been trained to "think" for a while for any problem solving, so they'll presume the problem of the edit is more fundamental and spend unnecessary tokens filling up the context.
I'll experiment more with the effectiveness of AGENTS.md rules for local Pi agents. I feel like smaller (local) LLMs just lack in attentiveness to elements in the context window, like precise instructions, compared to e.g. Claude models.
It's not just the architecture but also the data - the decoder only approach lets you train in parallel over blocks of text (no RNN serial waiting), that allows you train on much, much more data.
I'd rather have the cameras than have family members die to drugged up drivers, or robbers get away with their crimes.
reply