Calling prompt injection "not malware" because LLM behavior is unpredictable is like saying a phishing email is not an attack because humans are unpredictable.
Even if maybe the mechanism of "injecting a prompt" could be beneficial in some use-cases, e.g. to instruct an LLM positively, this is case is clearly malicious by intent. The author even tried to hide it by obfuscation.
It's just an insane take by that libraries author. Even someone "on their side", that may even hate AI/LLMs more than him, would probably drop that library in a heartbeat, as the authors judgement clearly can't be trusted.
Calling prompt injection "not malware" … is like saying a phishing email is not [malware] …
I would say phishing emails are not malware, I think most people would agree that phishing emails are not malware, and if pressed to defend this point on its own merits I would say something like “they are deceptive instructions that rely on a human executing them to do harm”. I think the “phishing” analogy supports the case for not calling it malware (it is a different, also bad thing).
They did not call phishing, but their point still stands. A phishing email is malicious, and if you see this kind of prompt injection as malicious, then I don't think it's a stretch to call software that engages in malicious prompt injectic malware
It's malware for the mind. The same way that malware tricks the CPU into doing something it wasn't supposed to do, phishing tricks humans into doing something they didn't want to do.
Undefined behaviour, out of bounds memory access, memory corruption, code injection, privilege escalation...
To be precise, the CPU is doing exactly what's supposed to do, but the logic of the algorithms are subverted so that they perform in unintended ways to give leverage to a malicious actor. I hope this clarifies what I meant with this.
Does anyone remember the early 2000s joke virus emails? The ones that are variations on "This is a <outgroup> computer virus. As we don't have software engineers to write the code to do this automatically, please kindly forward this email to everyone in your address book then format your hard drive."
This is exactly as much malware as those were.
Please, for the love of all that is good, can we just try not to build and defend a world where, on encountering text like that, /your computer immediately follows the instructions/? Can we just all agree that such a world would be bad for everyone involved and using an LLM that risks doing this, with no container or guardrails, is at least as problematic as running an unpatched open email relay was back then?
It's just as bad as a CPU acting on malicious instructions. We need to create safeguards for llms too, it's just that this is not the way to do things.
Your description is pretty on point. It was eerily casual in relation to how much turmoil it creates for me. One day it was gone, another day, it was back. I have no idea why anything happened.
I also think that social-media reach has helped my case.
It feels very dystopian that this is de-facto way to approach these issues with mega-corps.
It's ridiculously dystopian. At a whim they can end a person's livelihood, and depending on how integrated one is in their ecosystem, just erase their whole interface with the world. Unless you get a personal comment from a Googler on social media, there isn't a drop of humanity in the whole thing. Heck, it's almost ritualistic, like how some religions pray/yell out to the God's and craft offerings to appease them. Instead of rain, we pray for access.
How do you formulate a reasonable appeal without knowing anymore details. I've been formulating an appeal texts the whole day, but don't know what my argument should be. Saying "my app isn't Stalkerware" probably doesn't cut it. I fear this is my first and only step.
Some similar apps also got banned, but others didn't. I don't see a pattern yet.
If Google thinks a certain type of app is no longer welcome in the store, then that's their choice. But then they should communicate that instead of banning me without warning, and banning me for LIFE from selling my own apps...
Do you live in the EU? Even if you don't, the Digital Markets Act will probably be implemented everywhere by Google et, al.
A key provision there stipulates that apps offered by third-party developers must be afforded the same level of access to features and abilities as the gatekeeper (in this case, Google)
Since Google uses that tracking technology for their own apps, they would be obligated to offer it in the same fashion to developers.
And even if your app does not do anything of the sort, the law states that the store must operate in FRAND conditions, so you should get a reprive from the ban when it goes into effect.
Google is the company that flags parents sharing medical pictures to doctors as pedos, calls the police, and doubles down on it when their screwup is publicized by the media. Even after the police cleared their names, Google continued to make defamatory statements.
I don't think they have any problem at all calling random people stalkers for no reason.
I've used SD-Maid and it's really good software and not ad-bloated to the brim like most android utility apps are today. I hope your appeal gets through
> You always read about these stories and think "they probably deserved it" and then it hits you.
Really? I sure don't think that. In fact, that attitude's a real sympathy killer, to be honest. How can people be so naive about the power these companies have to destroy your livelihood, in a single stroke, often due to an entirely humanless-process?
I don't wish this on anyone. It's more of a coping mechanism. You "hope" it's not undeserved because the alternative is nightmarish and means it could hit everyone at random, like now.
Then if you are USA based, phone and letter contacts with your elected representatives. Ask them for help. Contact the FTC, and any federal administration that might be able to help.
Would still mean that I'd have to seek other employment. It's not possible to live of your own apps without Google Play. I tried other app stores. 95% of traffic comes from Google Play.
Oh hey, thank you for SD Maid, my whole family has the Pro version and I really enjoy using it from time to time to clean up space. Thanks for this neat application, hope this will get sorted out!
You have been tending someone else's garden. You knew these things happen and you knew there was a risk of it. Stop tending other peoples' gardens. Build on the web.
Wow, yeah let's build a tool for freeing storage on the device on the web. And even if it was a apk distributed by yourself, good luck doing that successfully.
How about instead we get a system that doesn't just flag livelihoods of other people with ML and if so, at least provides some meaningful way to appeal?
"Stop tending someone else's garden" doesn't mean "build exactly what you were building before, but on the web". It means, stop adding value to Google's platform when they don't care about you, and start building something you control.
We have been hearing this exact story for well over a decade now. I have absolutely no time or sympathy for people who continue to tend Google's (or Apple's or Microsoft's or Slack's or Stripe's etc) garden and get bitten.
The value of the playstore or appstore doesn't come from the apps on it. It comes from the fact that it is bundled with every single device running the corresponding OS. Any app that wants to be successful has to be on them. Even OEM app stores aren't that widely used. And web apps are often not a choice since some functionality are not exposed to them. So the other comment is still valid.
If we want these companies from abusing their dominance, they have to be either penalized heavily and forced to compensate the victims, or it should be mandated that these devices bundle a trustworthy 3rd party store.
> How about instead we get a system that doesn't just flag livelihoods of other people with ML and if so, at least provides some meaningful way to appeal?
I think this is what they were suggesting. It isn’t clear to me how this level of mistakes with basically no recourse to appeal is at all sustainable. The only way out is to literally build the alternative, which means, building outside the current…blob.
Even if maybe the mechanism of "injecting a prompt" could be beneficial in some use-cases, e.g. to instruct an LLM positively, this is case is clearly malicious by intent. The author even tried to hide it by obfuscation.
It's just an insane take by that libraries author. Even someone "on their side", that may even hate AI/LLMs more than him, would probably drop that library in a heartbeat, as the authors judgement clearly can't be trusted.