It's wrong. It's about "problem immigrants" and, to be honest, I didn't research further than that. It depends on the exact definition whether I agree or not with the proposal.
Ah yes I remember just masses of normal Americans chanting: "nationalize AI! Government should decide what private companies get to sell their products! Ideally they do it based on who is paying them! Tax me more for imports! Go to war with Iran!"
I like how straightforward "democratic mandates" are, especially when you're dealing with a pathological liar
> I think it is a reasonable term to use in the context of adversarial AI development
Distillation is neither an 'attack', nor 'adversarial' in any reasonable sense.
> Does that mean I am getting paid by the CIA or something
It at the very least means you're uncritically parroting the framing of a company that'd like nothing more than to successfully persuade lawmakers that something should be done about open Chinese models (eliminate choice), preferably so that Anthropic is close to anyone's only option.
It seems like a reasonable person could say that a model being distilled "against the model provider's wishes" is in some sense a cyber attack that is stealing information (eg the lower order bits of the model weights)
I think this is mostly a confusing way to describe it, but I'm not really sure why you say it isn't an attack or adversarial. One side is doing something the other side doesn't want. Seems to be pretty clearly adversarial.
> I'm not really sure why you say it isn't an attack or adversarial. One side is doing something the other side doesn't want. Seems to be pretty clearly adversarial.
This is Anthropic we're talking about here. A company that's infamous for adversarial scraping of copyrighted content. I generally don't accept their framing, especially when it's pretty clear what the end goal of that is.
> Theft presupposes a legitimate possessory claim by the victim. If A’s possession of X is itself wrongful because A stole X from B, then when C takes X from A, C has not violated A’s rightful ownership of X—because A has none.
I think the whole thing is a bit fraught. In the best case, all frontier model companies would have invested in a giant expansion of Wikipedia and thus distillation would be stealing because the base information is already public and available. Obviously that's not what happened.
However, at this point, I suspect the stolen books (and scraped websites) are largely a footnote of training. Something that was essential to create early models, but relatively minor given the work expended since to create new content and RL environments
I think you misunderstood my point. Distillation is not stealing. It's not that
"Theft presupposes a legitimate possessory claim by the victim. If A’s possession of X is itself wrongful because A stole X from B, then when C takes X from A, C has not violated A’s rightful ownership of X—because A has none."
Anthropic has no standing in claiming that using its model output is 'stealing' because that would basically eviscerate their business model where they claim that enterprises can use its model output and still claim the generated code as their IP.
My understanding is that distillation uses specific probing queries to extract model weights in some meaningful sense.
There's also the element of "how one uses model outputs can be in accordance with or against the TOS"
(This is true of any website! How I access and use the info on the website can be in accordance with or against the TOS)
Both of those things: doing something against TOS, using specific queries to illegitimately gain access to model weights - that's why calling it adversarial and stealing feel fair.
I didn't (and don't) understand your point, but in an earlier post you made some claim about it being Anthropic - you can't steal from Anthropic. The above is the only reason I can think one might believe that
> I didn't (and don't) understand your point, but in an earlier post you made some claim about it being Anthropic - you can't steal from Anthropic.
I do believe that Anthropic complaining about their TOS being violated is rich and should be thrown into the dustbin considering that Anthropic violates other people's TOS when it suits their business.
I do also think them complaining about it is aimed purely at influencing lawmakers to ban open Chinese models and eliminate choice so that them and OpenAI are the only game in town.
As far as other models distilling using Claude, that's just consuming the output of a service in a particular way. Anthropic should be very careful about imposing restrictions of how you can use their model output if they want to maintain marketshare. Enterprises are not going to like that.
That leaves the TOS violation of maybe hitting the service too fast, using too many accounts etc. Except again Anthropic did exactly the same thing when scraping for training data.
Do you believe The Pirate Bay complaining about a competing site stealing 'their' torrents should be taken seriously?
That's how I look at this.
For these reasons, I don't think Anthropic has a legitimate argument here.
Also distilling simply means using the answers from one model to specific questions to steer another model to answer a category of questions in a similar manner using its own capabilities. So as far as extracting weights, it's not exactly that, it's maybe slightly biasing the model that's being distilled so that its weights might look somewhat closer to the model used for distillation, but it's far from extracting them exactly/directly.
They could gain a shedload of market share by remaining cheap while pricing in the whole rest of the industry rockets up?
Especially after just releasing the Neo, their cheapest laptop ever. They could be taking chunks out of the PC market right now if they werent so greedy.
Because it could be strategically good to increase their market share and loyalty when everyone else is increasing their prices, for instance? But no, the "fiduciary duty" thing trumps everything, I guess.
Apple is free to make either choice (or anywhere on the spectrum) without violating any duty they have to shareholders.
“We chose to invest in future market share, believing that was in the best interest of the company” is a perfectly valid choice for the company leaders and board to make.
We are only talking about opinion here. Obviously Apple shareholders have their own and thats what they use to direct the company, thats fine. My opinion would be more along the lines of 'Lets sacrifice 1% of profit this year to gain huge chunks of market share by undercutting literally everybody else in the industry for the first time in our history'.
I had the same opinion when they released the neo. In 3-4 years time there are going to be absolutely loads of them looking shabby and tired and running very slowly. Apple has been so careful over the past 3 decades to avoid that image, I'm surprised they are happy to go down that road now.
But they also determine the token prices. What you describe could also be true if they take a 5x profit margin on api tokens and 2x margin on subscriptions.
