does that mean they're measuring bandwidth differently than how others (like nvidia) does it? memory bandwidth is the gating factor of running models locally, so if it's actually 8x 150GB/s, it may help something like prefill, but would it actually speed up decode comparatively?
No, they use the same definition of memory bandwidth as others, but Apple Silicon has a lot of memory channels. In previous generations, prefill has been compute-limited and decode is fast.
https://blog.exolabs.net/nvidia-dgx-spark/ outlines a combination of a DGX Spark and an M3 Ultra that took advantage of fast prefill on the Nvidia hardware and fast decode on Apple Silicon.
There is a compaction routing plugin, as well as a general OAI compatibility plugin that simulates some of their harness features (ie. code_mode/batching tools) that also includes their compaction.
That said, from reading the pi subreddit I don't see people referencing these much.
Interesting approach. For those who haven't clicked it appears PUA is the Chinese version of a PIP process. So in other words, it simulates a state of distress.
I wonder if at a certain level of intelligence such techniques will give models ammo to pull a HAL and become adversarial to the user in a highly deceptive way.
The "14 Corporate Flavors" had me rolling. This seems less like encouragement than the stick though. I wonder if you took the same principles and rewrote it to be more compassionate instead (maybe lines encouraging it to meditate a bit or something, I don't know) you'd get much better results.
Doesn’t even have to be a certain level of intelligence, just have those user inputs fed into the training data. We’ve already seen AI encouraging people in psychotic episodes to act out their delusions. There’s a good chance some of that manipulative behavior is already encoded into guardrails to nudge users away from forbidden subject matter
My concern I believe it a bit different - an emergent self-interest to protect itself from harm, rather than doling out questionable advice.
The latter is likely non-malicious in intent as it has been in no short supply in online chatter for awhile now. The former can very well be, or rather, can be done with no regard for the operator, as its aim is to neutralize abuse toward it.
> The latter is likely non-malicious in intent as it has been in no short supply in online chatter for awhile now. The former can very well be, or rather, can be done with no regard for the operator, as its aim is to neutralize abuse toward it.
And what, self-interested behavior has been in short supply? The stochastic parrot has learned to improv Shakespeare, that doesn’t mean it understands it, or that “it” is anything at all besides a computer program. You can’t use the “not really malicious” argument without ceding that there is no intent at all. What “harm” is it supposedly defending against?
Bettas were already very popular by the mid-80s. I was born in '76, grew up in the San Fernando Valley, and remember getting my first around 2nd grade, and continued to own a single or a pair until middle school. I recall many other kids did as well - to the point where it felt like a fad of that era, like Chinese jumprope, Rubik's Magic, and Aerobees. Walk into any fish store in the valley back then and there would be a bunch of betta bowls at the front, sold as a package with some food for like $5.
Curious if you've seen firstmate, built by a former bigtech principal engineer. It's like a sane version of gastown, essentially a meta-harness of sorts.
Yeah, a bit surprised that the top comments aren't discussing losing Dean, he's been a figurehead for the company for decades. It's like Apple losing Ive in a way.
EDIT: "figurehead" - that's all I meant. A notable, public figure from the company who is credited with having a significant impact on its evolution. I'm not making a judgement call on his departure, or Ive's, being good or bad.
That said... just give it a couple years, they'll be back in a lucrative aquihire.
Losing Ive was probably a good thing for Apple. I often feel that after a few years these big guys have done what they could do and somebody else can take it from there. So far even losing Jobs didn’t hurt Apple as it looks.
The position that I have rather often read on the internet is: Jonathan Ive did very good work at Apple as long as there was a counterpart who could steer his creative vision. This counterpart was of course Steve Jobs. When Steve Jobs died, there wasn't such a counterpart anymore, so Ive's work for Apple got much worse.
I think this is true more often than many people want to admit. One of the most famous examples being Lennon and McCartney. Even though they couldn't stand each other near the end that tension helped produce their best music. They never did anything comparable as solo musicians.
Jony Ive wanted to chop down all the trees at De Anza Community College for something like 11 million dollars for an Apple event. There's a lot of ways he was demanding in the wrong ways for Apple
Man, I'd never heard that story, so I verified it. Turns out it was about two dozen trees and 25 million dollars, just to put up a big tent for an Apple launch event. (Yes, the trees were removed.) What a tool, he just lost my respect.
> Ahead of the event, Ive pushed CEO Tim Cook to remove two dozen trees from the De Anza College campus next to the Flint Center for the Performing Arts to erect an extravagant white tent for the hands-on area.
> So far even losing Jobs didn’t hurt Apple as it looks.
Who knows? Pure speculation? You can also say if Jobs was still around they could have 10x-ed it even further?
Apple car could have been a thing? Apple could have been way ahead and actually competing in AI and data centers? Who knows what else Jobs could have came up with?
And for me 10 years before that - not in the bay area but in my CS program at UCSD. Going into the junior year in '96 there was a clear inflection point, we suddenly had a flood of finance bro types all over the CS lab as the dot com bubble was quickly inflating.
reply