Probably AVX-512. AFAIK Apple CPUs are still using ARM NEON instructions with 128-wide SIMD registers. I guess they are banking on you using the GPU if you want to parallelize those kinds of workload.
It might be conceptually similar to a single-output-token LLM (sort of). LLMs output next-token probabilities. You can ask LLMs to output yes/no, or to output only a color, or only a digit or something like that.
In this case I would imagine that they probably embed your input data into a vector space, and they embed your questions/outputs into another space, and manage to predict probabilities/classes/scores for your outputs very quickly. Embedding the output classes/questions into a vector spaces gives you something you can reuse across runs cheaply, as opposed to an LLM where you can prefill the KV cache but this is an expensive operation in terms of memory.
Agreeing with you. Trump's attitude on the global stage has been to disrespect, insult and threaten every ally. Come at everyone with a "we're better than you" attitude and show that he can't be trusted. The only rational response is to take steps to be more independent and find ways to work around the US. What's the alternative? Give in to a narcissistic bully who won't keep his word and will inevitably come asking for more later?
In the medium to long term, Trump is very old, his approval ratings are in the toilet. I hope that the democrats do well in the midterms and put up a viable candidate to beat him next election (if he's still alive). I also hope that the republicans manage to put up someone smarter and less morally reprehensible next time. Because I mean, this guy doesn't actually represent any kind of conservative ideal in any way.
The only people he appeals to are those one or two standard deviations on the left of the bell curve.
If the next US president is halfway sane, Canada and the US will likely begin to repair trade relationships, but I think that Canada and the rest of the world will be a bit more careful next time. The US may never regain the international standing they enjoyed before. Trump has made China look like the more stable and rational trading partner, and accelerated the decline of the US.
Most people (and most companies) can't tell bad ideas from good ideas. This is also true of LLMs to some degree. They need post-training in specific domains (e.g. coding) to become competent in that specific area.
You can use it for things like classifying documents, spam detection and detecting certain events, etc. You're right that it's not strong enough for coding but it can still be quite good for question answering and such. Or even simple things like converting documents from one format to another.
And yeah, for NPCs, this model and their other A1B model can fit inside of a game and run at a decent speed on CPU-only. It would probably be workable for game characters.
I would like to see an examples directory with a diverse set of example programs. What would be most interesting to me is programs with non-trivial memory usage and data structures, because that's where differences in the memory model would matter. I want to know what's possible and what's not allowed in this language. What makes this different and better than Rust's borrow checker.
The weights are very unlikely to be on the chip itself. That wouldn't work for SOTA models that are terabyte scale, even quantized. This is probably an accelerator for specific kernels in the model, but the weights are likely loaded from memory. The chip may have SRAM to store some of the weights temporarily during inference.
At least in the case of Taalas the weights are physically encoded directly on the chip.
It’s composed of 4-bit multiplier cells that compute all 16 possible results in parallel. The top metal wiring layer physically selects the one that corresponds to a multiplication with that cell’s constant weight, and routes it to the next layer.
I think that people will argue around that. Self-driving cars are already strong enough to kill people and even if you only made the robot strong enough to lift 15 pounds of weight, that's probably strong enough to slit someone's throat with a kitchen knife while they sleep... There probably isn't a level of strength that's weak enough to be completely safe but strong enough to be useful.
So at some point you have to trust that the tech is safe. Both in terms of "robot won't go off the rails", but also in terms of hostile actors can't remotely take over your robot while you sleep.
In terms of sleeping, personally, I would like for the law to mandate that robots must have a physical off switch, in a very visible location, that physically disconnects power. The switch should be illuminated while in the ON position. What makes me a bit pessimistic there is that we don't even have laws to mandate webcam indicator lights (e.g. a very tiny red LED) must be ON in hardware.
reply