We offer MCP and then consume it with our in-app assistant to go from a non technical prompt to a series of what is essentially API calls they can automate for themselves for repetitive tasks or things that require a few screens to accomplish can be done from the assistant widget itself, etc.
I agree with mostly all of this, but personally I wrote a toy LLM almost 5 years ago and while it never saw much use outside of boring my wife with a shitty command line demo with glee it did help me understand how they worked and how to apply them, played a lot with JAX and pytorch, ended up building a ghetto version of MCP and an LLM-Pool to proxy requests to my baby local models and so I didn't struggle to see the evolution of openrouter and MCP agentic workflows. The same way i'm really glad when I was younger I built a bad webserver by myself, a really painful SQLx type database, etc etc etc - none of these things led me to developing for Nginx or Oracle nor will knowing JAX get me a job at an AI research lab, but I do have a lot of depth in understanding how the technology works so that the flavors on top of them are easy to digest and make more use of immediately, and I think the same can be said for engineers coming into the field - if it's a spooky LLM box you aren't going to be squeezing the same amount of juice as the guy that knows how they work inside and out so having at least the understanding of a _babys first LLM_ is going to get you miles ahead of people who don't.
For anyone who wants to dork around there is https://github.com/rasbt/LLMs-from-scratch which is something amazing that I think anyone who wants to engineer things around LLMs should at least blast through and read.
Game cheating and reverse engineering MMO backends taught me a lot: databases, networking, securing a backend (and frontend), limitations of simpler languages when comparing them to more native options for building backends.
Agreed, I was very late to the game and was forced to learn VBA for excel sheets and that is how I finally broke into programming.
When I was a pre-teen I stumbled upon CD-rom hacking guide to bypass disc requirements on games, I remember opening up the file and the screen being filled with HEX code. I was so overwhelmed I just closed it and never touched programming after that for 15 years. My life would have been totally different if I had embraced the unknown instead of retreating.
Not specifically Anthropic but why are we allowing billing to take place in tokens that are nebulous and fully controlled by the operators who have no aligned incentives?
If I have a user input and then sanitize and inject that into a prompt to do something, I have no idea how much that is going to cost at all and no real way to measure this properly. A parallel example is digital ocean or aws, i can go and measure/limit my compute/fs/memory/startup times/etc and while it can be impossible to get down to the last flop of money allocated - i can run things on a real budget with real constraints, opposed to an LLM where I have to .. prerun a sanitized user prompt through a tokenizer and then ask an LLM to guess what it may do and give token consumption estimates and then act on those in any sane manner for the user?
Perhaps i'm missing something to do realistic and static rails on things but I don't see a serious way at scale to use the token billing model handling things requiring a users free text input short of having to go pander to VC money to throw money at it until someone else figures it out.
*to clarify my rambling...
We should be billed and given controls based on resource usage itself and not an opaque token concept on top of not being able to spin any knobs that control it's resource usage.
The model providers are quite aligned with concerns like customer retention. These arguments only work if there is no competition. We exist in a marketplace of black boxes. There's not just "the one" you must suffer. You have options. You can build your own too.
A per-token model roughly aligns with the providers' costs, and it is an objective measure, so it seems a reasonable way to charge.
I see posts all the time on HN about which models from which providers offer the most bang-for-the-buck, and how to minimize token usage and still get optimal results, so it appears that competition is working.
There is so obviously competition in this market it’s astounding to me that people attribute all this malicious behavior to the model companies.
They’re growing over 10x a year. They want users and revenue. In order to get users and revenue, they want to provide the smartest models at affordable prices. If they unnecessarily burn tokens, users will get less value and switch.
This thread is filled with competing comments about their monopolistic power and how when one model provider was no longer doing a good job people switched to a different one.
I was just complaining to someone that token billing is like letting a gasoline company control your gas pedal while you nicely ask them to use a specific gear that may or may not actually be in use and you guess what speed it's actually going based on how fast the trees go by because qualitative judgments have to replace the speedometer unless you can just burn money.
Don't forget also letting them dictate the route you drive while you are increasingly blindfolded till you arrive and then find out you need to tell them to drive you to your correct destination again because they drove you across town instead which burned half your tank.
“Claude, spend the next 10 hours trying to solve the Reimann Hypothesis”.
I agree that incentives are misaligned but there’s several competing model providers. If one gets funny with their costs people will jump ship, especially if the gap between the top 2 labs and everyone else keeps shrinking.
“I can take current sources and tell you how solved this is, but I am not willing to work to a timeframe or to solve things that aren’t yet solved by mathematicians or science”
These safeguards already exist when they get a whiff that you might be using Claude to fix security issues. Doesn’t seem farfetched given the incentives I outlined that they would apply to this kind of abuse.
How loose those controls are becomes a market force.
But just so we're both entirely clear on what an LLM is... it's a token prediction system.
it genuinely can't do things except recall things that have already existed.
People are having great success composing things together in new ways, but just like the english language has a finite number of sentences, and music has a finite number of chords: LLMs too are just combining things that have existed.
I don't want to sound condescending, it is remarkable how useful this technology is, but please don't evangelise them on capabilities that they genuinely can never have.
Laptop computers have incredible processing capabilities but nobody expects them to be able to walk your dog, no matter how useful they actually are at doing other things.
Yes and you suggested Anthropic have their coding agent refuse to attempt to solve anything it can’t find a preexisting solution to in case it turns out to be hard, if I understand you correctly.
Knowing how hard something will be to do before attempting it is precisely the sort of impossible thing that it couldn’t do.
As you claim to understand, they don’t know things, they sort of drift on vibes. Telling them to refuse things they think will be hard will only accomplish making them more annoying.
God I'm not an ai booster by any means but I'm so sick of this argument. If the maths proofs that have been put out recently are just 'just combining things that have existed' then that goes for everything and the term is meaningless. If LLMs are stochastic parrots then so are we.
Often LLMs seem to be aware that work is heavy, but knowing:
a) if something is possible
b) if something has been requested to take a long time ( a signal of abuse, like requesting illicit pictures in image generation)
is actually somewhat straightforward (I mean, if they are able to predict if something is a “substantial piece of work” as they seem to do already).
The mathematical proof thing is obviously marketing spin, you should pay more attention to what mathematicians are actually saying instead of hackernews folks.
These things are really good at being search engines and harnesses for iteration rather than some kind of advancing intelligence.
I'm not worried about the volatility in the definition, i'm worried that I give it 1 token today and receive 2 token output, tomorrow I receive 40. If i'm doing this a hundred thousand times a day it is difficult to price this in for users downstream or in the extreme cases be able to absorb that at all short of going into a failmode with degraded access until someone goes and buys more tokens or gets the bill. The alternative is just pass the buck and bill your non-technical customers with a "tokens" line iteim every month.
> If i'm doing this a hundred thousand times a day it is difficult to price this in
When you’re doing this 100K times per day you get an extremely good idea of what it costs. You also have all the tools to see when something starts changing quickly.
This change is for Claude Code the harness. If you’re using the API at scale and paying full price then you get exactly what you put into the request.
No, those doing this 100k times a day have very good data on this, good estimators and modeling. And the API has various knobs to change and evals will give you actionable data.
One guess is that their "primary" target audience/market is the large corporations that get their employees unlimited tokens, and not the individual developer who may worry about spending and token accounting.
It's the opposite. The enterprises have all the tooling to monitor token usage of employees, and to limit access. For example, we have a $300 month limit, and then need to file exception tickets when we need more to justify the cost. Pretty similar at other non-silicon valley company process. I don't know any enterprise who'se on unlimitaged token budget for their employees. that's not how enterprises sign contracts.
It is the automatic distribution and automatic update. The questions isn't "how does one download a text file to another persons computer?". It is "how does someone with a skill.md file on their computer discover that a new version of that file is available".
This isn't a "bottleneck" but rather a capability (or lack thereof). As you add more and more capabilities, especially ones relevant to enterprise situations like authentication, authorization, governance, etc. then MCP starts to pay off.
If you do not need those capabilities, then you do not need MCP. And then you shouldn't use it. But if you do need those capabilities then it might be worth using MCP rather than inventing your own way to do them.
I see. In my head it would be something like the agents harness having a list of services it interacts with, reaches out to service.com/agents.md for a fresh copy every so often and uses that to resolve the relevant tool calls.
>reaches out to service.com/agents.md for a fresh copy every so often and uses that to resolve the relevant tool calls.
that is basically what MCP is. except it answers all the questions that your version handwaves away - how often do you get a fresh copy, how do you describe the relevant tool calls, how are the tools organized, and how does auth work.
But for enterprise there may be teams, each developing their own way to do it. Then there will be many different ways that it is done throughout the enterprise, which is hard re: governance. Better/easier to adhere to an industry standard which can be audited, especially for enterprises where that is a legal requirement.
That isn't a reason you should use it, just an explanation about why someone has to use it.
this is awesome and I hope this gains some real steam, we're building everything in rust and locally if i'm watching youtube and running a build+tests and my vscodium starts running the analyzer at the same time I've seen my machine stutter out as it eats up the memory.
ha this is the first time i've seen this but honestly yes, if there was an accessible enough for a 4 year old OS to "explore" around in to find the applications I would be throwing it on the USB as I type
We've been on and off with water service because of it, we've got a bit of rain the last few days but the resovoirs in Puerto Rico were looking real skinny last week. Nothing humbles you like coming inside from doing yard work in the August Caribbean sunshine to realize your shower isn't running. RIP to my tomatos.
As someone who is fully bootstrapped and not looking for funding, I may be oblivious but some of the numbers I see versus the products I see i'm wondering if I should not be using my own capital and see if asking nicely can get me 200 million dollars
As a founder raising those big rounds, you'd enjoy a lot of personal luxury and develop a gtowing network of opportunity.
But the odds of your business catastrophocally collapsing on itself or being aggressively gutted in an acquisition would go sky high, and all those starry eyed people you recruited to help you along the way become way more like to encounter surprise layoffs or unexpected and severe equity dilutions.
A lot of it is really about where your heart is as a human. If you want to take a lucrative cut on selling an image to bigger and bigger investors and don't much care what happens to your idea or any of the "little people" along the way, then you probably should chase those funding rounds. But if you're actually passionate about your vision or have a more compassionate and paternalistic attitude towards the people you convince to work for you, you're doing the right thing with the bootstrapping.
(I'm sure you know all this, though. Nobody bootstraps without thinking about this stuff, and recongizing that that they just can't stomach the tradeoffs of doing otherwise.)
If it makes sense for your growth, why not? Just keep in mind that these founders are giving up control and saddling themselves with certain obligations in return.
reply