Interesting idea. I’ve been noodling about something similar myself for a few months, but I haven’t moved forward with testing it. What sort of outcomes are you seeing with it? IMO, we’re never going to get to AGI without fusing “soft” AI decision making with “hard” logic and symbolic algorithmic reasoning. Humans don’t realize this most of the time, but we routinely use them all.

Sorry, did not notice your comment until just now.

So far I am observing two things:

1. For smaller models, performance on Benchmarks such as DeepPlanning does increase significantly.

2. Context hygiene for sub agents becomes much simpler, since that can be expressed relatively concise and the mechanics are handled by the runtime automatically.

Still looking for a good test cases to study possible advantages, but running reliable benchmarks does take time and money...

Are you the Deepclause author? I tried it yesterday and my first reaction was that it was slow. Perhaps I’m doing something wrong, however. Running against the same model in Pi was lightspeed in comparison. Second reaction is the prompt editor needs to handle more than a single line of text and it needs Emacs editing key bindings like Pi. I’m happy to do some testing on it and provide further feedback. What’s the best way to submit that? GitHub issues?

Yes, I am the author and thanks so much for trying! Please do submit a github issue. My first suspicion about the speed is that maybe an inner loop is taking too many turns until the model finally realizes that a task is finished (so that in turn the runtime knows whether the predicate failed or not and can continue execution accordingly). Happy to take a closer look!

The point about multiline prompts is very valid obviously, that's on the todo list.

Cool, will do. Was playing around with it last night some more.