Octolane · Revenue Superintelligence reposted this
I studied the 21 most important research papers on agent building and harness engineering for Octolane · Revenue Superintelligence, here are 7 things I learned that you can apply this month: 1. Your harness is worth more than your next model upgrade: Prime Agent took ARC AGI 3 from 30% to 95.5% on identical weights. They only updated the loop, the context, and the tools. You should spend a week on what surrounds the one you have before you pay for a bigger model. 2. Optimize your prompts instead of hand writing them: You need to stop hand writing prompts. You can optimize them real time. GEPA reads its own failed traces in plain English and mutates the prompt from what it saw, beating RL with a fraction of the rollouts. Try this today, you can literally pick your flakiest prompt, build a 30 example eval set, run GEPA on it and see amazing result. 3. Never throw away a failure: Reflexion's whole idea is when the environment says "wrong," write that back into context as words before the retry. Cheapest version is Self-Refine which is basically same model grades its own draft and then rewrites. To implement this you can add one line to your loop that appends the error message and a one sentence reflection before the second retry. 4. Turn repeated tool chains into skills: Voyager chains tools into a routine, verifies it worked, and saves it to a library the agent searches later. That's actually the design behind every SKILLS.md in your repo today. To implement you can log your agent's tool sequences for a week. Anything it does 3+ times becomes a named skill file. 5. Give memory CRUD: MemGPT lets the model create, read, update, and delete a region of its own context. That's literally the difference between a transcript and managed state. To implement this you can carve out a "working memory" block the agent can overwrite. Please stop letting the transcript be the memory. 6. Make the action space "whatever you'll execute." (My favorite) Toolformer to InterCode to Recursive Language Models is one arc: once the action is code in a persistent REPL, the tool list stops being finite, and an LLM call inside the REPL can spawn more LLM calls over a document too big to read. To implement this you can give your agent a persistent Python REPL instead of 15 hand written tools. Watch how many of the 15 you can delete. 7. Measure task horizon instead of just accuracy: METR's paper argues harness progress only becomes visible when you track how long a task the system completes end to end. Single turn pass rates usually hide the thing that actually improved. To implement this you can add one metric to your dashboard: longest task completed without human intervention, in minutes. Track it weekly.