I am so charmed by AI (more specifically, agentic coding) ever since the end of July last year, that I did not even make a single post over a year. I knew I was not very good at coding, but AI lets me realize how bad my poor programming capability is limiting my imagination 😂 Ironically, I wished AI could free me up, but I actually got busier now…

I Was Wrong

Last year about this time, I was totally amazed by how much LLM can do, espacially have seen MCP can connect LLM to all kinds of tools so that the I/O of LLM is no longer just plain text. I had the feeling that within a couple months I could have an AI agent taking care of my daily work.

That is happening, but much slower than I thought, and MCP did not become the “type C for AI”.

What Is Wrong With MCP

At some time point, we tried to convert everything into MCP. Too many MCPs were made, very few are still been used. I made about (3+0.5+0.5) MCPs, 2 of them have no users at all (not even I would use). One 0.5 MCP is a meta MCP that proxies other MCP, which I use every day and have <5 other users. Another 0.5 MCP is the MCP assembled from a collection of tools dynamically in claude_agent_sdk, the automation pipeline is the only user.

Two problems of MCP were exposed immediately once Skill was invented:

1. loading MCP tools upfront consumes too much context, while Skill could do progressive disclosure
2. back-and-forth making individual MCP tools calls are slow on LLM inference and the intermediate results waste context

My meta MCP was designed to address the above shortcomings. Interestingly, Anthropic built solutions into Claude Code around the same time. That makes me feel:

• I am catching up the fast evolution of AI era.
• With with assistant of AI, individual developer could run ahead of big tech in terms of converting ideas into products.

Even with the two issues addressed, the question that MCP still need to answer is “Why not Skill?”. Authentication, input/output scheme checking are some aspects MCP does better than Skills, but they are not the moat: Skills scripts could take care of authentication, scheme checking as well. I bet the only unique niche MCP has is to establish the standard for UI. Or MCP should just join Skills: Skills over MCP is another feature I am looking for (and building my own implementation).

A recent release of latest MCP spec was out last week, and I saw some criticism saying it is going backwards to RESTful. Wish MCP could get a good advancement after all the struggling.

What Is Wrong With AI Agents

I feel both good and bad for the fact that AI agents have not yet take over my daily tasks. The challenges mostly come from two aspects: domain-specific knowledge and non-deterministic.

Domain-Specific Knowledge: There is an asymmetry between training and inference. At training time, the neural network gets to see a huge volume of data, spending months of computing time to “learn” general rules from the data. That is why LLM is so good at being coding agent, because coding is such a generalized task the LLM have seen tons of tons of examples in training. However, when we use the general LLM to build AI agents for our daily tasks (including a considerable amount of non-coding parts), there are many domain-specific knowledge, or just some unwritten conventions, or several tricky pitfalls. Those have to go into the context. When they are missing from the context, the AI agent’s behavior start to diverge from our expectation. When there are too many hints in the context, the AI agents could lose the focus, and ignore some of the rules even they are given higher weight.

The advancement in computing switch the mainstream of AI from hand-crafted programs to general scaling approach, and we learned our bitter lesson. While what I am saying is due to the unaffordable cost of training, we do not get models for each unique scenario, then the performance of generalized model on domain-specific tasks still fall short to our expectation. With context engineering, we are fixing the symptom to some extent, but does not address the root cause yet. The way we use neural network today, prevents a model from continously learning as working on the specific tasks.

Non-Deterministic: Due to the nature of neural network compute, applications using LLM cannot produce the outcome in the same deterministic way as conventional programs do. The negative impact are two-fold:

1. Reproducability: We call it “bug” if a program does not match to the spec, and looking at the source code, we can fix it; while no matter how hard we tweak the prompt, we cannot force LLM to 100% follow the instructions. With an AI application works very well 10 out of 10 times during development, users may still report “bug” in their runs on the exact same scenario tested, and we have little to do for “debugging”.
2. Integration: It is challenging to integrate other deterministic code with LLM, espacially adding the deterministic part as downstream. The output from LLM could vary dramatically, it is impossible to design the downstream processing to reliably parsing all the forms.

Structured output plus validation loop (that checks outputs and prompt LLM to reiterate on the answer with hints on how the scheme is violated) helps a bit on the second point. Agent observability is also very important, that if we ask LLM to “think loudly” and log its “reasoning” as well as every tool calls, the transcript would help us to “debug” how LLM stuck into a loop or hit a pitfall, then patch the prompt more precisely or fix the problem in the harness.

Postive Thinking

After a year of learning and practice, I have much deeper understanding on AI agents. I can clearly see that the gap between current status and my goal (i.e., taking over my daily tasks) needs quite a bit engineering work to fill. This is the opportunity for me.

Working with AI also accelerates my learning. Although I do not get the hand-on experience because Claude Code writes the most code, I got exposure to many more new concepts at high level. I am an introverted person, but I feel comfortable to ask AI agents as many questions as needed to fully understand something. No more worries on “silly questions”.

I had an endless list of things to do a year ago, and the list might have grown longer now. The difference is my current list has more “exciting” ideas. I was writing code slowly, clumsily, but now coding agents could write code much faster and better, so I can iterate on ideas almost instantly. Additionally, I was not “brave” enough to dream big, as I have to take conding into consideration, which was not cheap. That is much a less concern now, because code becomes cheap now, and you can get more than needed as long as you ask AI agents. As a result, I want to try many more insteresting things that was beyond my radar before.

The past year with AI is transformative. I am looking forward to the next evolution moment.

Credits

• Introducing advance tool use on the Claude Developer Platform
• The Bitter Lesson
• Learning the Bitter Lesson
• Defeating Nondeterminism in LLM Inference