Wasted tokens and ‘chaotic’ systems

Despite senior executives’ interest in AI agents that can handle office tasks like interns who never sleep, the underlying technology remains weak and potentially cost-cutting.

That was evident at two separate events held in Silicon Valley this week, where executives and engineers discussed the current excitement and challenges involving AI agents.

Kevin McGrath, CEO of AI startup Meibel He said during a session that “the biggest problem we’re working on in AI right now” stems from the misconception that everything has to be processed by a big language model, or LLM.

“Give all your tokens and all your money to an AI Claw bot that will waste millions of tokens,” McGrath said, before explaining how companies should be more careful in deciding which tasks are best suited for AI agents.

Since the recent rise of OpenClaw, a “harness suite” that allows developers to use various AI models to build and manage fleets of digital assistants, the tech industry has been pushing AI agents as the next big thing.

Nvidia CEO Jensen Huang told CNBC’s Jim Cramer in March that it “is definitely the next ChatGPT.”

But on Wednesday Generative AI and Agency AI Summit Technical staff of companies like the one in San Jose Google and DeepMind AI unit, Amazon, Microsoft And Meta It turns out that building and running AI agents is no easy task.

A session moderated by Google software engineer Deep Shah focused on new techniques aimed at helping manage the operational costs of running tons of AI agents.

Running AI agents costs money, and a poorly designed and maintained system to monitor these digital assistants and their actions could potentially end up burning money rather than saving it.

“If you think about a machine learning system or any multi-agent system, there are a lot of challenges that you face when you try to deploy that system at scale,” Shah said. “The first is the cost of extraction.”

Ravi Bulusu, CEO of startup Synchtron, highlighted the problem of complexity, noting the various ways companies organize data, choose technology platforms, and build and run software and workforces.

“No dimension is solved in isolation, and what makes this difficult, even chaotic, are the interdependencies,” Bulusu said, as running AI agents significantly touches on all these points.

The theme of the complexity of AI agents continued Thursday at an AI event in Mountain View, Calif., featuring ThinkingAI and MiniMax, both headquartered in Shanghai, China.

ThinkingAI recently rebranded as an AI agent management platform, moving away from its origins as a mobile gaming analytics company when it was known as ThinkingData.

As part of the rebranding, ThinkingAI has partnered with MiniMax, which went public in Hong Kong in January. One of China’s leading AI labs, it has become one of the country’s so-called “AI Tigers” by making powerful models available to the open source community for free.

ThinkingAI co-founder Chris Han said the move to AI agent management technology is part of an effort to expand from the video game industry into other industries that are excited about AI agents but lack the expertise.

Despite OpenClaw’s growing popularity in China, Han said it is too complex and too prone to security flaws for businesses.

“OpenClaw is a good tool for personal stuff, but it definitely doesn’t reach the enterprise level,” Han said. “You have to figure out a lot of things at the organizational level—your memory, how to manage your agents, your teams, your communications—there are a lot of things you have to figure out.”

Han declined to comment on possible national security concerns about Chinese AI models that could affect ThinkingAI’s strategy, but said the service could also support AI models from companies like OpenAI and others. Google.

If the US government were to ban China’s vulnerability-heavy AI models in the country, Han joked that he might take that as a good sign.

“If this happens, maybe we will be successful,” Han said.

WRISTWATCH: AI demand metrics are broken and only Anthropic is behaving realistically.