Compute Constraints Challenge Personal AI Agents

Meta’s Muse personal assistant has persuaded AI companies that personal agents, which proactively complete tasks, are a likely next step for the technology. But companies may not have enough computing capacity to power them on their expected timelines.

Elon Musk said this week that running Grok Bot, the personal AI assistant SpaceX introduced in August, costs “unsustainably high relative to revenue.” He suggested SpaceX was reducing the compute allocated to it, so users reach usage limits faster.

At a Monday dinner with reporters in New York, Adam Fry, head of ChatGPT’s consumer business, said OpenAI’s Dots assistants use substantial computing resources. Each Dot has its own cloud computer to carry out tasks for users, such as reading and replying to email and finding and booking event tickets. Fry said a “background mode” is planned to keep Dots available continuously.

Dots are currently limited to Pro subscribers, who pay $100 per month. Fry said broader access would require a breakthrough in computing efficiency, which he expects to reduce costs rapidly while model performance improves. However, more capable agents that run continuously also consume more computing resources.

On Tuesday, OpenAI announced a system that routes tasks to different AI models, including GPT-6.1 Terra. Terra is smaller and uses less computing power than GPT-6 Astra, which originally handled all Dots tasks. OpenAI says the automatic router will make Dots less compute-intensive.

Anthropic cited difficulty securing more compute as a major risk factor in its recent S-1 filing. OpenAI relies largely on Nvidia GPUs, while Anthropic uses four types of chips. OpenAI is developing an in-house chip called Jalapeno, which is scheduled to deploy at scale next year.

OpenAI and Anthropic are both unprofitable, and OpenAI is seeking funding. The Wall Street Journal reported that OpenAI’s latest fundraising round raised $11 billion, below its $30 billion target. The newsletter says an IPO may be the only way to fund a personal-agent expansion, while noting that its success is not guaranteed.

Meta’s Muse personal assistant has persuaded AI companies that personal agents, which proactively complete tasks, are a likely next step for the technology. But companies may not have enough computing capacity to power them on their expected timelines.

Elon Musk said this week that running Grok Bot, the personal AI assistant SpaceX introduced in August, costs “unsustainably high relative to revenue.” He suggested SpaceX was reducing the compute allocated to it, so users reach usage limits faster.

At a Monday dinner with reporters in New York, Adam Fry, head of ChatGPT’s consumer business, said OpenAI’s Dots assistants use substantial computing resources. Each Dot has its own cloud computer to carry out tasks for users, such as reading and replying to email and finding and booking event tickets. Fry said a “background mode” is planned to keep Dots available continuously.

Dots are currently limited to Pro subscribers, who pay $100 per month. Fry said broader access would require a breakthrough in computing efficiency, which he expects to reduce costs rapidly while model performance improves. However, more capable agents that run continuously also consume more computing resources.

On Tuesday, OpenAI announced a system that routes tasks to different AI models, including GPT-6.1 Terra. Terra is smaller and uses less computing power than GPT-6 Astra, which originally handled all Dots tasks. OpenAI says the automatic router will make Dots less compute-intensive.

Anthropic cited difficulty securing more compute as a major risk factor in its recent S-1 filing. OpenAI relies largely on Nvidia GPUs, while Anthropic uses four types of chips. OpenAI is developing an in-house chip called Jalapeno, which is scheduled to deploy at scale next year.

OpenAI and Anthropic are both unprofitable, and OpenAI is seeking funding. The Wall Street Journal reported that OpenAI’s latest fundraising round raised $11 billion, below its $30 billion target. The newsletter says an IPO may be the only way to fund a personal-agent expansion, while noting that its success is not guaranteed.