Council Post: Personal State Is The Missing Layer In AI Architecture
Regan Peng is an AI founder and former senior tech executive, focused on personal AI, privacy-preserving systems, and edge AI.gettyMost AI architecture debates begin with the model: How large should it be, and ...
Regan Peng is an AI founder and former senior tech executive, focused on personal AI, privacy-preserving systems, and edge AI.
getty
Most AI architecture debates begin with the model: How large should it be, and where should it run?
After working on personal AI systems, I think that puts the decisions in the wrong order. The harder question is what the system should be allowed to know and where that knowledge should live.
A useful assistant needs continuity. It may learn a user's relationships, routines, preferences, goals and communication style. It may also need access to calendars, messages, location and credentials. Centralizing that context in the cloud may be convenient, but convenience is a poor foundation for trust.
Personal AI should be designed around state, not only around models.
Architectures require three layers.
The first layer is private state, held on the device or in an environment the user controls. Identity, memory, behavioral context, permissions and credentials belong there. Local models can use that information without exporting it by default.
The second is general intelligence. Cloud models remain valuable for broad knowledge, complex reasoning and elastic compute, but they shouldn't need unrestricted access to a person's digital life.
Between them sits a trust and orchestration layer. It decides what runs locally, what goes to the cloud and what shouldn't run at all. The decision should reflect data sensitivity, model confidence, connectivity, energy, cost and the consequences of error.
This layer must also minimize what crosses the boundary. Removing names and phone numbers isn't enough; location, timing, relationships and behavior can still identify someone. NIST's de-identification guidance likewise cautions that masking obvious identifiers doesn't eliminate re-identification risk.
A better principle is semantic minimization: Send only the meaning needed to complete the task.
Suppose a user asks an assistant to tell a friend they'll be at the friend's apartment at 6:00 and will bring the gluten-free dessert they like. The cloud may only need to draft a warm message confirming the time and dietary preference. The device can resolve the recipient and then ask for permission to send it.
The cloud should receive a task, not a biography.
Routing is a trust decision.
Private, real-time, offline-dependent or action-bearing tasks should usually remain local. Requests requiring broader knowledge or intensive reasoning can move to the cloud. Many interactions will be hybrid; the cloud creates a plan or drafts language, while the device supplies private context, checks permissions and acts.
I keep coming back to one rule: The cloud may recommend, but the device should authorize. Before shipping a personal AI product, teams should answer four questions:
1. What information is never allowed to leave the device?
2. What is the minimum representation the cloud needs?
3. What evidence justifies escalating from a local model?
4. Where is the final authority to act enforced?
These are architecture decisions, not privacy language to be added later.
Tesla offers a useful analogy. The lesson isn't that every neural-network layer must be divided between a vehicle and a data center but that the intelligence life cycle can be distributed. Tesla says AI data used to recognize road features is processed in the vehicle by default, while users control whether eligible camera recordings are shared for fleet learning. Its neural networks are trained using fleet-sourced scenarios, while inference hardware runs the deployed software in vehicles.
The pattern is practical: Sense locally, learn globally, deploy locally and act locally.
Personal AI can follow a similar loop. Devices learn preferences locally. Privacy-preserving, aggregated signals improve shared models, and updated models or lightweight personal adapters return to the edge.
The router is the research frontier.
This is where the research gets interesting. A useful router must make decisions across uneven hardware, intermittent connectivity, limited memory and battery, uncertain outputs and different privacy requirements.
An April 2026 MIT CSAIL report on federated learning across resource-constrained devices, together with PockEngine's work on efficient on-device fine-tuning, points to the broader challenge. The problem is how to preserve personalization while managing computation, communication and privacy across a distributed system.
How should privacy leakage be measured at the task level? How should local-model uncertainty be calibrated before escalation? How can a personal model keep learning without exporting the behavior behind it? How should a router balance quality, latency, energy, cost and user control?
One metric I'd like to see the industry develop is personalization value per disclosed byte: How much meaningful benefit is created for every unit of personal information that leaves the device?
I also don't think technical privacy is enough. Private doesn't automatically mean aligned. A model on a phone can still optimize for clicks, spending or compulsive engagement. Users should be able to inspect, correct and delete what an AI believes about them, and consequential actions should remain behind explicit local permission.
As foundation models become widely available, the durable advantage will be the trusted personal state around them: memory, context, permissions and an architecture that knows when not to send information elsewhere.
The cloud can understand the world. The edge can remember the person. The orchestration layer must decide what each is allowed to know. That boundary—not parameter count—will define the next generation of personal AI.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?