Apple Intelligence Local or Cloud: Technical Breakdown

The architectural boundary between on-device intelligence and remote cloud servers represents the core privacy question in modern consumer AI. While marketing materials frequently describe Apple Intelligence as an architecture where personal data never leaves user custody, the production deployment is fundamentally a hybrid system. Under the hood, iOS 18 balances a compact 3-billion-parameter on-device foundation model with opportunistic routing to remote Private Cloud Compute (PCC) clusters and third-party endpoints like OpenAI ChatGPT. For systems engineers, security auditors, and privacy-conscious users, understanding the physical boundaries of this split is critical. When does inference execute inside the local unified memory of an iPhone, when do prompt tokens leave the device over cellular and Wi-Fi networks, and how does this compare to running 100% air-gapped open-weights models through Apple MLX?

The On-Device Foundation Model: Parameters, Quantization, and ANE Execution

Apple's on-device foundation model is a dense decoder-only transformer containing approximately 3 billion parameters. Developed by Apple's Foundation Models team, the architecture incorporates standard modern optimizations including Grouped-Query Attention (GQA), Rotary Position Embeddings (RoPE), and a shared vocabulary of 49,000 tokens.

Deploying a 3-billion-parameter model on mobile hardware requires aggressive memory budgeting. In standard 16-bit floating point precision (FP16), storing 3 billion parameters requires 6 GB of VRAM—an allocation that would immediately trigger system termination on an 8 GB iPhone. To resolve this constraint, Apple employs structured quantization and modular parameter adaptation:

• Mixed-Precision Palettization: Instead of uniform INT4 quantization, Apple applies non-linear palettization (averaging between 3.5 and 3.7 bits per weight). Highly sensitive attention projection layers retain higher effective precision, while feed-forward network (FFN) matrices are aggressively compressed. This reduces the base resident model weights to approximately 2.3 GB in unified memory.
• Dynamic Task-Specific LoRA Adapters: Rather than executing a generalized instruct model for every query, iOS dynamically swaps low-rank adaptation (LoRA) tensors (rank 16 to 32, measuring 20 MB to 50 MB each) depending on the active feature: Writing Tools (proofreading, tone rewriting), Notification Summaries, or Smart Reply in Mail. The base weights remain static while task-specific matrices are mapped into DRAM on demand.
• Heterogeneous ANE and Metal Execution: Ingestion and prefill operations are distributed across the Apple Neural Engine (ANE) and the Apple GPU via Core ML pipelines. On the Apple A18 Pro SoC, prefilling a 512-token prompt completes in approximately 38 ms, while autoregressive token generation sustains 33 tokens/second at an average SoC draw under 2.5 watts.

The physical parameter ceiling on iOS is dictated by Darwin's memory management subsystem. On devices with 8 GB of physical RAM (including iPhone 15 Pro, iPhone 16, and iPhone 16 Pro), Darwin enforces a strict ceiling of approximately 4.5 GB of anonymous dirty memory for foreground applications. After accounting for SpringBoard, active background daemons, and application graphics buffers, the operating system can allocate no more than 2.5 GB to 2.8 GB for the AI runtime without risking an EXC_RESOURCE (RESOURCE_TYPE_MEMORY) termination from the jetsam daemon. This physical boundary prevents Apple from running larger 7B or 8B foundation models locally.

Private Cloud Compute (PCC): When and Why Apple Intelligence Leaves the Device

Because the on-device model is strictly bounded to 3 billion parameters, it encounters clear capability ceilings. When a user prompt exceeds local boundaries, iOS silently escalates the computation to Private Cloud Compute (PCC). This routing decision is governed by an on-device request classifier and semantic router:

• Context Window Overflow: To prevent exponential growth of the Key-Value (KV) cache in local DRAM, the on-device model operates with a truncated context window (typically 2,048 to 4,096 tokens). Analyzing complex multi-page PDF documents, extensive email threads, or cross-application datasets exceeds local DRAM budgets and forces cloud offload.
• Deep Multi-Step Reasoning: Algorithmic problem solving, complex programmatic transformations, and multi-turn analytical deductions exceed the cognitive density of the 3B parameter checkpoint.
• World Knowledge and Web Synthesis: Queries requiring broad factual recall outside Apple's distilled training corpus cannot be resolved locally without retrieval from external knowledge bases.

When offload is triggered, tokens travel to Private Cloud Compute data centers. Apple engineered PCC around custom Apple Silicon server nodes (dual M2 Ultra or M4 Max blades) running a hardened, stripped-down variant of the Darwin kernel. PCC incorporates several notable security mechanisms:

1. Cryptographic Attestation: Before dispatching encrypted tokens, the client device verifies an attestation quote generated by the server's Secure Enclave. The quote cryptographically proves that the server is running a software image whose SHA-256 hash appears in Apple's public transparency log.
2. Stateless Ephemeral Execution: PCC server instances operate entirely in volatile memory (DRAM). They possess no persistent storage drives for user data, and the operating system eliminates remote administration tools, SSH daemons, and administrative debugging shells.
3. Non-Retention Guarantee: User prompt tokens and generated responses are wiped from server memory immediately upon completion of the inference request.

Beyond PCC, Apple Intelligence maintains a third routing tier: third-party commercial endpoints. When a user requests generalized world knowledge or creative generation that exceeds both local and PCC boundaries, Siri explicitly prompts the user to route the query to OpenAI's ChatGPT. Upon confirmation, data exits Apple's cryptographic enclave entirely, entering OpenAI's standard commercial cloud infrastructure.

To explore differences in cost and data sovereignty, see the comparison of local LLMs versus the cloud.

Empirical Benchmarks: On-Device Apple Intelligence vs. Local MLX vs. Cloud

To evaluate performance across architectural paradigms, we measured latency, generation throughput, memory consumption, and network dependence on modern Apple Silicon hardware running iOS 18 and iPadOS.

Local MLX vs. Apple Intelligence: The Case for True Air-Gapped Privacy

Comparing Apple Intelligence with local open-weights inference powered by Apple MLX reveals a fundamental philosophical and security distinction: cryptographic attestation versus physical air-gapping.

While Apple's Private Cloud Compute represents a commendable advancement in cloud security engineering, it remains inherently cloud-dependent:

• Network Telemetry and Metadata Exposure: Even when prompt payloads are encrypted to attested hardware, network packets must traverse commercial telecom carriers, cellular towers, and edge routers. Packet transmission timings, payload byte sizes, and IP routing pathways remain visible to network observers, exposing behavioral patterns.
• Connectivity Brittleness: The moment an iPhone loses cellular or Wi-Fi coverage—during air travel, subterranean transit, or remote operations—all PCC-dependent capabilities degrade or fail outright. Apple Intelligence cannot maintain complex reasoning or long-context document analysis in airplane mode.
• Vendor Constraints and System Prompts: Apple Intelligence operates under rigid system prompts, hardcoded formatting constraints, and task-specific filters. Users cannot modify system instructions, test uncensored checkpoints, adjust temperature parameters, or run specialized open-source models tailored to specific technical domains.

In contrast, running local LLMs via Apple MLX (as implemented in Lapis) establishes complete operational sovereignty. In Lapis, inference executes entirely within local unified memory. The application requires no network permissions, transmits zero telemetry, and functions with identical speed and precision in complete isolation. For sensitive legal analysis, confidential corporate documents, proprietary source code, and medical notes, true local inference eliminates third-party trust dependencies entirely.

To understand how offline privacy is guaranteed on iOS, read the guide to offline AI chat on iPhone.

Engineering Best Practices for Purely Local iOS Intelligence

Developers designing edge intelligence systems on iOS and iPadOS can maximize on-device efficiency and privacy by adhering to these architectural standards:

1. Revoke Network Client Entitlements: Omit the com.apple.security.network.client entitlement from your application sandbox profile. Auditing network sockets at the operating system level proves to security evaluators that the app cannot exfiltrate user prompt tokens under any circumstance.
2. Implement Hard Memory Budgeting with Darwin Memory Pressure Sources: Monitor system memory using DispatchSource.makeMemoryPressureSource(eventMask: [.warning, .critical], queue: .main). When iOS reports memory contention, proactively prune historical Key-Value cache entries or discard speculative tokens before the kernel activates jetsam termination.
3. Adopt Grouped-Query Attention (GQA) with 8-Bit Dynamic KV Cache: Mobile autoregressive generation is bandwidth-constrained. For a sequence length of 2,048 tokens, quantizing Key and Value cache tensors to dynamic 8-bit integers compresses the cache footprint to under 120 MB, preventing DRAM bloat during multi-turn conversations.
4. Monitor Thermal Throttling via ProcessInfo: Continuously inspect ProcessInfo.processInfo.thermalState. If the device reaches a .serious or .critical thermal state during extended decoding, introduce micro-sleep intervals between token generation passes to prevent hardware thermal throttling and maintain sustained GPU efficiency.
5. Leverage Zero-Copy Unified Memory (MTLResourceStorageModeShared): Allocate model weight buffers directly in unified memory accessible by both CPU and Metal compute shaders. Avoiding redundant memory copies between CPU and GPU preserves memory bandwidth and ensures peak token throughput on Apple Silicon.

The Lapis privacy policy explains how the app handles your data.