Run LLM Locally Without Internet: Offline AI Guide

Running a large language model locally without internet is the defining benchmark of computational autonomy. While cloud-dependent artificial intelligence routes confidential prompts through remote data centers and hybrid mobile features quietly fall back to remote servers, offline inference executes the entire machine learning pipeline directly on physical silicon inside the device. Accomplishing this on an iPhone or iPad requires navigating strict operating system boundaries: physical DRAM budgets governed by Apple's Jetsam daemon, memory bandwidth limitations of mobile processors, and the complete elimination of network stack dependencies. Through Apple MLX and unified memory architecture, modern Apple Silicon devices can run sub-4B open-weight foundation models completely disconnected from the internet at interactive speeds exceeding 30 tokens per second.

The Mechanics of Disconnected Inference: How an LLM Runs Offline

An offline large language model operates as a closed mathematical system. In a standard cloud architecture, an application simply serializes user text into an HTTP POST request and waits for streamed tokens from a server cluster. In contrast, running an LLM locally without internet requires the client application to host and execute every stage of the transformer architecture on device:

• Zero-Network Memory Mapping (mmap): The model weights—quantized to 4-bit integers—are stored on the local flash filesystem. Rather than allocating gigabytes of conventional heap memory, the application memory-maps the model file directly into unified memory using MTLResourceStorageModeShared. Weight tensors are paged into DRAM on demand by the Darwin virtual memory manager without network socket initialization.
• Deterministic Local Tokenization: The user input string is converted into token IDs using an embedded Byte-Pair Encoding (BPE) or SentencePiece tokenizer implemented in native Swift or C++. No remote tokenization endpoints or external API calls are touched during vocabulary lookup.
• Compute Graph Dispatch on Metal: The Apple MLX framework compiles the model's forward pass into optimized Metal compute pipelines. Attention layers, rotary positional embeddings (RoPE), and feed-forward networks (FFN) execute directly on the Apple Silicon GPU cores, exploiting shared hardware memory without IPC serialization penalties.
• Autoregressive Decoding Loop: The model generates one token per step, appending the new Key and Value states to an attention buffer (KV cache) residing in physical DRAM. Once an end-of-sequence (EOS) token or maximum length is reached, the token IDs are converted back to plain text and rendered to the user interface.

To explore sandbox isolation and data-at-rest encryption in depth, see the guide to air-gapped local LLMs on Apple Silicon.

True Offline vs. Pseudo-Offline: The Hybrid Architecture Illusion

Many commercial applications advertise on-device AI while maintaining covert dependencies on cloud infrastructure. These pseudo-offline architectures routinely break when connectivity drops:

First, several consumer AI implementations enforce silent network licensing handshakes, telemetry beacons, or model integrity pings at startup. If an iPhone is in Airplane Mode, deep in a subway transit tunnel, or operating within an air-gapped facility, these applications fail to launch or disable core functionality.

Second, hybrid systems rely on server-side arbitration models to route complex prompts to the cloud while handling trivial queries locally. When internet access disappears, their analytical depth collapses.

True offline execution requires provable Zero-Egress network isolation. On iOS, developers enforce this by removing com.apple.security.network.client and socket permissions from Entitlements.plist. The Darwin kernel intercepts and rejects any low-level POSIX socket syscalls (socket(), connect(), sendto()) with an immediate EPERM fault. The model runs because it is mathematically self-contained.

To understand DRAM ceilings and anonymous dirty memory under the jetsam subsystem on 8 GB devices, review how much RAM a local LLM needs.

Empirical Benchmarks: Offline Inference Across Apple Silicon

To evaluate performance without network connectivity, we tested leading open-source models under Apple MLX using affine 4-bit quantization (group size 64). Testing was performed with active hardware network isolation (Wi-Fi, Bluetooth, and Cellular disabled, verified via macOS Remote Virtual Interface packet capture with 0 outbound packets) across three Apple Silicon tiers: iPhone 15 Pro (A17 Pro, 8 GB RAM, 51.2 GB/s bandwidth), iPhone 16 Pro (A18 Pro, 8 GB RAM, ~68.0 GB/s bandwidth), and iPad Pro (Apple M4, 16 GB RAM, 120.0 GB/s bandwidth).

The empirical data shows that sub-4B foundation models deliver interactive throughput while operating completely disconnected. SmolLM2 1.7B provides instant responsiveness on the A18 Pro (78.0 tokens/second), while Llama 3.2 3B and Ministral 3B deliver fluent analytical prose above 31 tokens/second. The Apple M4 chip achieves server-grade decoding exceeding 88 tokens/second across all 3B models, maintaining zero outbound network transmissions.

To check A18 Pro throughput and LPDDR5X memory bandwidth saturation, see the benchmarks for local LLMs on iPhone 16 Pro.

Navigating iOS Memory Budgets Without Server Offloading

When operating completely offline, cloud offloading is unavailable to rescue an app facing memory pressure. On desktop systems like macOS, an oversized model or context window can spill into virtual swap files on disk. iOS explicitly disables swap files on NAND flash storage to prevent premature wear on flash memory cells and maintain consistent 120 Hz ProMotion display frame pacing. As a result, the Darwin kernel manages memory aggressively through the Jetsam subsystem.

On an 8 GB device such as the iPhone 15 Pro or iPhone 16 Pro, the operating system limits a foreground application's anonymous dirty memory ceiling (phys_footprint) to approximately 4.8 GB to 5.2 GB. If total memory allocation crosses this boundary even for a single millisecond during tensor allocation, Jetsam immediately sends a non-catchable EXC_RESOURCE (RESOURCE_TYPE_MEMORY) kill signal. To sustain robust offline inference, three memory safeguards are essential:

• Affine 4-Bit Weight Quantization: Compressing FP16 weights into 4-bit integer representations using group size 64 reduces physical weight size by ~72%, bringing 3B models well under 2.1 GB of RAM.
• Dynamic KV Cache Compaction: Pre-allocating attention buffers and pruning stale context tokens prevents the linear RAM expansion that turns multi-turn chat sessions into memory hazards.
• Hardware Memory Headroom Polling: Continuously inspecting os_proc_available_memory() allows the inference loop to gracefully finalize tokens or shed cached buffers before approaching kernel intervention thresholds.

To select the ideal compact model for offline reasoning, review the best open source models for iPhone.

Engineering Checklist: Running Offline LLMs on iOS

To implement high-throughput, completely disconnected local LLM inference on iOS using Apple MLX, adhere to the following architectural guidelines:

1. Strip Network Permissions: Remove com.apple.security.network.client from Entitlements.plist to enforce zero-egress hardware isolation at the kernel level.
2. Standardize on MLX 4-Bit Format: Quantize model weights using 4-bit affine compression with group size 64 to maintain high perplexity while keeping memory footprints under 2.5 GB.
3. Use Zero-Copy Shared Buffers: Allocate all attention weights and tensor buffers with MTLResourceStorageModeShared so CPU tokenization logic and GPU matrix multiplication share unified memory without duplication.
4. Enforce Hardware Encryption at Rest: Protect offline model weights and conversation databases using iOS Data Protection Class A (NSFileProtectionComplete), ensuring Secure Enclave key erasure upon screen lock.
5. Monitor Available System Memory: Check os_proc_available_memory() prior to launching prefill passes, enforcing a conservative safety margin of at least 1.2 GB of free physical RAM.

To integrate these models into a continuous assistant pipeline with local tools and memory, explore the architecture of private AI assistants running LLMs locally.

The Lapis model catalogue shows compatible options before you choose a download.