Gartner also estimates that AI PCs will become the norm by 2029.
Even when that day comes, running meaningful local AI workloads on client devices isn’tt a given, because not all AI PCs are created equal.
To understand AI PCs, we need to understand AI performance, starting with the TOPS metric, which stands for tera (trillion) operations per second.
But even a unified memory design is not enough to ensure high-performing local AI on a given client device when you consider how much RAM is required to run even slimmed-down AI models.
As we’ve explained, with memory constraints, these devices aren’t going to be very effective at running advanced AI models locally.
Now that major computer manufacturers are pushing “AI PCs” and often charging a premium for them, should businesses shell out the extra cash to buy them? On PC manufacturers’ websites, you’ll find no shortage of marketing literature claiming that AI-specific chips, known as neural processing units (NPUs), are necessary to unlock the true power of AI.
Dell says that “NPUs enable fasterOpens a new window , more efficient, and more power-conscious AI features right on the device,” while HP says NPU-equipped PCsOpens a new window help businesses “free up time to focus on what matters.”
But while NPUs can definitely help with AI workloads, organizations can’t just assume that having one guarantees that you can run a powerful LLM locally on a desktop or laptop at a reasonable speed, or even at all.
READ MORE: IT Job Watch: Cloud network specialist
Let’s explore what’s necessary to actually support different types of AI workloads on client devices and whether you can expect to do so at a reasonable price.
An NPU doesn’t guarantee fast local AI
Many of the hundreds of millions of PCs sold every year are marketed as AI PCs, which analyst firm Gartner defines as one “with an embedded neural processing unit (NPU).” According to Gartner’s 2025 estimatesOpens a new window , the number of AI PCs in the world is expected to grow rapidly, with an expected AI PC market share of 55% of PCs sold in 2026. Gartner also estimates that AI PCs will become the norm by 2029.
Even when that day comes, running meaningful local AI workloads on client devices isn’tt a given, because not all AI PCs are created equal. To understand AI PCs, we need to understand AI performance, starting with the TOPS metric, which stands for tera (trillion) operations per second. Often, manufacturers market how many TOPS their NPUs are capable of, but this is just one piece of the puzzle.
According to a Qualcomm blog post,Opens a new window “TOPS is a measurement of the potential peak AI inferencing performance.” But Qualcomm also points out that “a high TOPS number alone does not guarantee optimal AI performance; it’s the culmination of various factors working in tandem that genuinely define an NPU’s prowess.”
When Copilot+ PCs launched in 2024Opens a new window , they came with a minimum requirement of 40 TOPS to enable AI features such as live captions with translations, local real-time image generation, and enhanced video editing capabilities.
Smartphones also come equipped with NPUs: The latest iPhone 16 and iPhone 17 feature NPUs that can achieve 35 TOPS, facilitating sophisticated AI functionalities like local image generation. However, even devices with NPUs may face challenges when handling demanding AI tasks, particularly when memory limitations are present.
READ MORE: AI capabilities in VoIP/UC offerings: Ratings are a mixed bag (it’s complicated)
How memory can hold back AI PCs
When running AI workloads, the limiting factor is often memory, not just how much is available, but also the underlying memory architecture. An article on local models in IEEE SpectrumOpens a new window explains, “Powerful PCs typically have two pools of memory, system memory and graphics memory, which operate independently.” However, this split design is problematic for running local AI because “models require large amounts of memory, and the entire model must load into memory at once. The legacy PC architecture, which splits memory between the system and the GPU, is at odds with that requirement.”
In comments in IEEE Spectrum, Joe Macri, Vice President and CTO at AMD, expanded upon how legacy memory architecture introduces AI challenges: “When I want to share data between our [CPU] and GPU, I’ve got to take the data out of my memory, slide it across the PCI Express bus, put it in the GPU memory, do my processing, then move it all back,” which can lead to latency and sluggish performance.”
In simpler terms, using an older PC architecture means that having 16GB of fast VRAM alongside 16GB of slower system RAM won’t match the performance of 32GB of memory in a newer, AI-optimized computer. This is due to the slow data transfer between the different processors in your system, as well as the inherent performance differences between RAM and VRAM.
To get around these limitations, companies such as Apple, AMD, and NVIDIA have moved towards unified memory architectures optimized for AI, which place the CPU, GPU, and NPU on a single chip connected to shared memory via a high-bandwidth bus.
But even a unified memory design is not enough to ensure high-performing local AI on a given client device when you consider how much RAM is required to run even slimmed-down AI models.
Local AI models are memory-hungry
According to memory vendor CorsairOpens a new window , 8GB RAM is the entry point for small AI models coming in at up to 7B parameters and a high degree of compression, such as 4-bit quantization, which can reduce the quality of outputs. These small models, while relatively simple, are “capable for tasks like summarization, simple code generation, and conversational Q&A.”
Corsair calls 16 GB of memory the sweet spot and the realistic starting line for not feeling constantly constrained, 32 GB where you can start getting into more serious AI workloads, and 64 GB of RAM being the starting point for being able to work with cutting-edge, “frontier-class” AI models using quantization, but for the most accurate outputs, PCs are now available with 128 GB, 256 GB, or even 512 GB of unified memory.
With currently elevated memory prices, shelling out for high-end devices is expensive, and it’s going to be tough to convince an organization to splurge on hundreds or thousands of AI-optimized PCs for end users in the organization.
At the time of writing this, you could buy a business oriented laptop with a legacy (non-unified) memory architecture with a 40 TOPS NPU and 16 GB of (slower) system RAM for about $1300 on the Dell websiteOpens a new window . As we’ve explained, with memory constraints, these devices aren’t going to be very effective at running advanced AI models locally.
On the higher end of the AI PC spectrum, an NPU-enabled Macbook Pro with an M5 Max chip, 128 GB of unified shared memory (with fast interconnects between the CPU, GPU, and NPU), and 2 TB of SSD storage runs closer to $6,700Opens a new window . Similarly, a Windows laptop, the HP ZBook Ultra 14 inch G1a Mobile Workstation with a 50 TOPS NPU and 128 GB of unified memory, has a list price of around $7,500Opens a new window . The Mac Studio workstation with an M5 Ultra chip and 256GB of unified memory costs closer to $10,000.Opens a new window
Costs will likely come down, eventually
Clearly, you can see that procuring hardware to run cutting edge AI models locally is out of reach for many organizations today, or a privilege reserved for a select few power users.
Even with rising consumption-based AI prices, cloud-based AI subscriptions for ChatGPT, Claude, and Gemini are currently relatively inexpensive and fast for anyone with an internet connection when compared to shelling out six or seven thousand dollars for a highly capable client device.
That means from a cost perspective, it might not make sense to run AI workloads locally, unless there are privacy or compliance concerns about sharing data with cloud providers. For now, it makes sense for companies to purchase highly capable AI machines if they have legitimate use cases for AI, deep pockets, and they need to keep sensitive data local.
It’s also worth noting that companies like Apple also offer a lease option, with the $6,700 model referenced above leasing for approximately $150 a month for 36 months, which might make sense if replacing a higher-tier AI subscription.
In conclusion, even though AI PCs with limited local capabilities are already here, you can’t expect an experience comparable to advanced models from ChatGPT or Claude locally unless you shell out big bucks. That said, the math could change in the coming years as AI-optimized unified memory architectures become more common, NPUs become standard equipment on client devices, and memory eventually becomes more available in 2028Opens a new window (if analyst predictions are to be believed). After all, hardware prices have fallen rapidly over the long term since the beginning of computing.
