NVIDIA DGX Spark in the Homelab: Local AI From a 2U Rack Mount

The NVIDIA DGX Spark is the first box that makes "run the big model at home" a realistic homelab project instead of a multi-GPU science fair. It is a 150 x 150 x 50.5 mm machine, about 2.6 lbs, built on the GB10 Grace Blackwell Superchip: a 20-core Arm CPU and a Blackwell GPU sharing one 128 GB pool of LPDDR5x memory. That shared pool is the whole point. A model that would need two or three 24 GB graphics cards and a lot of sharding just loads.

This post covers what the Spark is actually good at for local AI, how it fits next to the rest of a homelab, and how to rack it so it keeps running at full tilt instead of cooking on a desk shelf.

The mount for this build:

What the DGX Spark is good at

The honest summary: capacity first, speed second. The 128 GB pool means you can hold a 70B model at a sensible quantization with room left for a long context window, or a 120B-class mixture-of-experts model, on one quiet box. NVIDIA rates it for models up to roughly 200 billion parameters for inference, and up to 1 petaFLOP of FP4 compute.

What it is not is a 5090 in disguise. Memory bandwidth is roughly 273 GB/s, which is a fraction of a high-end discrete card. Token generation on a model that would also fit in a gaming GPU's VRAM is slower on the Spark. Where it wins is the model that does not fit anywhere else you own. If your current setup is a 24 GB card running a 30B model and you keep wishing for the 70B, this is the box that closes that gap.

Good fits for it in a homelab:

• A private chat assistant on a 70B-class model, served to every device in the house.
• A local coding assistant for your editor, with a long context that actually holds a repo.
• Retrieval over your own documents: paperless archives, notes, manuals, without any of it leaving the network.
• Fine-tuning and LoRA work on mid-size models, which is what NVIDIA built the software stack around.
• Image and video generation pipelines that run out of VRAM on a consumer card.

The software side

It ships with DGX OS, which is Ubuntu with NVIDIA's drivers, CUDA and container tooling already set up. That matters more than it sounds. Arm plus a new GPU generation is usually where homelab projects die in dependency hell, and here most of that work is done for you. NVIDIA also publishes step-by-step playbooks for the common jobs.

The stack most people land on is familiar:

• Ollama or llama.cpp for the simplest way to pull and serve quantized models.
• vLLM when you want higher throughput for several users or agents hitting it at once.
• Open WebUI as the chat front end. It can run on the Spark or on an existing Docker host, pointed at the Spark's API over the network.
• ComfyUI for image work.

The pattern that works well in a rack: treat the Spark as an inference appliance. Run it headless, give it a static IP, expose an OpenAI-compatible API, and let the services you already run (Home Assistant's voice assistant, a notes app, your editor) talk to it like they would a cloud endpoint. Your Proxmox host keeps doing Proxmox things. The Spark does one job.

Networking: 10GbE now, QSFP later

The rear panel has a 10GbE RJ45 port, four USB-C ports (one of them is power), HDMI 2.1a, and two QSFP cages on a ConnectX-7 NIC rated at 200 Gb/s. Wi-Fi 7 is in there too, but a box serving models to the whole house belongs on a wire.

For one Spark, the 10GbE port is plenty. Prompts and responses are tiny; the only time you move real data is pulling model weights, and a 40 GB download over 10GbE is a coffee break. If your switch tops out at 2.5G, it still works fine, just slower on the first pull of each model.

The QSFP ports are for linking two Sparks. Cable two together and you get 256 GB of combined memory, which NVIDIA rates for models up to roughly 405 billion parameters. That is the upgrade path, and it is worth planning the rack around even if you only own one today.

Power and heat

Power comes from a 240 W external brick into a USB-C port. The GB10 chip itself is rated around 140 W, so the brick has headroom. Idle draw is low; a long generation or fine-tuning run is where it climbs. Size your UPS around the load number, not the idle one, and give the brick a home. It is not small, and a dangling brick behind a rack is how cables get yanked.

Cooling is front to back. The Spark pulls air through the metal-foam mesh face and pushes it out the other end, and NVIDIA asks for clear space at both ends (around 4 inches at the intake and considerably more at the exhaust). On a desk that usually means a wall or monitor arm ends up in the way. In a rack it means planning, which is easier:

• Do not put a solid blank or a closed panel directly against either face.
• Keep the rear of the rack open, or at least vented, behind it.
• Do not park it in the U directly above a hot PoE switch if you can avoid it. Heat goes up.

Under sustained load it is audible but not obnoxious, closer to a busy mini PC than a server. Once it lives in a rack in a closet or a utility room, you stop hearing it at all.

Racking it: 10-inch or 19-inch

The Spark is 50.5 mm tall, which is more than 1U (44.45 mm). Our mounts put it in 2U (88.9 mm) with the unit seated on the mount floor and the cage open lattice across the top, bottom and back, so nothing closes off the airflow path. Both versions are printed in PETG, not PLA, because PETG holds its shape wrapped around a machine running this warm.

In a 10-inch mini rack, the 10" NVIDIA DGX Spark Mini Rack Mount - 2U puts the Spark in the same small enclosure as your gateway, switch and NAS. It has an optional keystone holder on one or both sides, with jacks and cables included, so you can bring a network drop straight out to the front of the rack.

In a full 19-inch rack, the 19" NVIDIA DGX Spark Rack Mount - 2U (Modular) comes as a single with a blank half, or as a double that holds two Sparks side by side in one 2U row. The double is the one to look at if the QSFP pairing is in your future: both units sit right next to each other, so the link cable is short. On the single, the blank half swaps for any other 2U mount in our Modular Series later.

One note if you are shopping GB10 boxes: the ASUS Ascent GX10 uses the same chip but sits taller, so it has its own mounts. The 10" ASUS Ascent GX10 Mini Rack Mount - 2U is the 10-inch one. Dell, HP and Lenovo GB10 machines have not been tested in ours, so ask before ordering for one of those.

A sample local AI rack

Here is a layout that works in a 10-inch mini rack, top to bottom:

• 1U patch panel or keystone row for the drops coming in.
• 1U switch with at least one 10GbE port for the Spark (a 2.5G switch works if that is what you have).
• 1U vented blank, so the switch is not breathing straight into the next device.
• 2U DGX Spark.
• 1U or 2U mini PC running Proxmox: Open WebUI, Home Assistant, and whatever else talks to the Spark.
• Shelf at the bottom for the Spark's power brick and a small UPS.

The same idea scales to a 19-inch rack, with the difference that a second Spark slots in next to the first on the double mount without costing another 2U of height. Either way, the Spark sits in the middle of the stack where the cable runs are short, and the hot gear is spread out instead of stacked.

Is it worth it for a homelab?

If your homelab is containers, a NAS and a few VMs, no. A used mini PC does all of that for a fraction of the cost, and you can still run small models on it.

If you already run local models and you are bumping into VRAM limits, the Spark is the cleanest step up right now. One box, one power brick, real capacity, a supported software stack, and a clear path to a second unit. That is a much easier thing to live with than a tower full of graphics cards and a dedicated circuit.

Wrap-up

The DGX Spark trades raw speed for memory capacity, and for local AI at home that is usually the right trade. Run it as a headless inference box, put it on 10GbE, plan the rack so both faces breathe, and leave room next to it if a second unit is likely. Get those right and it is the quietest way to run a 70B model in your house.

Mounts in this build