Start here
Intended reader: Creators and developers building a dedicated local machine for generative image/video diffusion and local LLM inference. Practical outcome: A hardware dimensioning worksheet covering VRAM, PCIe lane bifurcation, power headroom, and system RAM ratios. A graphics card can look perfect on a specification sheet and still be wrong for the model you want to run. Before building a shopping cart, write down your actual workflow, output size and waiting-time limit.
Specify the workload first
List the exact model, precision, context or image dimensions, and concurrency you expect. A workstation for one image at a time has a different budget from one serving several long conversations. Ask whether the software loads your intended configuration and whether its measured throughput meets the job.
Capacity and speed answer different questions
NVIDIA lists the RTX 4090 with 24 GB of GDDR6X memory. Capacity is a useful constraint, not evidence that every 24 GB workflow performs equally well. Account for runtime memory beyond stored weights. Do not compare cards using a model-size number alone.
| Check | Evidence to use | What it does not prove |
|---|---|---|
| GPU memory capacity | Manufacturer specification; NVIDIA lists 24 GB for RTX 4090 | That every model or context size fits |
| Runtime memory | Measurements with your exact model, precision and concurrency | That stored weight size equals peak memory use |
| Turnaround | Comparable loading and repeated-run timings | A universal fastest GPU |
| Multi-GPU behavior | Application documentation and a reproducible test | Automatic memory pooling or doubled speed |
Offloading is a software strategy
Hugging Face Accelerate documents distributing large-model weights across GPU, CPU and disk when needed. Placement lets a workload use several memory tiers; it is not the same as giving one GPU a larger allocation. The actual framework and model must support your plan.
Before buying a second GPU
Treat a dual-card machine as an integration project. Start with the inference engine and end with the physical build. Do not buy an interconnect on the assumption that its presence makes every model use both cards.
- Confirm application support for the required multi-GPU mode.
- Check motherboard slot spacing and electrical lane allocation.
- Use the card maker's power, connector and clearance requirements.
- Allow airflow around both cards and the rest of the system.
- Find a repeatable result for the exact configuration, or test comparable rented hardware.
Budget for a month of real work
Compare acquisition cost, electricity, storage and maintenance time with the cloud spend you expect to avoid. For a small irregular workload, ownership may save less than a headline hourly comparison suggests. For a repeatable daily workload, measure utilization and accepted output. This guide makes no benchmark or payback-period claim.
A purchase checklist you can actually use
Bring a written workload to the purchase decision: required files, supported runtime, a known working command or graph, expected volume and an acceptable turnaround. Check current local prices and warranty terms. Keep enough budget for the supporting machine instead of spending everything on a card that the rest of the system cannot accommodate.
- Estimate system RAM from the exact workload, operating system and intended offloading behavior.
- Choose PSU capacity and connectors from component requirements and manufacturer guidance.
- Check motherboard electrical lane allocation and software support before planning multiple GPUs.
- Estimate storage from model files, datasets, working space and backup needs.
- Measure case clearance, slot spacing and cable routing against component documentation.
Write three workloads before making a shopping list
Complete the worksheet for an everyday task, a demanding job and a possible future task. This is our planning method, not a hardware benchmark. Keep unknown requirements visible until a comparable test is available. Choose for work you need today; assess future expansion as a separate budget decision.
| Workload | Configuration to record | Evidence to collect | Purchase decision |
|---|---|---|---|
| Everyday task | Exact model or graph, precision, context or image dimensions, batch size and concurrency | Successful loading, loading time and repeated-run time | Establish required capacity and acceptable turnaround |
| Demanding job | Larger configuration you actually expect to use | Whether it loads, whether offloading is supported and whether turnaround is acceptable | Decide whether its frequency justifies extra hardware |
| Future task | Intended model or workflow and expected concurrency | Runtime support and a reproducible comparable result | Assess expansion separately from current purchase |
Ask for evidence you can compare
When evaluating a benchmark, look for the complete configuration and workload. Model version, precision, context length or image dimensions, batch size and software settings can change the meaning of a speed number. A result without this context is a lead for further research, not a purchase guarantee. Prefer a test you can reproduce using the same job you need to run. Keep loading time and repeated-run time separate if both affect your working day.
Make the purchase reversible where possible
Check the seller's return terms and the physical compatibility of the whole build before ordering. List case clearance, power connectors, cooling space and the expansion slots your actual configuration will use. Plan where models, project inputs and backups will live. After assembly, run the smallest known working workflow, then your everyday workload, before adding the demanding one. Record failures and actual operating costs during this trial so the next upgrade follows evidence rather than a specification headline.
Upgrade or keep the machine? Use this decision path
A successful small model does not establish that a larger purchase is necessary. Diagnose the limiting resource before replacing the machine. Use the RTX 4050 local LLM setup for a small-model starting point, then the VRAM reduction guide for context and offloading trade-offs.
| Your result | Next decision | Evidence needed before buying |
|---|---|---|
| Workload loads and finishes within your limit | Keep the current machine | Repeatable acceptable outputs |
| Workload does not fit | Test supported quantization, shorter context or offload | Exact configuration and measured memory where available |
| Workload fits but runs too slowly | Compare the same job on candidate hardware | Separate loading and warm-run timing |
| You need multiple simultaneous users | Evaluate the intended serving runtime and concurrency | Load test with the required request mix |
| Usage is occasional | Compare ownership with hosted use | Real monthly volume and total operating cost |
Try this next
Complete the workstation around the work you do every day: check monitor readability before chasing refresh-rate numbers. Read A 4K monitor for coding: check text before refresh rate.
Frequently asked questions
Does more VRAM guarantee faster local AI?
VRAM capacity and processing speed answer different questions. Enough memory helps a workload fit, but speed must be evaluated with comparable model settings, precision, context or resolution and concurrency.
Can two GPUs automatically combine their VRAM?
Do not assume this. The application must support the required multi-GPU mode. Confirm software behavior and motherboard lanes, slot spacing, power and cooling before buying.
Can I run a model that does not fit entirely in GPU memory?
Supported runtimes can offload model weights to CPU memory or disk. Hugging Face Accelerate documents this approach, including transfer overhead. It does not increase the physical memory of one GPU.
How should I compare build prices across countries?
Use a complete local parts quote with currency, tax, shipping, warranty and availability. Compare the same workload and configuration. Use the country-and-budget planner as an estimate, then confirm current retailer prices.
Sources & further reading
Primary sources checked Sep 12, 2026. Vendor statements are attributed; editorial advice is our own.
- 1
- 2Big Model Inference ↗Hugging Face
Help us keep this useful. Report a factual correction →
