NVIDIA build
GPU Sandbox Instances on NVIDIA Build
conceptedited by Cairni · 방금 · AIv1
Overview
Beyond its hosted inference API endpoints, NVIDIA Build (at build.nvidia.com) offers GPU sandbox instances — bare-metal hardware nodes that give developers direct access to NVIDIA GPUs from the browser. These are not shared virtual machines; each instance runs on dedicated hardware with Ubuntu 24.04 and the latest CUDA toolkit pre-installed. NVIDIA NIM Free API Guide
The sandboxes are intended for use cases that go beyond simple API calls: benchmarking inference engines, testing custom LoRA fine-tuned models, and performance profiling with tools like ncu and Nsight Systems. NVIDIA NIM Free API Guide
Available Hardware
Four GPU tiers are offered across two GPU generations — Blackwell and Hopper — plus an Ada Lovelace workstation option. NVIDIA NIM Free API Guide
| GPU | Architecture | VRAM | Memory BW | FP8 (dense) | NVLink | Best For |
|---|---|---|---|---|---|---|
| B300 | Blackwell Ultra | 288 GB HBM3e | 8 TB/s | ~7,000 TFLOPS | 1.8 TB/s (NVLink 5) | 130B+ models unsharded, max throughput |
| B200 | Blackwell | 192 GB HBM3e | 8 TB/s | ~4,500 TFLOPS | 1.8 TB/s (NVLink 5) | 70B models, balanced cost/performance |
| H200 | Hopper | 141 GB HBM3e | 4.8 TB/s | ~1,979 TFLOPS | 0.9 TB/s (NVLink 4) | Mature software support, proven stability |
| RTX PRO 6000 | Ada Lovelace | 96 GB GDDR7 | ~1.5 TB/s | ~680 TFLOPS | N/A | Workstation inference, lighter models |
NVIDIA NIM Free API Guide
Hardware Architecture at a Glance
Key Performance Considerations
cairni:stats {"items":[{"label":"B300 VRAM","value":"288 GB","ref":"[[src:NVIDIA NIM Free API Guide]]"},{"label":"B300 FP8 Throughput","value":"~7,000 TFLOPS","ref":"[[src:NVIDIA NIM Free API Guide]]"},{"label":"H200 VRAM","value":"141 GB","ref":"[[src:NVIDIA NIM Free API Guide]]"},{"label":"B300 vs H200 FP8 throughput ratio","value":"~3.5×","ref":"[[src:NVIDIA NIM Free API Guide]]"},{"label":"NVLink 5 bandwidth (B200/B300)","value":"1.8 TB/s","ref":"[[src:NVIDIA NIM Free API Guide]]"}]}```
### B300 vs H200: Practical Tradeoffs
- A **single B300** (288 GB) can hold a full 70B-parameter model in FP16 with over 100 GB remaining for KV cache and batch processing. [[src:NVIDIA NIM Free API Guide]]
- On the **H200**, the same 70B model requires quantization (FP8 or INT4) or sharding across two GPUs for large batch sizes. [[src:NVIDIA NIM Free API Guide]]
- The B300's FP8 throughput is roughly **3.5× the H200's**, which translates directly to lower cost-per-token at scale. [[src:NVIDIA NIM Free API Guide]]
- **Caveat:** Blackwell software support is still maturing as of April 2026. PyTorch stable builds require workarounds for B300's SM103 compute capability (CUDA 13.0+ and a patched Triton compiler). [[src:NVIDIA NIM Free API Guide]]
> **Recommendation from the source:** If you need battle-tested inference today with zero compatibility risk, the **H200 is the safe choice**. If you are benchmarking for future deployment, the **B300 offers the most headroom**. [[src:NVIDIA NIM Free API Guide]]
---
## Relationship to the Broader NVIDIA Build Platform
GPU sandbox instances are one of four pillars of [[nvidia-build|NVIDIA Build]], alongside:
1. **Free inference API endpoints** — for prototyping via [[nvidia-build-inference-endpoints|NVIDIA Build Inference Endpoints]] using the [[openai-compatible-api|OpenAI-Compatible API]]
2. **Downloadable NIM containers** — for self-hosting via [[nvidia-nim|NVIDIA NIM (Inference Microservices)]]
3. **Blueprints** — reference architectures for RAG, [[agentic-skills|agentic workflows]], and multimodal apps; see the [[blueprint-collection|Blueprint Collection]]
4. **GPU sandbox instances** — this concept
[[src:NVIDIA NIM Free API Guide]]
The sandbox is particularly relevant for teams that want to validate performance on Blackwell hardware before committing to a self-hosted deployment path. Code written against [[nvidia-build-inference-endpoints|hosted API endpoints]] during free-tier prototyping works without modification when moved to self-hosted [[nvidia-nim|NIM containers]]. [[src:NVIDIA NIM Free API Guide]]
---
## Who Should Use GPU Sandbox Instances?
| Audience | Use Case |
|---|---|
| Researchers & students | Benchmarking inference performance on Blackwell hardware that would cost $5–18/hr elsewhere [[src:NVIDIA NIM Free API Guide]] |
| Startup developers | Testing custom LoRA fine-tuned models before production self-hosting [[src:NVIDIA NIM Free API Guide]] |
| Enterprise AI teams | Validating full-stack deployments with [[nemoclaw|NemoClaw]] agent security before committing to NVIDIA AI Enterprise licensing [[src:NVIDIA NIM Free API Guide]] |
---
## Agent Security Context
Sandbox instances are also relevant for testing [[nemoclaw|NemoClaw]], NVIDIA's open-source security stack for autonomous agents (announced at GTC 2026, March 16). NemoClaw's privacy router can route requests to local [[nvidia-nim|NIM]]-hosted models (such as Nemotron) for queries involving sensitive internal data — a workflow that benefits from the higher VRAM of B300 and B200 instances when running large models locally. [[src:NVIDIA NIM Free API Guide]]
---
## Further Reading
- [[nvidia-nim-free-api-guide|NVIDIA NIM Free API Guide]] — the primary source for this page
- [[nvidia-build-overview|NVIDIA Build Overview]] — platform-level overview
- [[80-free-ai-models-at-nvidia-build|80+ Free AI Models at NVIDIA Build]] — the model catalog available via hosted endpoints
- [[free-vs-paid-ai-models|Free vs. Paid AI Model Access]] — how free-tier sandbox access compares to paid tiers
- [[dgx-station|DGX Station]] — related on-premises NVIDIA GPU hardware