Capacity

This page describes what the single node physically provides: GPU compute and memory, system RAM, and on-node storage. These are the fixed resources the workload-governance framework allocates across the identified use cases.

GPU compute

The node carries eight NVIDIA H200 SXM5 GPUs on a single HGX baseboard.

Because the eight GPUs share a full-mesh NVLink fabric, multi-GPU training jobs can treat them as a tightly-coupled unit rather than eight isolated accelerators – the interconnect is fast enough that gradient synchronisation is not the bottleneck. For inference, GPUs are subdivided logically through time-slicing, so a single physical GPU can serve several concurrent lower-intensity workloads.

GPU memory as the binding constraint

For inference, the practical ceiling is how many model instances fit in 1,128 GB of HBM3e at once, not raw FLOPS. The identified use cases span a wide memory footprint:

• Lightweight forecasting / geospatial models – well under 1 GB of VRAM each.
• ASR (Whisper-large-v3) – ~1.5 GB per pipeline.
• Sub-20B language models – roughly 9–28 GB depending on model size and whether weights are served at FP16 or quantised to INT8.

Quantisation (INT8) and vLLM continuous batching are the two levers that raise how much concurrent demand a fixed pool of GPU memory can absorb.

System RAM & CPU

Several use cases are CPU- and RAM-bound rather than GPU-bound – geospatial ML, data-pipeline preprocessing, audio pre-processing for ORF, and the forecasting models behind liquidity management. The 2 TB of RAM and 224 threads give substantial headroom for these workloads to run alongside GPU jobs without contending for accelerator time.

On-node storage

The 7.6 TB scratch is finite, so datasets are staged per training window (load → run → archive) rather than held resident. Data logistics – not compute – dominate the storage-heavy use cases such as health-record digitisation.