hf-mem
Estimate the GPU memory needed to run a Hugging Face model (weights, KV cache and inference overhead) and whether it fits on given GPUs, without downloading it. Use when asked how much VRAM a model needs, whether it fits on a GPU, the longest context that fits, or which GGUF quantization to pick.
How do I install this agent skill?
npx skills add https://github.com/huggingface/skills --skill hf-memIs this agent skill safe to install?
- Gen Agent Trust Hubpass
This skill provides a utility to estimate the GPU memory required to run Hugging Face models. It leverages a specialized tool to calculate memory for weights and cache without requiring a full model download, which is a standard developer workflow.
- Socketpass
No alerts
- Snykwarn
Risk: MEDIUM · 1 issue
What does this agent skill do?
hf-mem
uvx hf-mem <model> -o json [options]
<model> is a Hub ID (org/name, org/name@revision), a Hub URL, a local path, or org/name-GGUF:Q4_K_M for one GGUF quantization.
Options:
--gpu-memory 24(GiB per GPU) or8x80(8 GPUs): addsfit.--max-model-len N: context length, defaults to the model's maximum, which can dominate the total.--batch-size N: concurrent sequences, defaults to 1.--kv-cache-dtype fp8(vLLM names for Safetensors, llama.cpp cache types such asq8_0for GGUF).--file <file or variant>and--format safetensors|gguf|onnxto pick the weights.--max-num-batched-tokens 2048on A100s.- Gated or private models need
HF_TOKENor--hf-token.
JSON (sizes in bytes, divide by 2^30 for GiB):
totalis the GPU memory for one copy of the model:weights+kv_cache+overhead(vLLM for Safetensors, llama.cpp for GGUF, none for ONNX).offloadedare weights the engine keeps in CPU memory: excluded fromweightsandtotal, but needed as free system RAM.fit(with--gpu-memory):fits,neededGPUs,max_model_len(the longest context that fits), and therequiredvsusablememory across the given GPUs. Userequired/ GPUs for the memory per GPU, nottotal/ GPUs, as each GPU also holds its own overhead and replicated KV cache.- GGUF and ONNX repositories with several files return a
fileslist to compare quantizations; pick one with:TAGor--fileto getfit. - Report
warningsto the user, and errors explain how to fix them.
How can the creator link this skill?
Add the canonical catalog link to the repository README so users can inspect current installs and available audits. The publishing guide covers the complete discovery path.
<a href="https://skillzs.dev/skills/huggingface/skills/hf-mem">View hf-mem on skillZs</a>