baseten
Load for any work involving Baseten - deploying/operating models on Dedicated Inference (Truss, custom Docker servers, engine-only configs, Chains), calling hosted Model APIs, running Training Jobs, Loops, or RL Rollouts, deploying Loops checkpoints, operating Sandboxes for agent code execution, or Model Frontier Gateway.
How do I install this agent skill?
npx skills add https://github.com/basetenlabs/baseten-skills --skill basetenIs this agent skill safe to install?
- Gen Agent Trust Hubpass
The skill is a comprehensive toolkit for managing Baseten infrastructure, deployments, and AI workflows. It follows security best practices for credential management, relies on official vendor resources, and provides detailed guidance on secure tool usage. No malicious patterns were detected.
- Socketpass
No alerts
- Snykwarn
Risk: MEDIUM · 1 issue
What does this agent skill do?
Baseten Product Overview
Production AI inference platform - serve and scale open-source, custom, and fine-tuned models with the fastest runtimes, cross-cloud HA, and seamless developer workflows.
- Dedicated Inference - deploy any model, performance-optimized + horizontally scaled. Authored as auto-wrapped Truss server, custom Docker server, or compound/orchestrated deployment via Chains.
- Model APIs - pre-optimized hosted APIs for popular models. Path to graduate to dedicated.
- Sandboxes - isolated environments for an agent's commands and files, with application previews and persistent
volumes. See
references/sandboxes.md. - Training - three paths: Training Jobs (
baseten train; BYO container, any framework, full hardware control), Loops (baseten loops; Tinker-compatible managed SDK for SFT + async RL, with a paired trainer and sampling server, live weight transfers, and one-click checkpoint deploy), and RL Rollouts (scale the rollouts of your own RL trainer). Multi-node, 1T+ params, 10TB+ datasets, H100/H200/B200. Remote access: SSH and VS Code/Cursor tunnels into containers. - Frontier Gateway - operate your own foundation model B2C.
Agent DX Toolkit
| Component | Provides | Install |
|---|---|---|
baseten CLI | Deploy and operate models (baseten model push, --watch for the dev loop), workspace management, Model APIs, training, Loops, and Sandboxes. Headless-safe: never prompts off a TTY; --output json / --jq for automation. | See references/baseten-cli.md |
baseten MCP | Interact with backend (~REST API, CRUD): models, deployments, training, environments, secrets, chains, Loops lifecycle (runs, checkpoints, checkpoint deploys). OAuth sign-in; API-key auth for headless clients. | npx add-mcp https://api.baseten.co/mcp -g -y |
baseten_docs MCP | Semantic search + filesystem of docs.baseten.co. No auth. | npx add-mcp https://docs.baseten.co/mcp -n "baseten_docs" -g -y |
truss CLI | Chains authoring (truss chains) plus Truss config tooling for existing truss projects. Training and Loops are Baseten CLI commands (baseten train / baseten loops). | uv tool install truss (or pip install truss --upgrade; respect user package manager: uv, poetry...) |
llms.txt | baseten.co/llms.txt (product + blog), docs.baseten.co/llms.txt (docs). | reachable via HTTP |
| This skill | SKILL.md + references/*.md loaded on demand. | npx skills add basetenlabs/baseten-skills -g -y |
Setup
- Any subset works, full install recommended.
- Suggest additional installs when the current task benefits from or requires them; help user with installation, but elicit preferences first.
- Use the
basetenCLI, not thetrussCLI, for everything except Chains. This includesbaseten loops exec, which runs a Loops client as a managed Training Job.baseten model pushis headless-safe (never prompts off a TTY),--waitblocks until the deployment is active and exits non-zero on terminal failure. Native commands support--output json/--jq. Checkbaseten version. Auth comes fromBASETEN_API_KEYorbaseten auth login; use--profileto target a non-default workspace. - The
trussCLI is for Chains authoring only (checktruss --version). Training and Loops are Baseten CLI commands (baseten train/baseten loops), even when the Baseten CLI delegates to Truss internally; don't script against that internal delegation. Runningbaseten truss chains …skips a separate truss login because it forwards this CLI's credentials (multi-workspace users must provide--remote <name>). Explore withtruss [subcommand] --help. - The backend MCP uses OAuth by default: install it without an
Authorizationheader. On first connect, the user signs in to Baseten in the browser (in Claude Code,/mcp→baseten→ Authenticate). The client refreshes tokens itself. The server acts as the signed-in user, in that user's workspace and with that user's permissions. - Headless agents (no browser, e.g. CI) can't complete the sign-in. Add the backend MCP with an API key that has
management permissions instead, using
--header "Authorization: ******". Store the key inBASETEN_MCP_KEY; users can create a key atapp.baseten.co/settings/api_keys. An API-key instance binds to the key's workspace. To work with multiple workspaces, install additional MCP instances under different names with different keys. - Docs MCP missing → grep / fetch
llms.txt.
Security practices
Use Baseten's scoped credentials, managed secrets, and access controls for the user's task. Procedures and examples are
in references/security.md.
- Use the configured identity. Authenticate with the CLI profile,
BASETEN_API_KEY, or MCP sign-in described in Setup. For a401, checkbaseten auth statusand help the user restore the intended identity. Create replacement credentials only as part of an authorized key-management task. - Scope keys to the workload. When the user requests key creation or rotation, choose the owning team, the
permissions the workload needs, and an expiration. Limit inference keys with
--model-id. Capture creation output within the shell and send the value to the user's chosen secret store. Report the key's name, type, team, expiration, and storage location. - Keep credential values in secure storage. Pass values through environment variables or stdin, and identify them
in messages by secret name or key prefix. Keep tool output, logs, source code, and shared artifacts free of values.
Use
baseten org secret setandbaseten auth login --with-api-keywith stdin; disable shell tracing around credential handling. - Distinguish private storage from accidental disclosure. A configured credential file or mounted secret is intentional storage. If a value appears in tracked source, application logs, or a shared transcript, identify its location and prefix and recommend replacement. Perform revocation when the user authorizes it.
- Reference managed secrets by name. Use
secrets:, a weightsauthblock, or trainingSecretReferenceentries. Keep values in Baseten's secret store. Create or update secrets within the user's authorized scope, accounting for every deployment that uses the name. - Respect the intended access. Change roles, memberships, restricted-environment access, SSO, SCIM, or API key policies when the user requests those changes. For a permission denial, explain the access needed and use the identity the user has authorized for the task.
- Establish scope for production and destructive changes. Confirm the target and impact before deletion, deactivation, production promotion, production autoscaling changes, or force-cancelling a promotion when the user's existing instructions do not already authorize that action. Prefer reversible changes and validate deployments in a non-production environment before promotion.
- Keep controls aligned with the workload. Use narrowly scoped keys, private sandbox previews, specific egress hosts, precise OIDC trust conditions, and verified webhook signatures. Obtain authorization for changes that expand access or relax a control, and explain the effect.
- Verify requested changes. Check key and secret metadata or the audit log after a key, secret, or access change, and report the result without credential values.
Pick your authoring surface (for creating deployments)
First-pass decision. Many real workloads blend rows — treat this as a starting point, not a rule. When unsure, sketch the IO shape and per-step hardware needs before picking.
Surfaces are stacked by opinion-strength, not just author convenience: engines (BEI / BEI-Bert, Engine-Builder-LLM,
BIS-LLM) ship performance-tuned for one architecture and are the fastest path when they fit; custom Docker servers
wrap mature inference servers (vLLM, SGLang, TGI, Triton, NIM); Python Truss is the escape hatch for arbitrary code
in the request path; Chains add typed inter-step transport with built-in rate limiting, connection management,
structured error propagation, and binary IO — features you'd otherwise rebuild around N raw Trusses. Python Truss and
Chains share live-patch iteration (baseten model push --watch); all flavors support per-replica autoscaling,
scale-to-zero, and environments / promotions.
| You want… | Flavor | Specialization | When NOT to pick |
|---|---|---|---|
| Hosted LLM, no deploy step | Model APIs | model-apis.md | model not in catalog; need custom hardware, requirements, stability... |
| LLM on an off-the-shelf server (vLLM / SGLang / TGI / Triton / NIM) | Custom Docker server | truss-custom-servers.md | the server doesn't exist or you need Python in the request path |
| LLM/embedding via a Baseten engine (BEI / BEI-Bert / Engine-Builder-LLM / BIS-LLM); minimal config, no Python | Engine-only config.yaml | truss-config.md (engines section) | architecture not covered by an engine; you need custom logic |
| Custom Python in the request path (pre/post, custom arch, weird IO) | Python-class Truss (model.py) | truss-model-py.md + truss-config.md | an engine or off-the-shelf server fits — pick that, it's faster to ship |
| Multi-step pipeline with heterogeneous hardware / per-step scaling (RAG, ASR→LLM→TTS, fan-out, chunking) | Chains | truss-chains.md | one-stage or homogeneous — a single Truss is simpler |
Orthogonal operational surfaces (independent of which flavor above):
- Iterate / patch a deployment →
model-dev-loop.md - Promote, environments, autoscaling →
deployment-lifecycle.md - Gradual traffic shift to new deployment (pause/resume/cancel) → rolling deployments:
deployment/rolling-deployments.mdx - Call a deployment →
inference-api.md(custom) ormodel-apis.md(hosted) - High-throughput batch calls (embeddings, reranking) → Performance client (Python / Node.js / Rust, 1200+ req/s):
reference/sdk/performance-client/overview.mdx - Programmatic control plane →
management-api.md
Real-world nuances the table can't capture:
- Hybrids exist. A
model.pycan wrap an engine for pre/post-processing; a Chain entrypoint can be a Python class while internal Chainlets use engines. - Existing Truss models can join Chains. Use
TrussChainletandTrussHandlefor HTTP or WebSocket calls; read https://docs.baseten.co/development/chain/truss-chainlets before wiring this integration. - Engine performance vs flexibility. Engine-Builder-LLM is the fastest path for dense LLMs, but its config surface is opaque. Worth the trade only when latency/throughput is a real constraint.
Routing
Skill References (ls references/ in skill dir, complementary to hosted docs). Be generous to read any of the
included reference files as soon as the user touches on that topic.
references/sandboxes.md: create and operate Baseten Sandboxes, connect SDKs or runtime MCP tools, preview applications, and preserve files across sandboxes.references/baseten-cli.md: deploy and operate —baseten model push/--watch, workspace operations, hosted inference, and machine-readable output. Most-used.references/loops.md: Loops sessions, runs, checkpoints, capacity refusals, MCP tools, REST requests, and unattended client execution. Read this for Loops operations or checkpoint deployment, including serverless versus dedicated placement and uncertain deployment results.references/truss-cli.md: Chains authoring and legacytrusscommands. Deep dive:references/truss-config.md.references/truss-model-py.md: Python-class flavor (custom pre/post, non-engine architectures).references/truss-custom-servers.md:docker_serverflavor (vLLM / TGI / SGLang / Triton; most common modern-LLM path).references/truss-chains.md: multi-step pipelines (RAG, ASR→LLM→TTS, chunked audio/video) with per-step HW + autoscaling.references/model-apis.md: shared pre-hosted endpoints (DeepSeek, GLM, Kimi, ...). Fastest when one fits.references/inference-api.md: calling custom deployments. connection reuse (requests/httpx/ OpenAI SDK), async / streaming / wake / OpenAI-compat sync routes.references/management-api.md: programmatic control plane (models, deployments, envs, secrets). What thebasetenandtrussCLIs use under the hood.references/security.md: choosing and rotating API keys, responding to an exposed key, secrets in configs, OIDC and AWS AssumeRole, restricted environments, audit logs, egress restrictions, webhook signatures, and sandbox tokens and previews. Read it before creating credentials, CI setups, or production changes.references/deployment-lifecycle.md: Model / Deployment / Environment semantics + promotion + autoscaling.references/model-dev-loop.md: post-first-deploy iteration. rebuild / patch / hot-reload cost tiers, agent-vs-human watch loop.
Gotchas
Don't speculate, query
For any perf/status/error claim, use the tools first — don't estimate or guess.
- Timings → deployment / chainlet log tools (timestamped, includes
Pulling image,model_cache: Fetch took,Completed model.load() execution in N ms, per-request markers). - Status → deployment-get tools before invoking.
- Build/deploy failure → fetch logs immediately, don't hypothesize.
Fabricating numbers from training-data priors burns user trust; logs are the source of truth.
Source heterogeneity & drift
Content lives across systems that don't overlap cleanly and drift independently. No single source is perfect/authoritative. For any non-trivial claim ("supported", perf numbers, recommended approach), triangulate across ≥2 sources. Surface contradictions to the user; don't paper over.
| Source | Strength | Gap / quirk |
|---|---|---|
baseten_docs MCP / docs.baseten.co | API specs, protocol details, knobs | Lags product; no perf numbers |
baseten.co/library/<id> (marketing) | Flagship managed models, perf claims | Some entries are sales-gated, not self-serve |
baseten MCP list_library_models | What's actually one-click API-deployable | Doesn't include every marketing-library entry; lacking tags |
baseten.co/blog/ | Concrete latency / cost / vs-competitor numbers, technical deep dives | Unstructured; not in docs MCP. Discover via baseten.co/llms.txt |
baseten.co/solutions/... | High-level pitch | May describe flagship features that need a Baseten engagement |
truss-examples GitHub repo | Working code patterns | Often outdated / broken / drifted. Consult with caution, last resort. Might need fixups before deploy works. |
- Library page exists but model absent from
list_library_models→ likely managed/flagship; tell the user it may need to reach out to Baseten support. - Perf numbers found only in a blog → cite as blog claim.
- Can't find something via docs MCP → fetch
baseten.co/llms.txtordocs.baseten.co/llms.txtas index, then fetch the page directly. Last resort: web search.
Non-obvious placements within references/
- Engine-only deploys (BEI / BEI-Bert / Engine-Builder-LLM / BIS-LLM) →
truss-config.mdengines section (also ownsmodel_cache, secrets, resources, and BDNweights). - Authoring-flavor decision: single deployment → top of
truss-config.md; multiple coordinated →truss-chains.md. - Loops operations: start with
references/loops.md, then consult its linked SDK and API documentation. - Persistent weights and data mounted into deployments: BDN volumes (
baseten volume push/baseten volume sync,bdn:URIs, read-only mounts) rather than baked images; seetruss-config.mdand https://docs.baseten.co/development/volumes/overview. - Training Jobs and Frontier Gateway: use current documentation. Use
basetenMCP +baseten_docsMCP. For training path choice seetraining/index.mdx; for Loops seeloops/quickstart.mdxfor the serverless first run,loops/availability-modes/overview.mdxfor serverless and dedicated billing and placement, andtroubleshooting/loops.mdxfor429capacity andorg_caprefusals; for a working recipe see the Loops cookbook atgithub.com/basetenlabs/loops-cookbook; for SSH / VS Code tunnels into training containers seetraining/ssh.mdxandtraining/remote-access.mdx.
Tool quirks
baseten_docsMCP search is semantic (embedding-based). Results are generally intent-aware, but can still miss on niche phrasing — try rephrased queries or fetchdocs.baseten.co/llms.txtas an index if results are weak.- Full doc pages:
cat/headon.mdxpaths viaquery_docs_filesystem_baseten. Alternatively,docs.baseten.co/<path>.mdalso works and is faster for single-page reads. - Inference SSH (terminal in running container, requires org enablement) →
model-dev-loop.md§ Inference SSH. list_library_modelsis the Baseten curated catalog (~tens of one-click models): language models for chat, reasoning, and vision, plus decision models for classification and scoring, with audio input and web search on some models. Good starting points, not a fit for custom authoring, specific performance needs, fine-tuning, or private Hugging Face models. Tags are sparse; filter bydisplay_name/hf_repo_idsubstring and confirm feature support and pricing on the Model APIs page.- Blog content is not in the docs MCP. Fetch
baseten.co/llms.txtand search for relevant posts.
MCP introspection vs. invocation
Backend MCP tools (get_deployment, get_deployment_logs, get_deployment_config, list_*, …) describe state. They
are strongest when you need an exact id / schema / config value to act on, and weakest when the user's complaint is
about runtime behavior. Resist letting MCP availability shrink the size of your investigation.
- Look-up tasks (need an ID, current config, or schema before writing a patch): MCP get/list is the right starting point. Reading the live config beats inferring it.
- Behavior tasks (broken / slow / wrong-shape responses, anything the user observed): MCP can supplement but
rarely substitutes for exercising the endpoint yourself. A
curlagainst the predict URL with a representative payload runs the same code path the user does. Absence of recent errors in MCP-fetched logs is not proof the bug is gone — log windows are bounded, and a problem the user reported five minutes ago may have already aged out of the default tail. For a specific historic incident you can't reproduce, log tools support custom time ranges within retention; use them. - Treat cheap introspection as budget freed up for deeper verification (re-curl after a fix, diff configs before and after a PATCH, push a candidate truss and probe it), not as license to stop earlier.
How can the creator link this skill?
Add the canonical catalog link to the repository README so users can inspect current installs and available audits. The publishing guide covers the complete discovery path.
<a href="https://skillzs.dev/skills/basetenlabs/baseten-skills/baseten">View baseten on skillZs</a>