skillZs
★ LIVE SKILL TAGS ★
>>> LIVE SKILLS INDEX <<<
* OPEN SOURCE *
NO LOGIN, NO TRACKING
※ REAL INSTALL DATA ※
← back to all skills
docs.promptlayer.com57 installs

promptlayer

Use when managing LLM prompts, evaluating prompt changes, building multi-step AI workflows, logging and tracing LLM requests, or running evaluations against datasets. Reach for this skill when working with prompt versioning, release management, observability, or building evaluation pipelines.

How do I install this agent skill?

npx skills add https://docs.promptlayer.com --skill promptlayer
view source ↗

Is this agent skill safe to install?

No partner audit is available yet. Read the source before installing.

What does this agent skill do?

PromptLayer Skill

Product summary

PromptLayer is a platform for managing, versioning, and evaluating LLM prompts outside your codebase. It provides a Prompt Registry for storing and releasing prompt templates with version control, Tables for building evaluation pipelines and datasets, observability tools for logging and tracing LLM requests, and Workflows for orchestrating multi-step AI systems. Agents use the Python SDK (promptlayer), JavaScript SDK, or REST API (authenticated with X-API-Key header) to fetch prompts, run them against LLM providers, log requests, and execute workflows. Key files: prompt templates stored in Prompt Registry, Tables for evaluations and datasets, Workflows for multi-step orchestration. Primary docs: https://docs.promptlayer.com

When to use

Use this skill when:

  • Managing prompts: Creating, versioning, or updating prompt templates without code changes
  • Deploying prompts: Using release labels to control which prompt version production uses
  • Evaluating changes: Building evaluation pipelines to score prompt outputs before release
  • Logging requests: Tracking LLM calls, tokens, costs, latency, and associating them with prompts
  • Tracing execution: Following multi-step AI operations across LLM calls, tools, and application code
  • Building workflows: Creating multi-step AI systems with prompts, tools, code, and conditionals
  • Running evals in CI/CD: Testing agents or prompts against datasets in your pipeline
  • Organizing resources: Using folders, external IDs, and workspaces to structure prompts and workflows

Quick reference

Authentication

export PROMPTLAYER_API_KEY="pl_your_key"
export OPENAI_API_KEY="sk_your_key"  # or other provider key

Core SDK methods (Python)

TaskCode
Run a promptclient.run(prompt_name="my-prompt", input_variables={...})
Run with release labelclient.run(prompt_name="my-prompt", prompt_release_label="prod")
Run a workflowclient.run_workflow(workflow_id_or_name="my-workflow", input_variables={...})
Get a templateclient.templates.get("my-prompt", {"label": "prod"})
Log a requestclient.log_request(provider="openai", model="gpt-4", input={...}, output={...})
Track metadataclient.track.metadata(request_id, {"key": "value"})
Track scoreclient.track.score(request_id, score_value)

REST API endpoints

ResourceEndpoint
Get promptGET /prompt-templates/{id} with label or version query param
List promptsGET /prompt-templates
Publish promptPOST /prompt-templates/{id}/publish
Run workflowPOST /workflows/{id}/run
Log requestPOST /log-request
Search logsPOST /search-request-logs
Get traceGET /traces/{trace_id}

Release labels

  • Create: hover version in UI → "Add Release Label" → enter name (e.g., "prod", "staging")
  • Retrieve: pass label parameter to templates.get() or run()
  • Move: click label → select new version
  • A/B test: use Dynamic Release Labels to split traffic between versions

Tables workflow

  1. Create table from prompt, import request history, or start blank
  2. Add text columns for inputs, expected outputs, metadata
  3. Add computed columns for prompts, code, assertions, comparisons
  4. Run sheet, column, or selected cells
  5. Configure scoring with scorecards
  6. Review history and analytics

Environment variables

Store API keys and secrets as workspace environment variables (Settings → Environment Variables), then reference them in prompts and MCP configurations with {{env.VARIABLE_NAME}}.

Decision guidance

ScenarioUseWhy
Running a single LLM callclient.run()Simplest, fetches prompt + executes in one call
Complex multi-step AI systemWorkflowsVisual builder, handles conditionals, tools, code
Evaluating prompt qualityTablesUnified workspace for inputs, outputs, scoring, history
Logging custom LLM callslog_request()For providers or clients PromptLayer doesn't auto-instrument
Following execution flowTraces + auto-instrumentationAutomatic for OpenAI/Anthropic/Google; manual spans for custom code
Deploying without code changesRelease labelsMove label to new version; production code stays the same
Organizing large prompt setsFolders + external IDsFolders for hierarchy; external IDs for syncing with external systems
Testing in CI/CDSDK evalsRun agents against datasets; fail CI on low scores

Workflow

Typical prompt management flow

  1. Create prompt: Dashboard → New → Prompt, add messages with {{variable}} syntax
  2. Test in Playground: Define input variables, click Run, verify output
  3. Save version: Click Save Template, add commit message
  4. Add release label: Hover version → Add Release Label → enter "prod" or "staging"
  5. Run from code: client.run(prompt_name="...", prompt_release_label="prod", input_variables={...})
  6. Monitor logs: Dashboard → Logs, search by prompt name or tags
  7. Iterate: Edit prompt → Save new version → Run eval → Move label to new version

Typical evaluation flow

  1. Create table: Dashboard → New → Table → Create from prompt
  2. Add rows: Import CSV, request history, or add manually
  3. Add computed columns: Prompt execution, code, assertions, comparisons
  4. Configure scoring: Scorecard → define criteria and aggregation
  5. Run evaluation: Click Run or use API to queue operation
  6. Review results: Check cell status, view scores, inspect failures
  7. Save version: Checkpoint meaningful states for comparison

Typical observability flow

  1. Enable auto-instrumentation: Set PROMPTLAYER_API_KEY, import SDK, call instrument_openai() or equivalent
  2. Make LLM calls: Your code calls OpenAI/Anthropic/Google SDK normally
  3. Spans created automatically: Each call becomes a span in a trace
  4. View in dashboard: Traces → search by trace ID or filter by tags
  5. Enrich with metadata: client.track.metadata(request_id, {...}) or client.track.score(request_id, score)
  6. Search and analyze: Use Search to find patterns; Analytics to aggregate metrics

Common gotchas

  • API key not set: Always export PROMPTLAYER_API_KEY before running code. Provider keys (e.g., OPENAI_API_KEY) must also be set; PromptLayer never sends them to its servers.
  • Template variables not filled: Use {{variable_name}} syntax (f-string or Jinja2). Verify input_variables dict keys match template variable names exactly.
  • Release label not found: Labels are case-sensitive and scoped to a single prompt. Verify the label exists on the version you expect.
  • Stale cells in Tables: After changing a computed column's source or configuration, cells show "stale" status. Click Run or use the API to recalculate.
  • Traces missing spans: Auto-instrumentation only works for supported providers (OpenAI, Anthropic, Google, Bedrock). For custom providers, use manual tracing with PromptLayer.run() or custom spans.
  • Deprecated endpoints: Datasets and Evaluations APIs are deprecated; use Tables API for new workflows.
  • Provider-specific schema changes: llm_kwargs and raw_response objects have provider-specific structures. Use prompt_blueprint.prompt_template for stable, provider-agnostic data.
  • Cache bypass: Requests with metadata_filters or model_parameter_overrides skip the SDK cache even if enabled.
  • Workflow timeout: Long-running workflows may timeout. Use timeout parameter or check execution results with workflow_version_execution_results().
  • External ID conflicts: External IDs must be unique per source within a resource type. Attempting to attach a duplicate external ID fails.

Verification checklist

Before submitting work with PromptLayer:

  • API key is set and valid (PROMPTLAYER_API_KEY exported)
  • Provider API key is set (e.g., OPENAI_API_KEY)
  • Prompt template exists in Prompt Registry and has the expected version/label
  • Input variables in code match template variable names exactly
  • Release label points to the correct version (verify in UI)
  • Evaluation pipeline runs without errors; check cell status for failures
  • Traces appear in dashboard with expected spans (for auto-instrumented code)
  • Request logs show correct metadata, tags, and scores
  • Table scorecards are configured and recalculated before reading results
  • Workflow nodes have correct dependencies and conditionals
  • External IDs are unique per source if used for syncing
  • No deprecated Datasets/Evaluations APIs used in new code

Resources


For additional documentation and navigation, see: https://docs.promptlayer.com/llms.txt

Add the canonical catalog link to the repository README so users can inspect current installs and available audits. The publishing guide covers the complete discovery path.

<a href="https://skillzs.dev/skills/docs.promptlayer.com/promptlayer">View promptlayer on skillZs</a>