databricks-mlflow-evaluation
MLflow GenAI evaluation and agent improvement on Databricks. Use when running mlflow.genai.evaluate(), building scorers or datasets, aligning judges with MemAlign, optimizing prompts, or monitoring production quality.
How do I install this agent skill?
npx skills add https://github.com/databricks/databricks-agent-skills --skill databricks-mlflow-evaluationIs this agent skill safe to install?
- Gen Agent Trust Hubwarn
The skill facilitates MLflow evaluation and agent improvement on Databricks. It involves processing production traces and external datasets, which presents an indirect prompt injection surface. Additionally, it utilizes dynamic execution for running agent wrappers and custom scorers, and performs shell-based command execution for testing and environment management.
- Socketpass
No alerts
- Snykpass
Risk: LOW · No issues
What does this agent skill do?
Databricks MLflow evaluation
Read databricks-core for authentication and workspace discovery. For tracing,
trace retrieval, observability, and trace-based debugging, use the separate
databricks-mlflow-tracing skill. Verify tracing when the selected evaluation mode or scorers require it; reuse
precomputed outputs or existing traces without rerunning the agent.
The guides below are references, not separately installed skills. When a guide names an upstream MLflow skill, use the matching guide here or the tracing skill.
| Request | Read this guide |
|---|---|
| Define quality criteria or scorers | Scorer design |
| Create a dataset and evaluate an agent | MLflow evaluation |
| Change agent behavior and verify improvement | Improve an agent |
For MLflow API details, consult the existing databricks-docs skill and
https://mlflow.org/docs/latest/llms.txt.
For Databricks judge models, prefer databricks:/system.ai.<model> with a
mlflow[databricks]>=3.16.0. A 404 commonly indicates an older client that
does not understand the system model URI; a 403 on a legacy endpoint name can
mean legacy serving endpoints are disabled. Verify the installed client and
available system.ai model before changing credentials or permissions.
For a new evaluation, establish a baseline with a fixed dataset and scorers,
change one thing, then run the same evaluation again. Use
databricks-mlflow-tracing first if traces are missing or a failure needs a
trace-grounded diagnosis.
How can the creator link this skill?
Add the canonical catalog link to the repository README so users can inspect current installs and available audits. The publishing guide covers the complete discovery path.
<a href="https://skillzs.dev/skills/databricks/databricks-agent-skills/databricks-mlflow-evaluation">View databricks-mlflow-evaluation on skillZs</a>