ae-experiment
Use when managing Atlas AB or external experiments, experiment approvals and approvers, traffic domains and layers, Features, metrics, buckets, and experiment reports through ae-cli
How do I install this agent skill?
npx skills add https://github.com/thinkingaiagenticengine/ae-cli --skill ae-experimentIs this agent skill safe to install?
- Gen Agent Trust Hubpass
The ae-experiment skill provides a comprehensive set of instructions and reference documentation for managing Atlas AB experiments using the `ae-cli` tool. It does not include any executable code, external scripts, or dependencies. The skill promotes safe usage by requiring user confirmation for high-risk actions, encouraging dry-runs, and providing strict schemas for data input to prevent injection or corruption.
- Socketpass
No alerts
- Snykpass
Risk: LOW · No issues
What does this agent skill do?
ae-experiment
AE CLI (ae-cli) exposes Atlas AB Experiment capabilities through the experiment domain.
Global Rules
- Prefer
ae-cli experiment <command>for Atlas AB Experiment work. - Use
--project-id/-pfor project-scoped commands. - Use
--reqJSON for complex save, status, and delete DTOs. - Do not invent experiment IDs, traffic domain IDs, traffic layer IDs, bucket IDs, Feature keys, metric IDs, or payload field names.
- Bind only metric IDs returned by
experiment metric list; create and verify a missing metric before saving the experiment. - Read commands can run directly after IDs are verified.
- Write commands require user intent. Only
high-risk-writecommands (including domain deletion) prompt for confirmation; use--yesonly after authorization. Use--dry-runbefore write calls when composing JSON.
Naming and response boundary:
- CLI command segments and flags use kebab-case.
- Outer Capability input and all response keys use snake_case.
- Nested business DTOs passed through
--reqkeep their native camelCase fields. - CRITICAL:
save build-guide/save validateresponses recursively snake_caseexample_args.req. Never copy those keys into--req. Use camelCase (expName,metricId, …). Authoritative names:ae-cli capability inspect experiment.experiment.save(or the matching final save id) →input_schema.properties.req.save validatevalid: trueis not a final-save schema pass — snake_casereqcan still fail onexperiment … save. - Audience QP is semantic at the CLI boundary: write
targeting.definitionRequest; readtargeting.definition_request. Never generate or submittargetConfig. - Metric QP is semantic at the CLI boundary: write
metricDefinition; readmetric_definition. Never generate or submitmetricConfig,calcType, orAxxxcodes. - Metric save supports event, formula, retention, and funnel definitions. Check
experiment.metric.savein the target server's capability schema and use the metric save contract to compose the request. - Event-property comparisons belong in
metricDefinition.filters, using{relation,items:[{field,operator,values}]}. Never put comparison fields directly onmetricDefinition, and never submit rawfilts,calcuSymbol, orftvfields. - For metric aggregations
sum/avg/max/distinct_count,metricDefinition.propertyis required and must be a concrete available event property from Analysis metadata. Never create property aggregations withoutproperty(for examplemetric_avg_no_property_*). - Preset event aggregations
total_count/user_count/active_daysomitproperty. - Experiment metric creation does not support
avg_per_user/A105. - Resolve event and property names with Analysis metadata before saving semantic definitions.
- Lists return
data.itemsanddata.total; detail commands returndata.item. - Readiness returns
data.readiness; reports returndata.report; save guides returndata.guide; save dry-run validation returnsdata.validation; writes returndata.result. - Query cancellation returns
data.success.
Typical Workflow
- Discover reusable assets:
experiment bucket listexperiment traffic-domain listfor Space quotas, current versions and domain permissionsexperiment traffic-layer listexperiment feature listexperiment metric list
- Create missing assets if needed:
experiment save build-guide --operation-mode save_metricwhen save validation fails or req shape is unclearexperiment save validate --operation-mode save_metric --req '{...}'before retrying a failed saveexperiment traffic-layer saveexperiment feature saveexperiment metric save
- Create or patch the experiment draft with
experiment experiment save. Useexperiment experiment update-metricswhen replacing metric bindings or assigning guardrail roles. - For an external experiment, use
experiment external-experiment save-submitinstead. Updated servers submit external experiments without approval, regardless of the approver roster. They do not use Features, traffic layers, allocation, targeting, or experiment cycles. Skip the Feature readiness/conflict/submission steps and read experiment detail to verify the result. Older approval-capable servers may still queue review; read server compatibility. On servers exposingreq.clearEndTime, omitting--end-timeon updates retains the saved deadline. Cancel it explicitly with--clear-end-time true; read external deadline compatibility before changing a deadline on older servers. - For a Feature draft, check readiness with
experiment experiment ready-check. - For a Feature on a non-mutex traffic layer, run
experiment experiment conflict-checkbefore submit (needsfeature_key_listfrom context orexperiment get). - Submit the saved Feature draft with
experiment experiment manage --target-status pending. This submission path works on both older and approval-enabled servers.experiment approval submitis an alternative after verifying capability support; submit once, never through both commands. Before choosing approval commands, read server compatibility. On updated servers, a non-empty project approver roster enables Feature approval; an empty roster submits directly using the existing manual or scheduled start. Readexperiment experiment getafter submission. When review is pending, readexperiment approval getand process the exactapproval_idonly for the user-requested action. Approval can start a due Feature experiment; verify approval and runtime status independently. For a scheduled Feature start, savestartTime(or use--start-time) before submission. Read Feature timing. Useexperiment experiment managefor existing lifecycle actions. Itsdraft → pendingcall uses the same server approval policy as other submission paths. To resumepausedtorunning, callmanagedirectly and verify the status withget; do not use the pre-launchready-checkas a resume prerequisite. - Query reports with
experiment report summary,experiment report sample-size, andexperiment report metric-trend.
If an experiment save returns error_code: METRIC_NOT_FOUND, list metrics for the same project. Create and verify the metric before retrying; never retry with another invented ID. Metric deletion returns error_code: METRIC_IN_USE while an active experiment binding exists.
Parameter Conventions
- Experiment save payloads distinguish two allocation fields: experiment-level
req.allocation(integer only; no decimals) and group-levelreq.groups[].allocation(integer only; sum must equal100exactly). - Shared traffic uses
req.isExpSharing=1withreq.trafficLayerIdand must not passreq.layerType. - Exclusive traffic uses
req.isExpSharing=0withreq.bucketId. Setreq.layerTypetoremoteorlocal; omitted mode defaults toremote. Local routing does not support#user_id. - Public experiment metric roles are
primary,secondary, andguardrail. The internalobservationrole is currently unavailable for saves. Guardrail is a binding role; updating metrics replaces the full binding list.
ae-cli experiment experiment get --project-id 1 --exp-id exp_123
ae-cli experiment experiment save --project-id 1 --req '{"expName":"Demo"}' --dry-run
ae-cli experiment experiment save --project-id 1 --req '{"expName":"Local routing draft","isExpSharing":0,"layerType":"local","bucketId":"#account_id"}' --dry-run
ae-cli experiment metric save --project-id 1 --req '{"metricId":"payment_gold_sum","metricName":"Payment gold sum","createType":"event","goalDirection":"up","metricPurpose":"confidence_statistics","metricDesc":"Gold sum for payments above 10","format":"float2","metricDefinition":{"type":"event","event":"payment","aggregation":"sum","property":"gold","filters":{"relation":"and","items":[{"field":"gold","operator":"gt","values":[10]}]}}}' --dry-run
ae-cli experiment report metric-trend --project-id 1 --exp-id exp_123 --metric-id metric_1 --start-time 2026-07-01 --end-time 2026-07-07
Optional global parameters work as in other domains: --host, --mcp-url, --format, --jq, --dry-run, and --yes.
References
Open the matching file in references/ before using a command, especially for write operations and JSON payloads.
Save Helpers
experiment save build-guide, experiment save validate
When a save command returns next_tool: experiment.save.build-guide, call the guide first, then experiment save validate, then retry the final save capability.
Read save_build_guide.md and
save_validate.md before using these helpers. Rebuild
--req in camelCase from inspect / skill references; do not paste example_args.req.
Experiment
experiment experiment save, experiment experiment update-metrics, experiment external-experiment save-submit, capability run experiment.experiment.save-submit, experiment experiment list, experiment experiment list-archived, experiment experiment get, experiment experiment ready-check, experiment experiment conflict-check, experiment experiment manage, experiment experiment update-group, experiment experiment batch-delete, experiment operation-log query
Read manage_guardrail_metrics.md before assigning or replacing metric roles.
External Experiment
experiment external-experiment save-submit
Read save_submit_external_experiment.md before creating or updating an external experiment.
Experiment Approval and Approvers
experiment approval get, experiment approval submit, experiment approval approve,
experiment approval reject, experiment approval withdraw, experiment approver list,
experiment approver add, experiment approver delete
Read experiment_approval.md for the approval workflow,
exact round IDs, proposed versus effective configuration, and the separate experiment approver list.
Approver list requires experiment settings view permission (experimentViewSetting);
approver add/delete require experiment settings edit permission (experimentEditSetting).
Approval actions use the separate experiment permissions described in the reference.
On updated servers, only Feature experiments with a non-empty approver roster
queue delivery changes for approval; a save returning true does not prove
the change is effective. Read experiment detail and any pending proposal.
Removing the last approver preserves existing pending reviews; withdraw and
resubmit when authorized. Adding approvers does not require retrospective
approval of effective configurations. Ineligible members do not make a
non-empty roster empty, and a failed roster read never proves approval is off.
Traffic Domains
experiment traffic-domain list, experiment traffic-domain create, experiment traffic-domain update, experiment traffic-domain delete
Read manage_traffic_domains.md before quota changes,
deletion, or binding domains to traffic layers and exclusive experiments. Domain quotas use
integer basis points (10000 = 100%) and current Space allocation_version. Do not auto-refresh
and retry a rejected write. Domain commands use typed flags and snake_case allocation rows.
List, create and delete return Space snapshots in data.items; data.total counts Spaces.
Update returns Boolean data.result; list again to obtain the new version.
Traffic Layer and Buckets
experiment traffic-layer save, experiment traffic-layer get, experiment traffic-layer list, experiment traffic-layer batch-delete, experiment bucket list
Reports
experiment report summary, experiment report sample-size, experiment report metric-trend, capability run experiment.query.cancel
Metric and Feature
experiment metric save, experiment metric get, experiment metric list, experiment metric delete, experiment feature save, experiment feature update-status, experiment feature get, experiment feature list, experiment feature version-list, experiment feature operation-log query, experiment feature batch-delete, experiment feature whitelist list, experiment feature whitelist save, experiment feature whitelist update-status, experiment feature whitelist batch-delete
Read manage_feature_whitelist.md before querying or changing Feature whitelist rules.
How can the creator link this skill?
Add the canonical catalog link to the repository README so users can inspect current installs and available audits. The publishing guide covers the complete discovery path.
<a href="https://skillzs.dev/skills/thinkingaiagenticengine/ae-cli/ae-experiment">View ae-experiment on skillZs</a>