genai-hub-orchestration

Rules

1. Orchestration always requires a running orchestration deployment as a prerequisite — invoke aicore-lifecycle-management first if one does not exist.
2. The orchestration endpoint is POST {DEPLOYMENT_URL}/v2/completion — not /v1/chat/completions.
3. Orchestration scenario_id and executable_id are always "orchestration" — these are fixed.
4. The Harmonized API normalizes all models to OpenAI message format — the same client code works for GPT, Gemini, and Bedrock.
5. If credentials are not configured, invoke aicore-admin-resources first.
6. generative-ai-hub-sdk is deprecated — always use sap-ai-sdk-gen. Import namespace gen_ai_hub is shared by both.
7. If the user's request does not explicitly specify an approach, use AskUserQuestion to ask whether they prefer the REST API (curl) or the Python SDK before writing or running any code.

---

Deployment Prerequisite

Before using the orchestration service, a deployment must be running:

# Check for an existing orchestration deployment:
uv run skills/aicore-lifecycle-management/scripts/get_deployments.py --scenario orchestration

# If none exists, create a configuration first (reuse if "orchestration" config already exists):
uv run skills/aicore-lifecycle-management/scripts/get_configurations.py --scenario-id orchestration --json | jq '.[] | select(.name == "orchestration")'
uv run skills/aicore-lifecycle-management/scripts/create_configuration.py \
  --name orchestration \
  --scenario-id orchestration \
  --executable-id orchestration

# Then deploy (takes up to 10 minutes):
uv run skills/aicore-lifecycle-management/scripts/create_deployment.py --configuration-id <config-id> --wait --timeout 600

---

Workflow

Step	Action	Key Gate
1	Verify orchestration deployment is RUNNING	get_deployments.py --scenario orchestration
2	If approach not stated: ask curl (REST API) or Python SDK via AskUserQuestion	User preference
3	Choose pipeline features (templating only / + filtering / + grounding)	User intent
4	Select template from assets/ or build from scratch	See asset map below
5	Run and verify output	uv run assets/<template>.py

---

Asset Map

Use case	Template
Basic chat via orchestration pipeline	assets/orchestration_basic.py
Chat with content filtering (input + output)	assets/orchestration_filtering.py
RAG with document grounding	assets/rag_grounding.py

---

Python SDK — Key Patterns

Basic pipeline

from gen_ai_hub.orchestration.service import OrchestrationService
from gen_ai_hub.orchestration.models.config import OrchestrationConfig
from gen_ai_hub.orchestration.models.llm import LLM
from gen_ai_hub.orchestration.models.template import Template, TemplateValue
from gen_ai_hub.orchestration.models.message import SystemMessage, UserMessage

config = OrchestrationConfig(
    template=Template(messages=[
        SystemMessage("You are a helpful assistant."),
        UserMessage("{{?user_query}}"),
    ]),
    llm=LLM(name="gpt-4o-mini", version="latest"),
)
svc = OrchestrationService(config=config)
result = svc.run(template_values=[TemplateValue("user_query", "Hello")])
print(result.orchestration_result.choices[0].message.content)

Add content filtering

from gen_ai_hub.orchestration.models.content_filtering import (
    ContentFiltering,
    InputFiltering,
    OutputFiltering,
)
from gen_ai_hub.orchestration.models.azure_content_filter import AzureContentFilter

filtering = ContentFiltering(
    input_filtering=InputFiltering(filters=[AzureContentFilter(hate=2, violence=2, sexual=2, self_harm=2)]),
    output_filtering=OutputFiltering(filters=[AzureContentFilter(hate=2, violence=2, sexual=2, self_harm=2)]),
)
config = OrchestrationConfig(template=..., llm=..., filtering=filtering)

---

REST API (curl)

export TOKEN=$(uv run skills/aicore-admin-resources/scripts/get_token.py)
export DEPLOYMENT_URL=$(uv run skills/aicore-lifecycle-management/scripts/get_deployments.py \
  --scenario orchestration --json 2>/dev/null | jq -r '.[0].deployment_url')
export RESOURCE_GROUP=$(uv run skills/aicore-admin-resources/scripts/check_setup.py 2>/dev/null | jq -r '.resource_group')

curl --request POST "${DEPLOYMENT_URL}/v2/completion" \
  --header "Authorization: Bearer ${TOKEN}" \
  --header "AI-Resource-Group: ${RESOURCE_GROUP}" \
  --header "content-type: application/json" \
  --data-raw '{
    "config": {
      "modules": {
        "prompt_templating": {
          "prompt": {
            "template": [
              {"role": "system", "content": "You are a helpful assistant."},
              {"role": "user",   "content": "{{?user_query}}"}
            ]
          },
          "model": {
            "name": "gpt-4o-mini",
            "params": {"max_completion_tokens": 512}
          }
        }
      }
    },
    "placeholder_values": {"user_query": "What is SAP AI Core?"},
    "messages_history": []
  }'

Extract the answer with: jq '.final_result.choices[0].message.content'

Reasoning models (Gemini 2.5 Pro, o3): set max_completion_tokens to at least 1024 — reasoning tokens consume the budget before the visible response.

---

Handoffs

• No orchestration deployment running? → invoke aicore-lifecycle-management
• Credential/auth error? → invoke aicore-admin-resources
• Need to find a model name? → invoke genai-hub-foundation-models
• Want full Java SDK boilerplate (non-orchestration)? → invoke aicore-sdk-java
• Want full JS SDK boilerplate (non-orchestration)? → invoke aicore-sdk-js

---

When to Load Reference Files

Trigger	Load
REST API details, request/response shape, Harmonized API	references/ORCHESTRATION.md
Document grounding setup, GroundingModule config, RAG	references/GROUNDING.md