di-agent-knowledge-engine-streamsets
Reference for StreamSets Data Collector engines and StreamSets environments — StreamSets environment configuration, StreamSets engine deployment (Docker/Podman), StreamSets job execution, StreamSets engine communication methods (tunneling/direct), StreamSets high availability and failover, StreamSets monitoring and resource management. Use ONLY when the user explicitly mentions StreamSets, Data Collector, or a StreamSets-specific concern.
How do I install this agent skill?
npx skills add https://github.com/ibm/ibm-watsonx-data-integration-skills --skill di-agent-knowledge-engine-streamsetsIs this agent skill safe to install?
- Gen Agent Trust Hubpass
This skill provides technical documentation and reference material for managing IBM StreamSets Data Collector engines and environments. It includes standard administrative instructions for container management and correctly guides users on secure credential handling using environment variables and secure vaults.
- Socketpass
No alerts
- Snykpass
Risk: LOW · No issues
What does this agent skill do?
StreamSets Engine
Flow authoring: This skill covers engine infrastructure and operations only. For building or modifying StreamSets flows, use your path's flow authoring skill.
When to Use This Skill
- Configuring or troubleshooting StreamSets environments and engines
- Engine deployment (Docker/Podman), communication methods (tunneling vs. direct)
- Job lifecycle: execution, monitoring, failover, offset management
- High availability setup and resource management
Engine Characteristics
- Container-based: Runs as Docker/Podman containers in your network
- Data ownership: All processing occurs within your infrastructure
- Continuous processing: Jobs run continuously, processing data as it arrives
- High availability: Multiple engines support failover and load distribution
- Flexible communication: Tunneling (default) or direct connection methods
Key Concepts
- Environments: Configure engines and compute resources for projects
- Engines: Data Collector containers that execute flows
- Jobs: Flow executions that run continuously on engines
- Offsets: Track processing progress for resumable execution
- VPCs: Virtual CPUs allocated to engine containers
Architecture
- Browser ↔ watsonx.data integration ↔ Engine (in your network)
- Engines process data locally, send metrics to watsonx.data integration
- Jobs fail over automatically between engines (6.4+)
- Resource thresholds prevent engine overload
Quick Start
- Create environment → environments.md
- Run engine → engines.md
- Execute job → jobs.md
References
- Environments - Environment creation and configuration
- Engines - Engine deployment, management, and monitoring
- Jobs - Job execution and lifecycle management
- Technical Reference - System requirements, commands, and troubleshooting
How can the creator link this skill?
Add the canonical catalog link to the repository README so users can inspect current installs and available audits. The publishing guide covers the complete discovery path.
<a href="https://skillzs.dev/skills/ibm/ibm-watsonx-data-integration-skills/di-agent-knowledge-engine-streamsets">View di-agent-knowledge-engine-streamsets on skillZs</a>