qa-testing-ios

QA Testing (iOS)

High-signal iOS test execution and flake control for XCTest, XCUITest, Swift Testing, xcodebuild, xcresult, and simctl.

Pair this skill with software-ios-native when the work is part of a native iOS rewrite or a Codex / Claude Code implementation loop.

Core docs:

• https://developer.apple.com/documentation/xctest
• https://developer.apple.com/documentation/testing
• https://developer.apple.com/documentation/xcode/testing-your-apps-in-xcode
• https://developer.apple.com/documentation/xcode/simctl
• https://developer.apple.com/documentation/xcode-release-notes/xcode-27-release-notes

Quick Reference

Need	Go to
Run the iOS test workflow	## Workflow
Load current defaults and command patterns	## Defaults and ## xcodebuild Patterns
Control flakes and destinations	## Flake Control
Load templates and references	## Navigation

Release Validation Boundary

• Do not treat simulator green or archive green as full iOS release proof when the feature depends on distribution signing, real hardware, APNs, camera, biometrics, purchases, or background execution.
• For those cases, require the release path that matches production behavior: signed archive, beta/distribution channel validation where relevant, and a real-device user-visible outcome.
• Keep project-specific rollout procedures, backend endpoints, tester-account cleanup, and internal runbooks out of this portable core. Put them in project docs or scoped references.
• Use references/ios-ci-general.md for release-CI and archive-path checks, and qa-testing-mobile/references/release-and-rollout.md for distribution-channel planning.

Defaults

• New unit and integration tests: prefer Swift Testing unless the project is already standardized on XCTest.
• UI and performance tests: keep using XCTest/XCUITest.
• PR gate: thin simulator smoke coverage with xcresult artifacts always enabled.
• Release confidence: add a real-device pass only where hardware behavior matters.
• Flake posture: prove the flake first, then fix isolation, waits, or environment drift; retries are a debugging aid, not a success criterion.
• Locale, region, timezone, permissions, and app state must be explicit in automation.
• Cross-framework assertions can lose failures depending on interop mode. Never assume test discovery implies assertion compatibility; use the version-specific modes and CI checks in references/swift-testing.md.
• After adopting the iOS 26 SDK, compare existing snapshot baselines against Liquid Glass rendering and regenerate affected baselines. Review each diff before accepting.

Inputs to Gather

• Xcode entrypoint: -workspace or -project
• -scheme and optional -testPlan
• Destination strategy: simulator, real device, or both
• Required hooks: launch arguments, launch environment, test data, auth bypass, animation toggles
• Artifact needs: xcresult, coverage, screenshots, logs, diagnostics
• CI environment: local, GitHub Actions, Xcode Cloud, or self-hosted macOS
• Whether the task first needs fresh uninstall/install/launch proof before trusting UI observations
• Whether the task includes distribution-channel or hardware-specific behavior that requires a real-device validation pass

Quick Start

Command	Purpose
xcodebuild -list -workspace MyApp.xcworkspace	List schemes
xcodebuild -scheme MyApp -showdestinations	Show valid destinations
xcodebuild -scheme MyApp -showTestPlans	Show available test plans
xcrun xctrace list devices	List physical and simulator devices
xcrun simctl list devices available	List available simulators
xcrun simctl boot "<simulator-name>"	Boot a simulator
xcrun simctl bootstatus booted -b	Wait for boot completion
xcrun simctl uninstall booted <bundle-id>	Remove stale installed app before a smoke pass
Persist booted UDID to .simulator-udid (gitignored) in the select-simulator.sh step	Stop scripts read this file back and call xcrun simctl shutdown <UDID> plus xcrun simctl terminate <UDID> <bundle-id> — terminating exactly the simulator that was booted. Replaces pkill -f Simulator, which shotguns unrelated dev / CI simulators. The UDID file is the contract between run and stop scripts.
xcodebuild test -scheme MyApp -destination 'platform=iOS Simulator,name=<simulator-name>,OS=latest' -resultBundlePath TestResults.xcresult	Run tests on a simulator
xcodebuild test -scheme MyApp -destination 'platform=iOS,id=<UDID>' -resultBundlePath TestResults.xcresult	Run tests on a device
xcodebuild build-for-testing ... then xcodebuild test-without-building ...	Faster reruns
xcrun xcresulttool get test-results summary --path TestResults.xcresult	Pass/fail and counts (get test-results tests for the tree). The deprecated object API requires --legacy; check xcrun xcresulttool get object --help for the selected Xcode
xcodebuild ... -destination "generic/platform=iOS" build	Compile-only build without a simulator; use when simulator services are unavailable or you only need compile/link proof
xcodebuild archive -scheme MyApp -destination 'generic/platform=iOS' -archivePath MyApp.xcarchive	Exercise the archive/signing path (produces an .xcarchive; -exportArchive makes the .ipa). Use before calling a release candidate ready

Workflow

• Resolve the build inputs first: workspace or project, scheme, test plan, destination, and required launch hooks.
• Make the environment repeatable: simulator boot, permissions, locale, region, and app state reset.
• If the task depends on whether the current binary is really on screen, do a fresh uninstall/install/launch smoke pass before interpreting screenshots or UI-test failures.
• If a simulator screenshot path is missing or expired, treat it as a tooling artifact, not as no evidence. Re-capture from the current simulator or use the reported visible symptom plus source inspection to choose the next focused check.
• If the task includes push, purchases, deep links, or other distribution-channel behavior, split transport proof from user-visible outcome proof and use a real-device pass when required.
• Keep compile, test, archive, install/launch, TestFlight, and user-journey evidence distinct. Record source revision, scheme/configuration, archive or app identity, destination/OS, exact test plan or selector, and result-bundle path; stop the release claim at the last observed stage.
• Run with artifacts enabled: -resultBundlePath, and add coverage or diagnostics only when they serve the task.
• Triage from xcresult first, then reproduce a single failing test with -only-testing.
• If CoreSimulatorService returns “Connection refused”, stop retrying the same destination; use generic/platform=iOS for compile-only proof and route service recovery to software-ios-runtime-debugging.
• For backend-coupled flows, exercise the account recreation, cache isolation and web/API parity checklist in references/e2e-harness-and-selectors.md.
• Treat rerun-pass as a flake that needs ownership and a root-cause fix.

Runtime Proof Boundary

• Use this skill for test execution, xcresult, destinations, and flake control after the app is buildable and installable.
• If the core problem is stale installs, simulator drift, malformed .app bundles, missing executables, or install/launch failures, route to software-ios-runtime-debugging.
• If the app cannot be installed or launched reliably, that is a runtime-debugging problem first and a test problem second.
• A simulator or explicitly unsigned build does not exercise device signing. A development-signed device build can prove development signing for that destination, but does not prove distribution signing, export, TestFlight processing, or release-artifact installability. A green archive alone does not prove export, installation, or launch, and a simulator pass does not prove APNs, StoreKit, entitlements, background modes, camera, biometrics, or other device/channel behavior.

xcodebuild Patterns

# Enumerate before an expensive run
xcodebuild test \
  -scheme MyApp \
  -testPlan Smoke \
  -destination 'platform=iOS Simulator,name=<simulator-name>,OS=latest' \
  -enumerate-tests \
  -test-enumeration-format json

# Target one test
xcodebuild test \
  -scheme MyApp \
  -destination 'platform=iOS Simulator,name=<simulator-name>,OS=latest' \
  -only-testing:MyAppUITests/LoginFlowTests/testHappyPath \
  -resultBundlePath TestResults.xcresult

# Parallelize only when the suite is isolation-safe
xcodebuild test \
  -scheme MyApp \
  -destination 'platform=iOS Simulator,name=<simulator-name>,OS=latest' \
  -parallel-testing-enabled YES \
  -maximum-parallel-testing-workers 4 \
  -resultBundlePath TestResults.xcresult

# Controlled retry for CI triage
xcodebuild test \
  -scheme MyApp \
  -destination 'platform=iOS Simulator,name=<simulator-name>,OS=latest' \
  -retry-tests-on-failure \
  -test-iterations 2 \
  -test-repetition-relaunch-enabled YES \
  -collect-test-diagnostics on-failure \
  -resultBundlePath TestResults.xcresult

# Prove a flake locally
xcodebuild test \
  -scheme MyApp \
  -destination 'platform=iOS Simulator,name=<simulator-name>,OS=latest' \
  -only-testing:MyAppUITests/LoginFlowTests/testHappyPath \
  -run-tests-until-failure \
  -test-iterations 25

Flake Control

• Prefer waitForExistence, expectations, and state-based assertions over sleeps.
• Disable or reduce animations in UI-test runs where the app allows it.
• Stub or redirect third-party boundaries; do not depend on live external services in UI tests.
• Reset permissions and app state between tests.
• Pin -testLanguage and -testRegion when locale affects assertions.
• Use test plans for matrix-style coverage across device classes, locales, and environments.
• Keep UI suites thin. Put most business logic coverage in lower layers.
• For multi-locale apps, use qa-testing-mobile/references/localization-testing.md for layered coverage.

Localization and Visual Regression Proof

• Missing-key crashes in string-catalog or generated-accessor lookups are testable defects, not acceptable runtime assertions. Add or run catalog coverage before returning to UI polish.
• Verify both key presence and translated value quality. A locale file containing the English fallback is still a failed localization gate for user-visible copy.
• When new UI copy is added for a feature, run a focused key/value check for the new keys across every shipped locale, then run the broader static-key coverage suite.
• Pair locale coverage with narrow-width visual smoke for high-risk locales such as German, Russian, Japanese, and Arabic. Container width, wrapping, and overlay occlusion are part of the localization test, not a separate design nicety.
• For dense data-visualization screens, include a targeted smoke pass that proves controls, help/info cards, and detail affordances do not cover the primary diagram and remain usable after zoom/filter changes.

Deterministic E2E Harness

For auth, onboarding, billing, and other backend-coupled flows, prefer a fixture-backed launch-environment harness over live credentials. Keep user variants explicit, keep the fixture branch in the same function as the real path, and treat .accessibilityIdentifier() strings as part of the test contract.

Load references/e2e-harness-and-selectors.md for:

• the canonical launch-environment fixture pattern
• harness rules and reset-hook discipline
• selector rules for identifiers vs labels
• the atomic-commit pattern for landing a new E2E suite
• multi-account and backend-parity edge cases

Test Plan Organization

Xcode test plans (.xctestplan) control which tests run, with what configuration, and in which environment. Use them to manage matrix-style test execution:

• Smoke plan: thin critical-path tests for PR gates. Fast, reliable, minimal device matrix.
• Full plan: complete unit + integration + UI suite for nightly or release-candidate runs.
• Locale plan: same UI tests with different -testLanguage / -testRegion overrides per configuration.
• Keep test plans in the project directory alongside the scheme. Reference via -testPlan PlanName in xcodebuild.
• Each configuration within a plan can override launch arguments, environment variables, and enabled tests independently.
• Prefer separate plans over complex multi-configuration single plans — easier to run, triage, and maintain.

AI-Agent Testing for iOS

AI-native test tools (Maestro MCP and others) complement XCUITest; keep deterministic PR-gate smoke in XCUITest. For tool positioning and the decision framework, load qa-testing-mobile/references/ai-native-testing.md. Vendor capability claims change quickly — verify them against the vendor's current docs before recommending one.

When NOT To Use

Scenario	Use Instead
Product architecture, app implementation, SwiftUI rewrite, or Xcode agent workflow	software-ios-native
Build/install failures, stale app suspicion, bundle executable missing, or simulator/package debugging	software-ios-runtime-debugging
Cross-platform mobile test strategy	qa-testing-mobile
Release-wide quality strategy	qa-testing-strategy

Resources

Resource	Purpose
references/e2e-harness-and-selectors.md	Deterministic fixture harnesses, selector discipline, and E2E landing rules
references/swift-testing.md	Swift Testing assertions, traits, confirmations, SDK-dependent features, and XCTest migration
references/xctest-patterns.md	XCTest patterns for unit, integration, and performance tests
references/xcuitest-patterns.md	XCUITest authoring and flake control
references/simulator-commands.md	Current simctl commands worth using in automation
references/snapshot-testing-ios.md	Snapshot testing with current caveats
references/ios-ci-general.md	Provider-neutral iOS CI guidance, including archive-path and fresh-clone checks
references/ios-ci-github-actions.md	GitHub Actions specifics and runner drift checks
references/ios-version-and-vision-pro.md	iOS 26 Liquid Glass snapshot impact, visionOS 26 destination syntax, Reality Composer Pro asset testing, hand-tracking simulator limits
../qa-testing-mobile/references/localization-testing.md	Layered localization coverage for mobile UI and backend-served content
../qa-testing-accessibility/SKILL.md	Accessibility-specific QA gates, screen-reader coverage, and conformance boundary guidance
data/sources.json	Curated external references

Scripts

Script	Purpose
scripts/xcresult_to_junit.py	Convert .xcresult to JUnit XML (Xcode 16+, fixtures from Xcode 27; stdlib only). Counts Test Case nodes, reads failures from their children, and exits 4 if its counts disagree with xcresulttool get test-results summary. Regression test: python3 -m unittest discover -s scripts/tests
scripts/README.md	Usage guide and CI integration examples (GitHub Actions, Bitrise)

Templates

Template	Purpose
assets/template-ios-ui-test-stability-checklist.md	Review checklist for UI-test determinism

Navigation

• ## Workflow, ## xcodebuild Patterns, and ## Flake Control for the baseline sequence
• ## Resources and ## Templates for deeper materials

Learnings Loop

When prior decisions or pitfalls are relevant, consult learnings.consolidated.md if present; use learnings.md only for needed history or as the available fallback. Otherwise skip both.

After applying it, if you encountered a pattern worth remembering, a mistake worth preventing, or a domain fact that surprised you, append one dated bullet to learnings.md via agents-skills-feedback-loop/scripts/append_learning.py. Do not modify SKILL.md itself.