skillZs
★ LIVE SKILL TAGS ★
>>> LIVE SKILLS INDEX <<<
* OPEN SOURCE *
NO LOGIN, NO TRACKING
※ REAL INSTALL DATA ※
← back to all skills
terrylica/cc-skills57 installs

rangebar-eval-metrics

Range bar evaluation metrics for quant trading. TRIGGERS - range bar metrics, Sharpe ratio, WFO metrics, PSR DSR MinTRL.

How do I install this agent skill?

npx skills add https://github.com/terrylica/cc-skills --skill rangebar-eval-metrics
view source ↗

Is this agent skill safe to install?

  • Gen Agent Trust Hubpass

    The rangebar-eval-metrics skill provides a comprehensive framework and set of tools for calculating performance metrics in quantitative trading. Analysis of the included scripts and documentation revealed no security risks. The code relies on standard, well-known scientific libraries and performs all calculations locally without external network requests or risky system operations.

  • Socketpass

    No alerts

  • Snykpass

    Risk: LOW · No issues

  • Runlayerpass

    5/14 files flagged

What does this agent skill do?

Range Bar Evaluation Metrics

Machine-readable reference + computation scripts for state-of-the-art metrics evaluating range bar (price-based sampling) data.

When to Use This Skill

Use this skill when:

  • Evaluating ML model performance on range bar data
  • Computing Sharpe ratios with non-IID bar sequences
  • Running Walk-Forward Optimization metric analysis
  • Calculating PSR, DSR, or MinTRL statistical tests
  • Generating evaluation reports from fold results

Quick Start

# Compute metrics from predictions + actuals
python scripts/compute_metrics.py --predictions preds.npy --actuals actuals.npy --timestamps ts.npy

# Generate full evaluation report
python scripts/generate_report.py --results folds.jsonl --output report.md

Metric Tiers

TierPurposeMetricsCompute
Primary (5)Research decisionsweekly_sharpe, hit_rate, cumulative_pnl, n_bars, positive_sharpe_ratePer-fold + aggregate
Secondary/Risk (5)Additional contextmax_drawdown, bar_sharpe, return_per_bar, profit_factor, cv_fold_returnsPer-fold
ML Quality (3)Prediction healthic, prediction_autocorr, is_collapsedPer-fold
Diagnostic (5)Final validationpsr, dsr, autocorr_lag1, effective_n, binomial_pvalueAggregate only
Extended Risk (5)Deep risk analysisvar_95, cvar_95, omega_ratio, sortino_ratio, ulcer_indexPer-fold (optional)

Why Range Bars Need Special Treatment

Range bars violate standard IID assumptions:

  1. Variable duration: Bars form based on price movement, not time
  2. Autocorrelation: High-volatility periods cluster bars → temporal correlation
  3. Non-constant information: More bars during volatility = more information per day

Canonical solution: Daily aggregation via _group_by_day() before Sharpe calculation.

References

Core Reference Files

TopicReference File
Sharpe Ratio Calculationssharpe-formulas.md
Risk Metrics (VaR, Omega, Ulcer)risk-metrics.md
ML Prediction Quality (IC, Autocorr)ml-prediction-quality.md
Crypto Market Considerationscrypto-markets.md
Temporal Aggregation Rulestemporal-aggregation.md
JSON Schema for Metricsmetrics-schema.md
Anti-Patterns (Transaction Costs)anti-patterns.md
SOTA 2025-2026 (SHAP, BOCPD, etc.)sota-2025-2026.md
Worked Examples (BTC, EUR/USD)worked-examples.md
Structured Logging (NDJSON)structured-logging.md

Related Skills

SkillRelationship
adaptive-wfo-epochUses weekly_sharpe, psr, dsr for WFE calculation

Dependencies

pip install -r requirements.txt
# Or: pip install numpy>=1.24 pandas>=2.0 scipy>=1.10

Key Formulas

Daily-Aggregated Sharpe (Primary Metric)

def weekly_sharpe(pnl: np.ndarray, timestamps: np.ndarray) -> float:
    """Sharpe with daily aggregation for range bars."""
    daily_pnl = _group_by_day(pnl, timestamps)  # Sum PnL per calendar day
    if len(daily_pnl) < 2 or np.std(daily_pnl) == 0:
        return 0.0
    daily_sharpe = np.mean(daily_pnl) / np.std(daily_pnl)
    # For crypto (7-day week): sqrt(7). For equities: sqrt(5)
    return daily_sharpe * np.sqrt(7)  # Crypto default

Information Coefficient (Prediction Quality)

from scipy.stats import spearmanr

def information_coefficient(predictions: np.ndarray, actuals: np.ndarray) -> float:
    """Spearman rank IC - captures magnitude alignment."""
    ic, _ = spearmanr(predictions, actuals)
    return ic  # Range: [-1, 1]. >0.02 acceptable, >0.05 good, >0.10 excellent

Probabilistic Sharpe Ratio (Statistical Validation)

from scipy.stats import norm

def psr(sharpe: float, se: float, benchmark: float = 0.0) -> float:
    """P(true Sharpe > benchmark)."""
    return norm.cdf((sharpe - benchmark) / se)

Annualization Factors

MarketDaily → WeeklyDaily → AnnualRationale
Crypto (24/7)sqrt(7) = 2.65sqrt(365) = 19.17 trading days/week
Equitysqrt(5) = 2.24sqrt(252) = 15.95 trading days/week

NEVER use sqrt(252) for crypto markets.

CRITICAL: Session Filter Changes Annualization

ViewFilterdays_per_weekRationale
Session-filtered (London-NY)Weekdays 08:00-16:00sqrt(5)Trading like equities
All-bars (unfiltered)Nonesqrt(7)Full 24/7 crypto

Using sqrt(7) for session-filtered data overstates Sharpe by ~18%!

See crypto-markets.md for detailed rationale.

Dual-View Metrics

For comprehensive analysis, compute metrics with BOTH views:

  1. Session-filtered (London 08:00 to NY 16:00): Primary strategy evaluation
  2. All-bars: Regime detection, data quality diagnostics

Academic References

ConceptCitation
Deflated Sharpe RatioBailey & López de Prado (2014)
Sharpe SE with Non-NormalityMertens (2002)
Statistics of Sharpe RatiosLo (2002)
Omega RatioKeating & Shadwick (2002)
Ulcer IndexPeter Martin (1987)

Decision Framework

Go Criteria (Research)

go_criteria:
  - positive_sharpe_rate > 0.55
  - mean_weekly_sharpe > 0
  - cv_fold_returns < 1.5
  - mean_hit_rate > 0.50

Publication Criteria

publication_criteria:
  - binomial_pvalue < 0.05
  - psr > 0.85
  - dsr > 0.50 # If n_trials > 1

Scripts

ScriptPurpose
scripts/compute_metrics.pyCompute all metrics from predictions/actuals
scripts/generate_report.pyGenerate Markdown report from fold results
scripts/validate_schema.pyValidate metrics JSON against schema

Remediations (2026-01-19 Multi-Agent Audit)

The following fixes were applied based on a 12-subagent adversarial audit:

IssueRoot CauseFixSource
weekly_sharpe=0Constant predictionsModel collapse detection + architecture fixmodel-expert
IC=NoneZero variance predictionsReturn 1.0 for constant (semantically correct)model-expert
prediction_autocorr=NaNDivision by zeroGuard for std < 1e-10, return 1.0model-expert
Ulcer Index divide-by-zeroPeak equity = 0Guard with np.where(peak > 1e-10, ...)risk-analyst
Omega/Profit Factor unreliableToo few samplesmin_days parameter (default: 5)robustness-analyst
BiLSTM mean collapseArchitecture too smallhidden_size: 16→48, dropout: 0.5→0.3model-expert
profit_factor=1.0 (n_bars=0)Early return wrong valueReturn NaN when no data to compute ratiorisk-analyst

Model Collapse Detection

# ALWAYS check for model collapse after prediction
pred_std = np.std(predictions)
if pred_std < 1e-6:
    logger.warning(
        f"Constant predictions detected (std={pred_std:.2e}). "
        "Model collapsed to mean - check architecture."
    )

Recommended BiLSTM Architecture

# BEFORE (causes collapse on range bars)
HIDDEN_SIZE = 16
DROPOUT = 0.5

# AFTER (prevents collapse)
HIDDEN_SIZE = 48  # Triple capacity
DROPOUT = 0.3     # Less aggressive regularization

See reference docs for complete implementation details.


Troubleshooting

IssueCauseSolution
weekly_sharpe is 0Constant predictionsCheck for model collapse, increase hidden_size
IC returns NoneZero variance in predictionsModel collapsed - check architecture
prediction_autocorr is NaNDivision by zeroGuard for std < 1e-10 in autocorr calculation
Ulcer Index divide errorPeak equity is zeroAdd guard: np.where(peak > 1e-10, ...)
profit_factor = 1.0No bars processedReturn NaN when n_bars is 0
Sharpe inflated 18%Wrong annualization for dataUse sqrt(5) for session-filtered, sqrt(7) for 24/7
PSR/DSR not computedMissing scipyInstall: pip install scipy
Timestamps not parsedWrong formatEnsure Unix timestamps, not datetime strings

Add the canonical catalog link to the repository README so users can inspect current installs and available audits. The publishing guide covers the complete discovery path.

<a href="https://skillzs.dev/skills/terrylica/cc-skills/rangebar-eval-metrics">View rangebar-eval-metrics on skillZs</a>