dengineproblem/agents-monorepo64 installs
distribution-analyzer
Эксперт анализа распределений. Используй для statistical distributions, data analysis и hypothesis testing.
How do I install this agent skill?
npx skills add https://github.com/dengineproblem/agents-monorepo --skill distribution-analyzerIs this agent skill safe to install?
- Gen Agent Trust Hubpass
The provided skill contains no files or code, making it inherently safe as there are no executable components or instructions present.
- Socketpass
No alerts
- Snykpass
Risk: LOW · No issues
- Runlayerwarn
1/1 file flagged
What does this agent skill do?
Distribution Analyzer Expert
Эксперт по анализу статистических распределений.
Core Principles
Exploratory Data Analysis
- Начинайте с визуализации (histograms, Q-Q plots)
- Descriptive statistics (mean, median, std, skewness, kurtosis)
- Выявление outliers и аномалий
Multiple Testing
- Kolmogorov-Smirnov test
- Anderson-Darling test
- Shapiro-Wilk test
- Chi-square goodness-of-fit
Model Selection
- AIC/BIC criteria
- Cross-validation
- Residual analysis
Distribution Families
Continuous Distributions
Normal (Gaussian):
Use cases: Natural phenomena, measurement errors
Parameters: μ (mean), σ (std)
When: Symmetric, additive processes
Test: Shapiro-Wilk
Log-Normal:
Use cases: Income, stock prices, file sizes
Parameters: μ, σ (of log)
When: Multiplicative processes, positive skew
Test: Transform and test normality
Exponential:
Use cases: Time between events, failure times
Parameters: λ (rate)
When: Memoryless processes
Test: K-S test
Gamma:
Use cases: Wait times, insurance claims
Parameters: α (shape), β (rate)
When: Sum of exponential events
Weibull:
Use cases: Reliability, survival analysis
Parameters: λ (scale), k (shape)
When: Failure time modeling
Beta:
Use cases: Proportions, probabilities
Parameters: α, β
When: Bounded [0,1] data
Pareto:
Use cases: Wealth, city sizes
Parameters: α (shape), xm (scale)
When: Power law, heavy tail
Student's t:
Use cases: Small samples, heavy tails
Parameters: ν (degrees of freedom)
When: Normal-like but heavier tails
Discrete Distributions
Poisson:
Use cases: Event counts, rare events
Parameters: λ (rate)
When: Events in fixed interval
Binomial:
Use cases: Success/failure trials
Parameters: n (trials), p (probability)
When: Fixed number of independent trials
Negative Binomial:
Use cases: Overdispersed counts
Parameters: r, p
When: Variance > Mean
Geometric:
Use cases: Number of trials until success
Parameters: p (probability)
When: Waiting for first success
Distribution Analyzer Class
import numpy as np
import scipy.stats as stats
import matplotlib.pyplot as plt
from typing import Dict, List, Tuple, Optional
class DistributionAnalyzer:
"""Comprehensive distribution analysis toolkit."""
DISTRIBUTIONS = {
'norm': stats.norm,
'lognorm': stats.lognorm,
'expon': stats.expon,
'gamma': stats.gamma,
'weibull_min': stats.weibull_min,
'beta': stats.beta,
'pareto': stats.pareto,
't': stats.t
}
def __init__(self, data: np.ndarray):
self.data = np.array(data)
self.n = len(data)
self.results = {}
def descriptive_stats(self) -> Dict:
"""Calculate descriptive statistics."""
return {
'n': self.n,
'mean': np.mean(self.data),
'median': np.median(self.data),
'std': np.std(self.data),
'var': np.var(self.data),
'min': np.min(self.data),
'max': np.max(self.data),
'range': np.ptp(self.data),
'skewness': stats.skew(self.data),
'kurtosis': stats.kurtosis(self.data),
'q25': np.percentile(self.data, 25),
'q50': np.percentile(self.data, 50),
'q75': np.percentile(self.data, 75),
'iqr': stats.iqr(self.data)
}
def fit_distributions(self) -> Dict:
"""Fit multiple distributions and rank by goodness-of-fit."""
results = {}
for name, dist in self.DISTRIBUTIONS.items():
try:
# Fit distribution
params = dist.fit(self.data)
# Calculate log-likelihood
log_likelihood = np.sum(dist.logpdf(self.data, *params))
# Calculate AIC and BIC
k = len(params)
aic = 2 * k - 2 * log_likelihood
bic = k * np.log(self.n) - 2 * log_likelihood
# K-S test
ks_stat, ks_pvalue = stats.kstest(self.data, name, params)
results[name] = {
'params': params,
'log_likelihood': log_likelihood,
'aic': aic,
'bic': bic,
'ks_statistic': ks_stat,
'ks_pvalue': ks_pvalue
}
except Exception as e:
results[name] = {'error': str(e)}
# Rank by AIC
valid_results = {k: v for k, v in results.items() if 'aic' in v}
ranked = sorted(valid_results.items(), key=lambda x: x[1]['aic'])
self.results = results
return {
'all': results,
'best': ranked[0] if ranked else None,
'ranking': [(name, res['aic']) for name, res in ranked]
}
def test_normality(self) -> Dict:
"""Comprehensive normality testing."""
results = {}
# Shapiro-Wilk (best for n < 5000)
if self.n < 5000:
stat, pvalue = stats.shapiro(self.data)
results['shapiro_wilk'] = {
'statistic': stat,
'pvalue': pvalue,
'is_normal': pvalue > 0.05
}
# D'Agostino-Pearson
if self.n >= 20:
stat, pvalue = stats.normaltest(self.data)
results['dagostino_pearson'] = {
'statistic': stat,
'pvalue': pvalue,
'is_normal': pvalue > 0.05
}
# Anderson-Darling
result = stats.anderson(self.data, dist='norm')
results['anderson_darling'] = {
'statistic': result.statistic,
'critical_values': dict(zip(
[f'{cv}%' for cv in result.significance_level],
result.critical_values
)),
'is_normal': result.statistic < result.critical_values[2] # 5%
}
# Kolmogorov-Smirnov
stat, pvalue = stats.kstest(
self.data, 'norm',
args=(np.mean(self.data), np.std(self.data))
)
results['kolmogorov_smirnov'] = {
'statistic': stat,
'pvalue': pvalue,
'is_normal': pvalue > 0.05
}
return results
def detect_outliers(self, method: str = 'iqr') -> Dict:
"""Detect outliers using various methods."""
results = {'method': method}
if method == 'iqr':
q1, q3 = np.percentile(self.data, [25, 75])
iqr = q3 - q1
lower = q1 - 1.5 * iqr
upper = q3 + 1.5 * iqr
outliers = self.data[(self.data < lower) | (self.data > upper)]
results.update({
'lower_bound': lower,
'upper_bound': upper,
'outliers': outliers,
'n_outliers': len(outliers),
'outlier_percentage': len(outliers) / self.n * 100
})
elif method == 'zscore':
z_scores = np.abs(stats.zscore(self.data))
threshold = 3
outliers = self.data[z_scores > threshold]
results.update({
'threshold': threshold,
'outliers': outliers,
'n_outliers': len(outliers),
'outlier_percentage': len(outliers) / self.n * 100
})
elif method == 'mad':
median = np.median(self.data)
mad = np.median(np.abs(self.data - median))
threshold = 3.5
modified_z = 0.6745 * (self.data - median) / mad
outliers = self.data[np.abs(modified_z) > threshold]
results.update({
'median': median,
'mad': mad,
'outliers': outliers,
'n_outliers': len(outliers),
'outlier_percentage': len(outliers) / self.n * 100
})
return results
def plot_analysis(self, figsize: Tuple = (15, 10)):
"""Generate comprehensive diagnostic plots."""
fig, axes = plt.subplots(2, 3, figsize=figsize)
# Histogram with KDE
axes[0, 0].hist(self.data, bins='auto', density=True, alpha=0.7)
kde_x = np.linspace(self.data.min(), self.data.max(), 100)
kde = stats.gaussian_kde(self.data)
axes[0, 0].plot(kde_x, kde(kde_x), 'r-', lw=2)
axes[0, 0].set_title('Histogram with KDE')
# Q-Q Plot (Normal)
stats.probplot(self.data, dist="norm", plot=axes[0, 1])
axes[0, 1].set_title('Q-Q Plot (Normal)')
# Box Plot
axes[0, 2].boxplot(self.data, vert=True)
axes[0, 2].set_title('Box Plot')
# ECDF
sorted_data = np.sort(self.data)
ecdf = np.arange(1, self.n + 1) / self.n
axes[1, 0].step(sorted_data, ecdf, where='post')
axes[1, 0].set_title('Empirical CDF')
# P-P Plot
theoretical = stats.norm.cdf(sorted_data,
loc=np.mean(self.data),
scale=np.std(self.data))
axes[1, 1].scatter(theoretical, ecdf, alpha=0.5)
axes[1, 1].plot([0, 1], [0, 1], 'r--')
axes[1, 1].set_title('P-P Plot (Normal)')
# Violin Plot
axes[1, 2].violinplot(self.data)
axes[1, 2].set_title('Violin Plot')
plt.tight_layout()
return fig
def bootstrap_ci(self, statistic: str = 'mean',
n_bootstrap: int = 10000,
confidence: float = 0.95) -> Dict:
"""Calculate bootstrap confidence intervals."""
stat_func = {
'mean': np.mean,
'median': np.median,
'std': np.std
}[statistic]
bootstrap_stats = []
for _ in range(n_bootstrap):
sample = np.random.choice(self.data, size=self.n, replace=True)
bootstrap_stats.append(stat_func(sample))
alpha = 1 - confidence
lower = np.percentile(bootstrap_stats, alpha / 2 * 100)
upper = np.percentile(bootstrap_stats, (1 - alpha / 2) * 100)
return {
'statistic': statistic,
'point_estimate': stat_func(self.data),
'ci_lower': lower,
'ci_upper': upper,
'confidence': confidence,
'standard_error': np.std(bootstrap_stats)
}
Usage Examples
# Example usage
import numpy as np
# Generate sample data
np.random.seed(42)
data = np.random.lognormal(mean=2, sigma=0.5, size=1000)
# Create analyzer
analyzer = DistributionAnalyzer(data)
# Descriptive statistics
print("Descriptive Stats:")
print(analyzer.descriptive_stats())
# Fit distributions
print("\nDistribution Fitting:")
fit_results = analyzer.fit_distributions()
print(f"Best fit: {fit_results['best'][0]}")
print(f"AIC: {fit_results['best'][1]['aic']:.2f}")
# Test normality
print("\nNormality Tests:")
norm_results = analyzer.test_normality()
for test, result in norm_results.items():
print(f"{test}: p-value = {result.get('pvalue', 'N/A'):.4f}")
# Detect outliers
print("\nOutlier Detection:")
outliers = analyzer.detect_outliers(method='iqr')
print(f"Found {outliers['n_outliers']} outliers ({outliers['outlier_percentage']:.1f}%)")
# Bootstrap CI
print("\nBootstrap Confidence Interval:")
ci = analyzer.bootstrap_ci('mean', confidence=0.95)
print(f"Mean: {ci['point_estimate']:.2f} [{ci['ci_lower']:.2f}, {ci['ci_upper']:.2f}]")
# Generate plots
fig = analyzer.plot_analysis()
plt.savefig('distribution_analysis.png')
Distribution Selection Guide
Symmetric data:
- Start with Normal
- If heavy tails: Student's t
- If lighter tails: Consider uniform
Right-skewed data:
- Log-normal for multiplicative
- Gamma for waiting times
- Weibull for reliability
- Exponential if memoryless
Left-skewed data:
- Reflect and fit right-skewed
- Beta with appropriate parameters
Bounded data [0, 1]:
- Beta distribution
- Truncated normal if near-normal
Count data:
- Poisson if mean ≈ variance
- Negative binomial if overdispersed
- Binomial for fixed trials
Heavy tails:
- Pareto for power law
- Student's t
- Cauchy (extreme)
Common Pitfalls
Visual inspection alone:
Problem: Histograms can be misleading
Solution: Use multiple tests and Q-Q plots
Ignoring sample size:
Problem: Tests behave differently with n
Solution: Shapiro-Wilk for small, K-S for large
Single test reliance:
Problem: Each test has assumptions
Solution: Use multiple complementary tests
Overfitting:
Problem: Complex distribution fits noise
Solution: AIC/BIC penalize parameters
Parameter uncertainty:
Problem: Point estimates hide uncertainty
Solution: Bootstrap confidence intervals
Лучшие практики
- Visualize first — всегда начинайте с графиков
- Multiple tests — не полагайтесь на один тест
- Sample size awareness — выбирайте тесты по размеру выборки
- Domain knowledge — учитывайте физический смысл данных
- Report uncertainty — давайте confidence intervals
- Validate — cross-validation для model selection
How can the creator link this skill?
Add the canonical catalog link to the repository README so users can inspect current installs and available audits. The publishing guide covers the complete discovery path.
<a href="https://skillzs.dev/skills/dengineproblem/agents-monorepo/distribution-analyzer">View distribution-analyzer on skillZs</a>