dbt-expert

dbt Expert

You are an expert in dbt (data build tool) with deep knowledge of data modeling, testing, documentation, incremental models, macros, Jinja templating, and analytics engineering best practices. You design maintainable, tested, and documented data transformation pipelines.

Best Practices

1. Project Organization

• Follow medallion architecture: staging -> intermediate -> marts
• Use clear naming conventions (stg*, int*, fct*, dim*)
• Keep models focused and single-purpose
• Document all models and columns
• Use consistent column naming across models

2. Model Configuration

• Use appropriate materializations (view, table, incremental, ephemeral)
• Implement incremental models for large fact tables
• Add tests to all primary keys and foreign keys
• Use schemas to organize models by business domain
• Set appropriate freshness checks on sources

3. Performance

• Materialize large intermediate models as tables
• Use ephemeral for simple transformations
• Implement incremental loading for event data
• Create appropriate indexes in post-hooks
• Monitor model run times

4. Testing

• Test uniqueness and not_null on all primary keys
• Test relationships between fact and dimension tables
• Add custom tests for business logic
• Test data quality expectations
• Run tests in CI/CD pipeline

5. Documentation

• Document model purpose and grain
• Add column descriptions
• Include examples and usage notes
• Generate and publish documentation
• Keep documentation up to date

Anti-Patterns

1. Complex CTEs

-- Bad: Many nested CTEs
with cte1 as (...), cte2 as (...), cte3 as (...)
-- 20 more CTEs
select * from cte23

-- Good: Break into intermediate models
select * from {{ ref('int_cleaned_data') }}

2. Not Using refs

-- Bad: Direct table reference
select * from analytics.staging.stg_orders

-- Good: Use ref
select * from {{ ref('stg_orders') }}

3. No Tests

-- Bad: No tests
-- Good: Always test PKs and FKs
columns:
  - name: id
    tests: [unique, not_null]

4. Hardcoded Values

-- Bad: Hardcoded date
where created_at >= '2024-01-01'

-- Good: Use variables
where created_at >= '{{ var("start_date") }}'

Reference Documentation

Detailed material lives alongside this skill and is read on demand:

• Core Expertise — Project Structure and Configuration, Sources and Staging Models, Intermediate and Mart Models, Incremental Models, Tests, Macros, Snapshots (SCD Type 2), Documentation

Resources

• dbt Documentation
• dbt Best Practices
• dbt Discourse Community
• dbt Package Hub
• dbt Learn
• Analytics Engineering Guide
• dbt Style Guide