polars
Use when "Polars", "fast dataframe", "lazy evaluation", "Arrow backend", or asking about "pandas alternative", "parallel dataframe", "large CSV processing", "ETL pipeline", "expression API"
How do I install this agent skill?
npx skills add https://github.com/eyadsibai/ltk --skill polarsIs this agent skill safe to install?
- Gen Agent Trust Hubpass
The skill provides comprehensive documentation and guidance for using the Polars DataFrame library, including comparisons with Pandas and performance optimization tips. No security issues were detected.
- Socketpass
No alerts
- Snykpass
Risk: LOW · No issues
- Runlayerpass
1 file scanned · No issues
What does this agent skill do?
Polars Fast DataFrame Library
Lightning-fast DataFrame library with lazy evaluation and parallel execution.
When to Use
- Pandas is too slow for your dataset
- Working with 1-100GB datasets that fit in RAM
- Need lazy evaluation for query optimization
- Building ETL pipelines
- Want parallel execution without extra config
Lazy vs Eager Evaluation
| Mode | Function | Executes | Use Case |
|---|---|---|---|
| Eager | read_csv() | Immediately | Small data, exploration |
| Lazy | scan_csv() | On .collect() | Large data, pipelines |
Key concept: Lazy mode builds a query plan that gets optimized before execution. The optimizer applies predicate pushdown (filter early) and projection pushdown (select columns early).
Core Operations
Data Selection
| Operation | Purpose |
|---|---|
select() | Choose columns |
filter() | Choose rows by condition |
with_columns() | Add/modify columns |
drop() | Remove columns |
head(n) / tail(n) | First/last n rows |
Aggregation
| Operation | Purpose |
|---|---|
group_by().agg() | Group and aggregate |
pivot() | Reshape wide |
melt() | Reshape long |
unique() | Distinct values |
Joins
| Join Type | Description |
|---|---|
| inner | Matching rows only |
| left | All left + matching right |
| outer | All rows from both |
| cross | Cartesian product |
| semi | Left rows with match |
| anti | Left rows without match |
Expression API
Key concept: Polars uses expressions (pl.col()) instead of indexing. Expressions are lazily evaluated and optimized.
Common Expressions
| Expression | Purpose |
|---|---|
pl.col("name") | Reference column |
pl.lit(value) | Literal value |
pl.all() | All columns |
pl.exclude(...) | All except |
Expression Methods
| Category | Methods |
|---|---|
| Aggregation | .sum(), .mean(), .min(), .max(), .count() |
| String | .str.contains(), .str.replace(), .str.to_lowercase() |
| DateTime | .dt.year(), .dt.month(), .dt.day() |
| Conditional | .when().then().otherwise() |
| Window | .over(), .rolling_mean(), .shift() |
Pandas Migration
| Pandas | Polars |
|---|---|
df['col'] | df.select('col') |
df[df['col'] > 5] | df.filter(pl.col('col') > 5) |
df['new'] = df['col'] * 2 | df.with_columns((pl.col('col') * 2).alias('new')) |
df.groupby('col').mean() | df.group_by('col').agg(pl.all().mean()) |
df.apply(func) | df.map_rows(func) (avoid if possible) |
Key concept: Polars prefers explicit operations over implicit indexing. Use .alias() to name computed columns.
File I/O
| Format | Read | Write | Notes |
|---|---|---|---|
| CSV | read_csv() / scan_csv() | write_csv() | Human readable |
| Parquet | read_parquet() / scan_parquet() | write_parquet() | Fast, compressed |
| JSON | read_json() / scan_ndjson() | write_json() | Newline-delimited |
| IPC/Arrow | read_ipc() / scan_ipc() | write_ipc() | Zero-copy |
Key concept: Use Parquet for performance. Use scan_* for large files to enable lazy optimization.
Performance Tips
| Tip | Why |
|---|---|
| Use lazy mode | Query optimization |
| Use Parquet | Column-oriented, compressed |
| Select columns early | Projection pushdown |
| Filter early | Predicate pushdown |
| Avoid Python UDFs | Breaks parallelism |
| Use expressions | Vectorized operations |
| Set dtypes on read | Avoid inference overhead |
vs Alternatives
| Tool | Best For | Limitations |
|---|---|---|
| Polars | 1-100GB, speed critical | Must fit in RAM |
| Pandas | Small data, ecosystem | Slow, memory hungry |
| Dask | Larger than RAM | More complex API |
| Spark | Cluster computing | Infrastructure overhead |
| DuckDB | SQL interface | Different API style |
Resources
- Docs: https://pola.rs/
- User Guide: https://docs.pola.rs/user-guide/
- Cookbook: https://docs.pola.rs/user-guide/misc/cookbook/
How can the creator link this skill?
Add the canonical catalog link to the repository README so users can inspect current installs and available audits. The publishing guide covers the complete discovery path.
<a href="https://skillzs.dev/skills/eyadsibai/ltk/polars">View polars on skillZs</a>