Pandas patterns

Skill param087/agent-ml-skills/skills/pandas-patterns

Production-grade Machine Learning, Data Science & MLOps skills for AI coding agents (Codex, Claude Code, Cursor, OpenCode). One npx command to install.

Install

npx -y skills add param087/agent-ml-skills --skill pandas-patterns

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

7 stars7 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Use when writing or reviewing pandas code. Covers idiomatic, vectorized, memory-efficient patterns; avoiding SettingWithCopyWarning, chained indexing, and slow apply loops.

SKILL.md

2.7 KB, as published. Nobody here has run it

Pandas Patterns

Overview

Most pandas pain comes from three things: chained indexing, row-wise apply, and ignoring dtypes/memory. This skill encodes the idioms that keep pandas correct and fast.

When to use

Writing data-wrangling code.
Code is slow, leaks memory, or throws SettingWithCopyWarning.
Reviewing someone's pandas for correctness.

Core rules

Assign with .loc, never chained.

df.loc[df["age"] > 30, "segment"] = "senior"   # correct
# df[df["age"] > 30]["segment"] = "senior"      # WRONG: SettingWithCopyWarning, no-op risk

Vectorize instead of apply(axis=1). Row-wise apply is a Python loop.

df["bmi"] = df["weight"] / df["height"] ** 2          # fast
# df.apply(lambda r: r.weight / r.height**2, axis=1)  # 100x slower

Use np.select / np.where for conditional columns.

import numpy as np
df["tier"] = np.select(
    [df.spend > 1000, df.spend > 100],
    ["gold", "silver"],
    default="bronze",
)

Downcast dtypes to cut memory: category for low-cardinality strings, int32/float32 where safe.
```
df["country"] = df["country"].astype("category")
```

Prefer merge over loops for joins, and validate join cardinality:

df = orders.merge(users, on="user_id", how="left", validate="m:1")

Performance toolkit

df.groupby(..., observed=True).agg(...) — observed=True avoids exploding categorical combinations.
pd.eval / df.query() for large boolean filters.
Read big files in chunks (chunksize=) or switch to Polars/DuckDB when pandas is the bottleneck.
df.pipe(fn) to compose transformations without intermediate variables.

Method chaining (readable + copy-safe)

result = (
    df
    .query("status == 'active'")
    .assign(revenue=lambda d: d.qty * d.price)
    .groupby("region", observed=True)
    .agg(total=("revenue", "sum"))
    .reset_index()
)

Pitfalls

inplace=True rarely saves memory and breaks chaining — avoid it.
Iterating with iterrows — almost always replaceable with vectorization or itertuples.
Floating-point group keys — round or use integer/category keys.
Silent dtype upcasts (int → float when NaN appears) — use nullable Int64 if you must keep integers.

Hand-off

Clean, vectorized transformations that downstream skills (feature-engineering, model-evaluation) can run quickly on full datasets.