agentsclimarketplace
Sign in

Dask

Skill G1Joshi/Agent-Skills/skills/ai-ml/dask

Dask parallel computing library. Use for scaling pandas.From its SKILL.md

Install
npx -y skills add G1Joshi/Agent-Skills --skill dask

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 12 stars12 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

1.0 KB, 236 tokens by cl100k_base, as published. Nobody here has run it

Dask

Dask scales Python. It looks like Pandas/NumPy but runs on clusters. 2025 updates focus on High Performance Shuffle and GPU integration.

When to Use

  • Big Data: When data > RAM but < BigQuery scale.
  • Cluster Computing: Utilizing a Kubernetes cluster for Python functions.
  • Xarray: Backend for geospatial data.

Core Concepts

Collections

dask.dataframe, dask.array, dask.bag.

Scheduler

Decides where to run tasks (Local Threads, Processes, or Distributed Cluster).

Dashboard

Real-time visualization of task progress (port 8787).

Best Practices (2025)

Do:

  • Use dask-expr: The new query optimization engine for Dask DataFrames.
  • Use Parquet: CSVs are distinctively slow in distributed settings.

Don't:

  • Don't use for small data: The overhead of the scheduler makes it slower than Pandas for <1GB.

References

What ships with it

Read from the repository

Just SKILL.md. No reference files, no scripts.

Keep looking

Skills are one crate of 325,949. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.