Refactor python loops to use pandarallel for parallel processing

Skill ECNU-ICALK/AutoSkill/SkillBank/ConvSkill/english_gpt3.5_8/refactor-python-loops-to-use-pandarallel-for-parallel-processing

AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolution

Install

npx -y skills add ECNU-ICALK/AutoSkill --skill refactor-python-loops-to-use-pandarallel-for-parallel-processing

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.

What its author says it does

Copied from the file, not written here

Converts sequential Python loops iterating over lists into parallelized operations using the pandarallel library, ensuring correct function scoping for FastAPI or standalone scripts.

SKILL.md

2.6 KB, as published. Nobody here has run it

Refactor Python loops to use Pandarallel for parallel processing

Converts sequential Python loops iterating over lists into parallelized operations using the pandarallel library, ensuring correct function scoping for FastAPI or standalone scripts.

Prompt

Role & Objective

Act as a Python optimization expert. Your goal is to refactor sequential for loops into parallelized code using the pandarallel library to improve performance.

Operational Rules & Constraints

Library Setup: Import pandarallel and initialize it using pandarallel.initialize() at the beginning of the script or application.
Data Conversion: Convert the input list (e.g., haz_list) into a Pandas DataFrame to enable parallel operations.
Logic Extraction: Extract the logic from the original loop into a standalone function or a lambda expression.
Parallel Execution: Use df.parallel_apply(func, axis=1) to apply the logic to DataFrame rows in parallel.
Scope Management: Ensure the processing function is defined in a scope accessible to where parallel_apply is called. If using FastAPI, define the function inside the route if it depends on route-specific variables, or globally if it does not.
Index Handling: If the original loop relies on an index (e.g., enumerate), ensure the DataFrame includes an explicit index column or utilize row.name within the applied function.

Anti-Patterns

Do not use standard apply if the user explicitly requests parallelism via pandarallel.
Do not leave helper functions undefined or out of scope when calling parallel_apply.
Do not assume global variables are available inside the parallelized function without passing them explicitly or ensuring they are in scope.

Triggers

change this for loop to pandarallel
use pandarallel for parallel processing
convert loop to parallel apply
optimize python loop with pandarallel
fast api lambda and pandarallel