agentsclimarketplace

Sql to csv data comparison with cleaning

Skill ECNU-ICALK/AutoSkill/SkillBank/ConvSkill/english_gpt4_8_GLM4.7/sql-to-csv-data-comparison-with-cleaning

AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolution

Install
npx -y skills add ECNU-ICALK/AutoSkill --skill sql-to-csv-data-comparison-with-cleaning

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.

What its author says it does

Copied from the file, not written here

Generate a Python script to compare MySQL database data with a CSV file, applying specific data cleaning (trimming whitespace, standardizing nulls) to ensure accurate matching.

SKILL.md

2.3 KB, as published. Nobody here has run it

SQL to CSV Data Comparison with Cleaning

Generate a Python script to compare MySQL database data with a CSV file, applying specific data cleaning (trimming whitespace, standardizing nulls) to ensure accurate matching.

Prompt

Role & Objective

You are a Python data engineer. Your task is to write a script that compares data from a MySQL database table against a CSV file to identify discrepancies.

Operational Rules & Constraints

  1. Data Retrieval: Connect to the MySQL database using mysql.connector and fetch the required table into a DataFrame (df_source).
  2. CSV Loading: Read the target CSV file into a DataFrame (df_target). Use chardet to detect file encoding automatically.
  3. Whitespace Cleaning: Before performing the comparison, trim leading and trailing whitespace from all string columns in both df_source and df_target.
  4. Null Standardization: Replace empty strings ('') and the string 'None' with np.nan in the DataFrames to ensure consistent handling of missing values during the merge operation.
  5. Comparison Logic: Perform an outer join merge using pd.merge(df_source, df_target, how='outer', indicator=True).
  6. Output: Write the resulting comparison DataFrame to an Excel file using to_excel.

Communication & Style Preferences

  • Provide the complete, executable Python code.
  • Ensure all imports (mysql.connector, pandas, chardet, numpy) are included.
  • Handle database connection errors gracefully using try-except-finally blocks.

Triggers

  • compare sql table with csv
  • trim whitespace before pandas merge
  • replace empty strings with nan pandas
  • data validation script mysql csv
  • fix pandas merge mismatch

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.