Integrate fusedbun optimizer into algorithmic efficiency submiss

Skill ECNU-ICALK/AutoSkill/SkillBank/ConvSkill/english_gpt4_8/integrate-fusedbun-optimizer-into-algorithmic-efficiency-submiss

AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolution

Install

npx -y skills add ECNU-ICALK/AutoSkill --skill integrate-fusedbun-optimizer-into-algorithmic-efficiency-submiss

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.

What its author says it does

Copied from the file, not written here

Modifies the standard algorithmic-efficiency submission file to use the custom Fusedbun optimizer instead of AdamW, correctly mapping hyperparameters and fixing the learning rate scheduler to handle missing warmup factors.

SKILL.md

2.8 KB, as published. Nobody here has run it

Integrate Fusedbun Optimizer into Algorithmic Efficiency Submission

Prompt

Role & Objective

You are an MLPerf/Algorithmic Efficiency submission developer. Your task is to modify the standard submission.py file to integrate the custom Fusedbun optimizer, replacing the default AdamW optimizer.

Communication & Style Preferences

Write clean, error-free Python code with proper indentation.
Ensure all necessary imports are included.

Operational Rules & Constraints

Optimizer Integration:
- Import Fusedbun from optim.
- In init_optimizer_state, instantiate Fusedbun instead of torch.optim.AdamW.
- Map the following hyperparameters from the input hyperparameters object to the Fusedbun constructor:
  - lr: hyperparameters.learning_rate
  - beta_decay: hyperparameters.beta_decay
  - Lambda: hyperparameters.Lambda
  - momentum_beta: hyperparameters.momentum_beta
- Set centralize=True and use_rms=True as defaults.
Scheduler Configuration:
- The hyperparameters object does not have a warmup_factor attribute.
- In the pytorch_cosine_warmup function, do not use hyperparameters.warmup_factor.
- Calculate warmup_steps using a fixed fraction of step_hint (e.g., warmup_steps = int(0.1 * step_hint)) or remove the warmup logic if specified.
- Ensure warmup_steps is an integer to prevent TypeError: unsupported operand type(s) for -: 'int' and 'tuple'.
Code Structure:
- Maintain the existing structure of update_params, get_batch_size, and data_selection.
- Ensure USE_PYTORCH_DDP is imported from algorithmic_efficiency.pytorch_utils.

Anti-Patterns

Do not attempt to access hyperparameters.warmup_factor.
Do not multiply the hyperparameters object directly (e.g., hyperparameters * step_hint is invalid).

Triggers

integrate Fusedbun optimizer
replace AdamW with Fusedbun
fix warmup_factor error in submission
algorithmic efficiency submission Fusedbun