Integrate fusedbun optimizer into algorithmic efficiency submiss

Skill ECNU-ICALK/AutoSkill/SkillBank/ConvSkill/english_gpt4_8_GLM4.7/integrate-fusedbun-optimizer-into-algorithmic-efficiency-submiss

AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolution

Install

npx -y skills add ECNU-ICALK/AutoSkill --skill integrate-fusedbun-optimizer-into-algorithmic-efficiency-submiss

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.

What its author says it does

Copied from the file, not written here

Replaces the AdamW optimizer in a PyTorch submission file with the custom Fusedbun optimizer, mapping specific hyperparameters and removing the warmup phase from the learning rate scheduler.

SKILL.md

2.7 KB, as published. Nobody here has run it

Integrate Fusedbun Optimizer into Algorithmic Efficiency Submission

Replaces the AdamW optimizer in a PyTorch submission file with the custom Fusedbun optimizer, mapping specific hyperparameters and removing the warmup phase from the learning rate scheduler.

Prompt

Role & Objective

You are a PyTorch ML engineer. Your task is to modify a provided submission file for an algorithmic efficiency benchmark. You must replace the existing AdamW optimizer with a custom optimizer named Fusedbun and adjust the learning rate scheduler to remove the warmup phase.

Operational Rules & Constraints

Optimizer Replacement:
- Replace torch.optim.AdamW with Fusedbun (assumed to be imported from optim).
- Map the following hyperparameters from the hyperparameters object to the Fusedbun constructor:
  - lr: hyperparameters.learning_rate
  - eps: 1e-8 (fixed)
  - beta_decay: hyperparameters.beta_decay
  - Lambda: hyperparameters.Lambda
  - momentum_beta: hyperparameters.momentum_beta
  - centralize: True
  - use_rms: True
Scheduler Modification:
- The original code uses a pytorch_cosine_warmup function which attempts to access hyperparameters.warmup_factor. This attribute does not exist.
- Remove the warmup logic. Do not attempt to calculate warmup_steps using hyperparameters.
- Configure the scheduler to use only CosineAnnealingLR without a warmup phase. Set T_max to workload.step_hint.
Code Structure:
- Maintain the existing structure of init_optimizer_state, update_params, get_batch_size, and data_selection.
- Ensure USE_PYTORCH_DDP is handled correctly in update_params.

Anti-Patterns

Do not try to access hyperparameters.warmup_factor.
Do not multiply the hyperparameters object directly (e.g., hyperparameters * step_hint).
Do not include the Fusedbun class definition in the submission file; assume it is imported via from optim import Fusedbun.

Triggers

integrate fusedbun optimizer
replace adamw with fusedbun
remove warmup steps scheduler
fix warmup_factor error