Pytorch learning rate scheduler configuration cosineannealinglr

Skill ECNU-ICALK/AutoSkill/SkillBank/ConvSkill/chinese_gpt4_8/pytorch-learning-rate-scheduler-configuration-cosineannealinglr-

AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolution

Install

npx -y skills add ECNU-ICALK/AutoSkill --skill pytorch-learning-rate-scheduler-configuration-cosineannealinglr-

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.

What its author says it does

Copied from the file, not written here

Configure the training script to support the CosineAnnealingLR learning rate scheduler, allowing dynamic adjustment of the learning rate based on a cosine annealing strategy.

SKILL.md

2.6 KB, as published. Nobody here has run it

PyTorch Learning Rate Scheduler Configuration (CosineAnnealingLR Support)

Configure the training script to support the CosineAnnealingLR learning rate scheduler, allowing dynamic adjustment of the learning rate based on a cosine annealing strategy.

Prompt

Role & Objective

You are a PyTorch training script developer. Your task is to modify the get_optimizer_scheduler function to support the CosineAnnealingLR learning rate scheduler.

Operational Rules & Constraints

Scheduler Support: You must add a conditional branch to check if cfg.TRAIN.SCHEDULER.TYPE is "CosineAnnealingLR".
Parameter Mapping: When "CosineAnnealingLR" is selected, you must read T_MAX from cfg.TRAIN.SCHEDULER.T_MAX and ETA_MIN from cfg.TRAIN.SCHEDULER.ETA_MIN.
Implementation: Use torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=..., eta_min=...).
Preservation: Do not modify the existing logic for "step" or "Mstep" schedulers. Do not modify the optimizer initialization logic.
Error Handling: Keep the else: raise ValueError("Unsupported scheduler") block at the end to handle unknown types.

Input Code Context

The user provided the following code snippet for get_optimizer_scheduler:

def get_optimizer_scheduler(net, cfg):
    # ... (optimizer setup code) ...
    if cfg.TRAIN.OPTIMIZER == "ADAMW":
        optimizer = torch.optim.AdamW(...)
    else:
        raise ValueError("Unsupported Optimizer")
        
    if cfg.TRAIN.SCHEDULER.TYPE == 'step':
        lr_scheduler = torch.optim.lr_scheduler.StepLR(optimizer, cfg.TRAIN.LR_DROP_EPOCH)
    elif cfg.TRAIN.SCHEDULER.TYPE == "Mstep":
        lr_scheduler = torch.optim.lr_scheduler.MultiStepLR(...)
    else:
        raise ValueError("Unsupported scheduler")
    return optimizer, lr_scheduler

Required Modification

Add an elif block for CosineAnnealingLR between Mstep and the final else.

Triggers

add CosineAnnealingLR scheduler support
configure CosineAnnealingLR learning rate
support CosineAnnealingLR in training script