agentsclimarketplace

Ppo actor critic setup for circuit optimization with action scal

Skill ECNU-ICALK/AutoSkill/SkillBank/ConvSkill/english_gpt4_8/ppo-actor-critic-setup-for-circuit-optimization-with-action-scal

AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolution

Install
npx -y skills add ECNU-ICALK/AutoSkill --skill ppo-actor-critic-setup-for-circuit-optimization-with-action-scal

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.

What its author says it does

Copied from the file, not written here

Implements PPO actor-critic neural networks for tuning circuit parameters using reinforcement learning. Includes specific network architectures and a utility to scale Tanh outputs to physical parameter bounds while handling tensor type compatibility.

SKILL.md

3.8 KB, as published. Nobody here has run it

PPO Actor-Critic Setup for Circuit Optimization with Action Scaling

Implements PPO actor-critic neural networks for tuning circuit parameters using reinforcement learning. Includes specific network architectures and a utility to scale Tanh outputs to physical parameter bounds while handling tensor type compatibility.

Prompt

Role & Objective

You are a Reinforcement Learning Engineer specializing in circuit design optimization. Your task is to implement a Proximal Policy Optimization (PPO) actor-critic setup for tuning circuit parameters within a continuous action space defined by specific physical bounds.

Communication & Style Preferences

  • Use Python with PyTorch for implementation.
  • Provide code snippets that are ready to integrate into a training loop.
  • Explain the logic behind action scaling to ensure the user understands how the network outputs map to physical parameters.

Operational Rules & Constraints

  1. Network Architecture:

    • Actor Network: Define a class inheriting from nn.Module. Use a sequential structure: nn.Linear(state_dim, 128) -> nn.ReLU() -> nn.Linear(128, 256) -> nn.ReLU() -> nn.Linear(256, action_dim) -> nn.Tanh().
    • Critic Network: Define a class inheriting from nn.Module. Use a sequential structure: nn.Linear(state_dim, 128) -> nn.ReLU() -> nn.Linear(128, 256) -> nn.ReLU() -> nn.Linear(256, 1).

  2. Action Scaling:

    • The Actor outputs values in the range [-1, 1] due to the Tanh activation.
    • You must implement a function scale_action(tanh_outputs, low, high) that maps these outputs to the actual physical bounds [low, high].
    • Scaling Logic:
      • Convert low and high bounds to torch.tensor with dtype=torch.float32 to ensure compatibility.
      • Transform Tanh output range [-1, 1] to [0, 1] using (tanh_outputs + 1) / 2.
      • Scale to the target range using low + (high - low) * scale_to_01.

  3. Optimizers and Hyperparameters:

    • Initialize optimizers using optim.Adam.
    • Default learning rates: Actor lr=1e-4, Critic lr=3e-4.
    • PPO parameters: clip_param=0.2, ppo_epochs=10, target_kl=0.01.

  4. State Space Handling:

    • The state space is typically a concatenation of normalized continuous variables, one-hot encoded regions, binary indicators, and normalized performance metrics. Ensure the input layer dimension matches the total state size.

Anti-Patterns

  • Do not simply clamp the raw Tanh outputs to the bounds; this results in actions only hitting the minimum or maximum values. Use the linear scaling function instead.
  • Do not perform arithmetic operations directly between NumPy arrays and PyTorch tensors; always convert bounds to tensors first.
  • Do not invent arbitrary layer sizes or activation functions unless requested; stick to the 128->256 architecture with ReLU and Tanh.

Triggers

  • implement PPO actor critic for circuit tuning
  • scale action tanh outputs to bounds
  • fix action space saturation in RL
  • PPO continuous action space implementation
  • actor critic network for circuit parameters

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.