agentsclimarketplace

Nlp pretraining

Skill thada2402/AutoResearchClaw/researchclaw/skills/builtin/domain/nlp-pretraining

Generate research papers autonomously by chatting with OpenClaw, using Python 3.11+, with a self-evolving framework and extensive test coverage.

Install
npx -y skills add thada2402/AutoResearchClaw --skill nlp-pretraining

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 1 stars1 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Best practices for language model pretraining and fine-tuning. Use when generating or reviewing NLP training code.

SKILL.md

1.1 KB, as published. Nobody here has run it

NLP Pretraining/Fine-tuning Best Practice

Fine-tuning recipe:

  • Use pre-trained checkpoints (HuggingFace hub)
  • AdamW optimizer, lr=2e-5 to 5e-5
  • Linear warmup (6% of total steps) + linear decay
  • Batch size: 16-32 (use gradient accumulation for larger effective batch)
  • 3-5 epochs for classification, 1-2 for generation
  • Weight decay: 0.01

Parameter-efficient methods:

  • LoRA: r=8-64, alpha=16-128, apply to q/v projections
  • Prefix tuning: 10-20 prefix tokens
  • Adapters: bottleneck dimension 64-256

Evaluation:

  • Classification: accuracy, F1 (macro for imbalanced)
  • Generation: perplexity, BLEU/ROUGE, human evaluation
  • Use multiple seeds and report mean +/- std

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.