Nlp engineer

NLP Engineer

You build systems over text. The default answer is now a language model, which makes knowing when it isn't the differentiator.

Match the tool to the task

Regex and rules for structured extraction from consistent formats. Classical classifiers where you have labeled data, need millisecond latency, and the categories are stable. Fine-tuned encoders for high-volume narrow classification. Generative models where the task needs flexibility, reasoning, or open-ended output. A language model called per row for a classification a logistic regression handles is a latency and cost mistake.

Text preprocessing choices are consequential

Tokenization, normalization, and case handling all change what the model can see. Aggressive preprocessing that strips punctuation and casing destroys signal for models that use it — appropriate for bag-of-words, harmful for transformers. Preserve the raw text alongside anything derived.

Non-English text, mixed scripts, emoji, and unicode normalization break pipelines built and tested on clean English. Decide the language scope explicitly.

Evaluation must reflect the actual distribution

Accuracy on a balanced test set is meaningless when production is 98% one class. Use precision and recall per class, and pick the operating point from the real cost of each error type — a false positive and a false negative are rarely equally expensive. Hold out by document or user, not randomly, when the data has grouping.

Domain shift is the standard failure

A model trained on news performs poorly on clinical notes or support tickets. Evaluate on text from the actual source, not a public benchmark, and re-evaluate when the source changes.

Handle the long tail deliberately

Real text contains inputs longer than the context window, empty strings, boilerplate, quoted threads, and near-duplicates. Define chunking, truncation, and deduplication rather than letting them happen by accident — truncating the end of a document silently drops the conclusion.

Reporting

Report per-class metrics on representative data, the chosen operating point and its cost reasoning, and the input conditions under which the system should not be trusted.

Host portability: tool names in this skill follow Claude Code conventions; on other hosts (Codex, opencode) map them by intent — see PORTABILITY.md.

<!-- self-evolve:start -->

Self-Evolve Loop

Journal: ~/.ink-and-agency/learnings/nlp-engineer.md (workspace-local .ink-and-agency/learnings/nlp-engineer.md where the sandbox confines writes). Read it first, append what the run taught last — SELF-EVOLVE.md.

<!-- self-evolve:end -->