Serialization performance
Skill Amey-Thakur/AI-SKILLS/skills/performance/serialization-performance
Plug-and-play skills and prompts for every AI coding agent
npx -y skills add Amey-Thakur/AI-SKILLS --skill serialization-performanceAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
2 things to look at
- 21 days oldThe repository was created 21 days ago. New is not bad, but a brand new repository carrying a familiar-sounding name is the shape a typosquat arrives in, and there has been no time for anyone else to find a problem with it.
- 4 stars4 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
What its author says it does
Copied from the file, not written here
Cut the CPU and bytes spent turning objects into wire or disk format by choosing the right format, reusing schemas and buffers, and avoiding copies. Use when serialization shows up in the profile, payloads are large, or encode/decode latency dominates a hot request path.
SKILL.md
2.9 KB, 585 tokens by cl100k_base, as published. Nobody here has run it
Serialization performance
Every service that crosses a process boundary pays to encode and decode, and on a hot path that cost can rival the work itself. JSON is readable and slow; binary formats are fast and opaque. The wins come from choosing the format for the job, reusing the expensive setup, and not copying bytes you could read in place.
Method
- Measure encode and decode separately. Profile both directions on real payloads; decode is often the more expensive half. Confirm serialization is a genuine share of the request, not a rounding error you are about to over-engineer.
- Match the format to the boundary. Human-facing or debug APIs can afford JSON. High-volume internal RPC wants a schema binary format: Protobuf or Cap'n Proto for RPC, Avro for data pipelines, MessagePack for a compact JSON-shaped drop-in. Pick for throughput and payload size, not familiarity.
- Reuse schemas, codecs, and parsers. Compile the Protobuf descriptor, Avro
schema, or JSON schema once and hold it; recompiling per call is pure waste.
Reuse a configured encoder (a pooled encoder, a
TSerializer) rather than constructing one per message. - Reuse buffers to cut allocation. Encode into a pooled or preallocated byte
buffer (
sync.Pool, a reusedbytes.Buffer, a scratch array) so a high-QPS path does not allocate and free a fresh buffer per message and feed the collector. - Read in place with zero-copy where the format allows. Flatbuffers and Cap'n Proto let you access fields with no parse step; memory-mapped Arrow and read-only byte views avoid copying large blobs. Slice or view the underlying bytes instead of duplicating them into new objects.
- Serialize only what the consumer needs. Trim fields the reader ignores, project columns for a columnar format, and stream a large collection record-by-record instead of building one giant in-memory document. Fewer bytes encoded is less CPU and less GC.
Litmus tests
- Was the format chosen from measured throughput and size, not defaulted to JSON?
- Is the schema or codec compiled once and reused, not rebuilt per call?
- Does a high-QPS encoder reuse buffers instead of allocating per message?
- For large payloads, do you view bytes in place rather than copy them?
Boundaries
This covers turning objects into bytes and back. Compressing those bytes after encoding, and picking the codec, is compression-tradeoffs; reducing the number of round trips that carry them is io-optimization. Schema evolution and versioning are the format's own contract, not a performance question.
What ships with it
Read from the repository
Just SKILL.md. No reference files, no scripts.