agentsclimarketplace

Fablelayer benchmark

Skill VoidLight00/fablelayer/.claude/skills/fablelayer-benchmark

FableLayer blind benchmark, Fable similarity, 비용 절감, Sonnet/Opus/local LLM 비교, RESULTS.md 자동 생성, rubric 설계 요청이면 반드시 사용한다.From its SKILL.md

Install
npx -y skills add VoidLight00/fablelayer --skill fablelayer-benchmark

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 10 stars10 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

1.0 KB, as published. Nobody here has run it

FableLayer Benchmark

목적

FableLayer의 품질 주장을 재현 가능한 benchmark로 환원한다. “Fable 같다”는 인상평이 아니라 blind pairwise rubric, raw JSON, 비용 추정으로 남긴다.

벤치 축

  1. Coding task: 작은 기능 구현 + 테스트 통과.
  2. Agentic task: 다단계 조사/수정/검증 루프.
  3. Long planning task: 요구사항 분해와 drift prevention.

결과 형식

  • bench/README.md: 실행 방법, fixture 설명.
  • bench/RESULTS.md: 사람이 읽는 요약. 미실행이면 미실행이라고 표시한다.
  • _workspace/benchmark_schema.json: raw run schema.

판정 원칙

  • 모델명은 평가자에게 숨긴다.
  • 비용은 입력/출력 token과 단가 가정으로 계산한다.
  • 85% similarity 같은 수치는 실제 run이 없으면 목표값으로만 표시한다.

Keep looking

Skills are one crate of 610,677. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.