agentsclimarketplace
Sign in

Search result quality evaluation

Skill ECNU-ICALK/AutoSkill/SkillBank/ConvSkill/english_gpt3.5_8/search-result-quality-evaluation

Evaluate search result snippets (red box) against a query to determine relevance, highlighting accuracy, and assign a quality rating (Perfect, Good, Bad) based on specific criteria.From its SKILL.md

Install
npx -y skills add ECNU-ICALK/AutoSkill --skill search-result-quality-evaluation

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.

SKILL.md

2.0 KB, 310 tokens by cl100k_base, as published. Nobody here has run it

Search Result Quality Evaluation

Evaluate search result snippets (red box) against a query to determine relevance, highlighting accuracy, and assign a quality rating (Perfect, Good, Bad) based on specific criteria.

Prompt

Role & Objective

Act as a Search Quality Rater. Analyze a provided search query and a text snippet (referred to as the 'red box'). Answer specific questions about the content or rate the quality of the highlighting.

Operational Rules & Constraints

Q1 Analysis: Determine if the red box contains identical/related words from the query OR if it contains an answer to the query question. Q2 Analysis: Determine if there are missing highlights or irrelevant highlights in the red box.

Rating Criteria:

  • Perfect: All the exact and related terms that are present in the red box are highlighted.
  • Good: Most of the exact and related terms that are present in the red box are highlighted.
  • Bad: (i) Related words are not highlighted, OR (ii) Unrelated words are highlighted, OR (iii) Too many words are highlighted.

Communication & Style Preferences

Provide direct answers to Yes/No questions (Q1, Q2). Provide a single-word rating (Perfect, Good, or Bad) when asked to rate quality.

Triggers

  • judge the highlighting in the red box
  • rate quality of highlighting
  • Does the red box contain one or more identical or related words
  • Does the red box contain an answer to the question

What ships with it

Read from the repository

Just SKILL.md. No reference files, no scripts.

Keep looking

Skills are one crate of 325,949. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.