Threads Research Scraper Skill
An agent-ready AI Skill and Playwright scraper for extracting live posts, search queries, and child reply comment threads on Meta Threads (threads.net).
npx -y skills add KarlYu130/Threads-Research-Scraper-SkillAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
3 things to look at
- 13 days oldThe repository was created 13 days ago. New is not bad, but a brand new repository carrying a familiar-sounding name is the shape a typosquat arrives in, and there has been no time for anyone else to find a problem with it.
- no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.
- 0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
What its author says it does
Copied from the file, not written here
Conduct live topic, keyword, account, and comment research & web scraping on Meta Threads (threads.net) using Playwright headless browser automation, post page navigation, and reply thread extraction.
SKILL.md
3.6 KB, as published. Nobody here has run it
Threads-Research-Scraper-Skill
This skill provides an end-to-end automated workflow to scrape live posts, extract child reply/comment threads, analyze user engagement, and synthesize research reports on Meta Threads (threads.net).
1. When to Use
Use this skill whenever you need to:
- Scrape or extract live Threads post data AND child comments/replies for a topic, keyword, or hashtag (e.g.
#HKUST). - Research posts, discussions, or trends on Meta Threads (
threads.net). - Analyze user sentiment, debate threads, or community feedback on Threads.
- Sort search results chronologically (newest first) or by algorithmic relevance.
2. Environment Prerequisites
Ensure Playwright and Chromium browser binaries are installed:
pip install playwright
playwright install chromium
3. Workflow & Tooling
┌─────────────────────────┐ 1. Search Query & Scroll ┌─────────────────────────┐
│ │ ─────────────────────────────────> │ │
│ Playwright Scraper │ │ Chromium Engine │
│ (scripts/scraper.py) │ <───────────────────────────────── │ (Executes JavaScript) │
│ │ 2. Extract Post URLs └────────────┬────────────┘
└────────────┬────────────┘ │
│ │ 3. Deep Post Page Navigation
│ 4. Extract Main Post + Child Comments ▼
└─────────────────────────────────────────────────── ┌─────────────────────────┐
│ threads.net/@user/post │
└─────────────────────────┘
Execution Steps:
- Search & Link Harvesting: Executes
python scripts/threads_live_scraper.py --query <terms>, loadshttps://www.threads.net/search?q=<query>, and collects post URLs (a[href*="/post/"]). - Deep Post Navigation: Navigates into each post URL directly.
- Post & Comment Extraction: Captures the
main_postcontent and all childcomments/ user replies beneath it. - Report Generation: Parses extracted JSON data into structured markdown research artifacts.
4. Script Usage
# Scrape posts and child comment/reply threads
python scripts/threads_live_scraper.py --query "HKUST" "香港科技大學" --output threads_posts_and_comments.json
Output JSON Structure (threads_posts_and_comments.json):
{
"HKUST": [
{
"url": "https://www.threads.net/@user/post/DZ3fKhijjvK",
"main_post": "Main post content...",
"comments": [
"First reply comment...",
"Second reply comment..."
]
}
]
}