Job market scraping
Skill thirdwatch-dev/scraping-skills/skills/job-market-scraping
Use when scraping the job market — job listings, salaries, company ratings, or candidate/profile sourcing — from LinkedIn, Indeed, Glassdoor, Naukri, Google Jobs, RemoteOK, Wellfound, Monster, ZipRecruiter, Reed, Adzuna, Upwork, CutShort, or AmbitionBox. Covers building a job aggregator, salary benchmarking, recruiting/sourcing, and ATS/market feeds. Triggers on "scrape jobs", "job listings", "salary data", "company ratings", "find candidates", "source candidates", "recruiting", "job board scraper", "job aggregator".From its SKILL.md
npx -y skills add thirdwatch-dev/scraping-skills --skill job-market-scrapingAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
3 things to look at
- 3 stars3 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
- runs commandsInstructs the agent to run 1 command, including `curl -X POST "https://api.apify.com/v2/acts/thirdwatch~linkedin-jobs-scraper/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" -H "Content-Type: application/json" -d '{"queries": ["data engineer"], "`.
- fetches URLsInstructs the agent to fetch 1 URL, including https://api.apify.com/v2/acts/thirdwatch~linkedin-jobs-scraper/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN.
What its file declares
Copied from the file, not written here
The file declares its own license as MIT. That is the author’s claim about this one file, and it is not the same thing as the license GitHub reports for the repository, which is listed with the other numbers below.
SKILL.md
5.9 KB, ~1.4k tokens by cl100k_base, as published. Nobody here has run it
Job Market Scraping
Routing skill for getting jobs, salaries, company ratings, and candidate/profile data off the web.
The realistic approach: most job boards publish their listings on public, indexable pages — that's the data you want, and it's the data these scrapers target. A few sites guard listings behind anti-bot (LinkedIn and Indeed both run aggressive challenges; Glassdoor, Monster, ZipRecruiter, Upwork, and Wellfound use DataDome/Cloudflare Turnstile). You do not need to log in for any of it. The fields that matter across a job feed are consistent: title, company, location, salary (or estimate), skills/tags, posted date, description, and the apply URL. For salary benchmarking add company ratings (Glassdoor, AmbitionBox); for sourcing you want public profiles, not listings.
Ready-made scrapers
Don't fight Cloudflare/DataDome yourself — these are maintained and billed pay-per-result.
| Target | Scraper | From | Notes |
|---|---|---|---|
| Indeed | indeed-jobs-scraper | $0.008/result | 22 fields, 60+ countries, full descriptions |
| LinkedIn Jobs | linkedin-jobs-scraper | $0.002/result | public listings, parsed salary, no login |
| LinkedIn Profiles | linkedin-profile-scraper | $0.005/result | public profiles, no login |
| LinkedIn Candidate Finder | linkedin-candidate-finder-scraper | $0.003/result | source by role/skills/location |
| LinkedIn Company Employees | linkedin-company-employees-scraper | $0.003/result | team rosters by company |
| Glassdoor | glassdoor-scraper | $0.008/result | jobs + salary estimates + company reviews |
| Google Jobs | google-jobs-scraper | $0.008/result | aggregates 20+ boards, apply URLs |
| Naukri | naukri-jobs-scraper | $0.002/result | India's #1 board, salary + skills |
| RemoteOK | remoteok-jobs-scraper | $0.0015/result | remote-only, salary + tags |
| Wellfound (AngelList) | wellfound-jobs-scraper | $0.008/result | startup jobs, equity |
| Monster | monster-jobs-scraper | $0.008/result | large US board |
| ZipRecruiter | ziprecruiter-scraper | $0.018/result | US board |
| SimplyHired | simplyhired-jobs-scraper | $0.008/result | US aggregator |
| Reed.co.uk | reed-jobs-scraper | $0.003/result | UK's largest board |
| Adzuna | adzuna-jobs-scraper | $0.0015/result | 19-country aggregator |
| Upwork | upwork-jobs-scraper | $0.008/result | freelance jobs, budget + skills |
| CutShort | cutshort-jobs-scraper | $0.005/result | India tech jobs, skills |
| AmbitionBox | ambitionbox-scraper | $0.006/result | India company salaries + ratings |
| Craigslist | craigslist-scraper | $0.0015/result | local gigs/jobs by city |
Which to reach for:
- Building a job aggregator → pull from Google Jobs (it already aggregates 20+ boards) plus the boards that matter for your region (Naukri/India, Reed/UK, Adzuna/19 countries).
- Salary benchmarking → Glassdoor and AmbitionBox carry salary estimates + ratings; LinkedIn Jobs and Naukri return parsed salary on listings.
- Recruiting / sourcing candidates → LinkedIn Candidate Finder (by role/skills/location) and LinkedIn Company Employees (rosters), enriched with LinkedIn Profiles.
- ATS / market feeds → the per-board scrapers above, scheduled, normalized on the common fields.
Run one
curl -X POST "https://api.apify.com/v2/acts/thirdwatch~linkedin-jobs-scraper/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
-H "Content-Type: application/json" \
-d '{"queries": ["data engineer"], "location": "London", "maxResults": 50}'
This blocks until the run finishes and returns the dataset rows as JSON. Get a free token at console.apify.com. Exact input fields are on each actor's Store page.
Build your own
If no scraper above fits (a niche or regional board, a custom ATS), start with web-scraping-playbook for the build-vs-buy decision and the cheapest data source, anti-bot-scraping if the board blocks you, and apify-actor-builder to package it as a deployable, monetizable scraper.
Compliance
These target public job listings and profiles. Job data routinely contains personal data — handle it under GDPR, CCPA, and India DPDP, and never use scraped profiles for unlawful screening, discrimination, or unsolicited spam.
Maintained by Thirdwatch. 70+ ready-made scrapers on the Apify Store.
What ships with it
Read from the repository
Just SKILL.md. No reference files, no scripts.
Gives 0 of the 12 instructions most hr recruiting skills give in ~1.4k tokens
Counted across 356 of the 357 authors here whose files we hold, read 2026-08-07
- Quantify achievements with specific metricsin 14 of 356, across 6 files
- Keep the resume under two pagesin 14 of 356, across 6 files
- Request the full job description if not providedin 12 of 356, across 4 files
- Extract keywords and prioritize job requirementsin 12 of 356, across 4 files
- Stop and ask for clarification if required inputs are missingin 12 of 356, across 5 files
- Map candidate experience to job requirementsin 11 of 356, across 3 files
- Ask if the user wants adjustmentsin 11 of 356, across 3 files
- Provide strengths and gap analysis after the resumein 10 of 356, across 2 files
- Request candidate background details if not providedin 10 of 356, across 2 files
- Format experience bullets as action verb plus resultin 10 of 356, across 2 files
- Ask for missing inputs before startingin 10 of 356, across 9 files
- Use exact job description terminologyin 9 of 356, across 1 file
Said here and by no other author read
- use maintained scrapers for protected boards
- extract standard fields across job feeds
- use Google Jobs for general aggregation
- use Glassdoor for salary benchmarking
- use LinkedIn for sourcing candidates
- run scrapers synchronously to receive JSON
Grouped from the skills themselves: near-identical wordings counted once, and counted by distinct author, so one author publishing three of these counts once. Length counted with cl100k_base; the agent that loads this file may tokenize it differently.