agentsclimarketplace

Linkedin profile scraper

Skill yennanliu/linkedin-skill/skills/linkedin-profile-scraper

Scrape LinkedIn profile data (name, current company, country, work history, industry) filtered by company, country, and industry. Uses Playwright MCP browser automation.From its SKILL.md

Install
npx -y skills add yennanliu/linkedin-skill --skill linkedin-profile-scraper

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

2 things to look at

  • no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.
  • 3 stars3 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

6.3 KB, as published. Nobody here has run it

LinkedIn Profile Scraper Skill

Search LinkedIn people by company, country, and/or industry, then extract structured profile data for each result.

Specialized Agents

This skill is backed by three specialist agents. Invoke them for deeper help:

AgentFileWhen to Use
Automation Agentskills/agents/automation-agent/SKILL.mdTiming strategy, retry logic, rate limiting
Web Structure Agentskills/agents/web-structure-agent/SKILL.mdBroken selectors, missing fields, lazy load fixes
QA Agentskills/agents/qa-agent/SKILL.mdValidate scraped data, completeness reports

Orchestrated Run Order

1. QA Agent      → preFlightCheck(page)          # must PASS — abort if it fails
2. [run batch scrape]
3. QA Agent      → validateBatchResults()         # completeness scoring
4. QA Agent      → generateReport()              # PASS/WARN/FAIL summary

Escalation:

  • Fields returning null / empty → Web Structure Agent (selector updates + lazy-load fix)
  • Getting rate-limited → Automation Agent (back-off strategy)
  • Low completeness scores → QA Agent (identify which fields need fixing)

LinkedIn Search Filter Limitation

The scraper builds a keyword query by joining company + country + industry as plain text. LinkedIn's people search does not parse this as structured filters — it treats everything as keywords.

For structured filtering (e.g., company by URN, geo by URN), LinkedIn's search API requires internal IDs not exposed in the UI. The most reliable workaround is to:

  1. Navigate to LinkedIn People search manually, apply filters in the UI
  2. Copy the resulting URL (it will contain facetCurrentCompany, facetGeoRegion, etc.)
  3. Pass that URL directly to the scraper instead of building it from scratch

Ask the Web Structure Agent for current URN lookup patterns if you need fully structured URL construction.

Extracted Fields

FieldDescription
nameFull name
headlineProfessional headline
locationCurrent country / city
currentCompanyMost recent employer
currentTitleCurrent job title
industryIndustry label
workHistoryArray of { title, company, dateRange, location }
profileUrlLinkedIn profile URL

Script Files

  • scrapeLinkedInProfiles.js — batch scraper: searches by filters, visits each profile, returns array of profiles
  • scrapeSingleProfile.js — scrape one profile by URL

Prerequisites

  • User logged into LinkedIn
  • Playwright MCP browser tools available

Usage

Single Profile

// Paste scrapeSingleProfile.js first, then:
const profile = await scrapeSingleProfile(page, 'https://www.linkedin.com/in/username/');
console.log(JSON.stringify(profile, null, 2));

Batch by Company / Country / Industry

// Paste scrapeLinkedInProfiles.js first, then:

// Example 1: engineers at Google in the US
const results = await scrapeLinkedInProfiles(page, {
  company: 'Google',
  country: 'United States',
  industry: 'Software Development',
  maxProfiles: 20
});

// Example 2: finance professionals in Singapore
const results = await scrapeLinkedInProfiles(page, {
  industry: 'Financial Services',
  country: 'Singapore',
  maxProfiles: 15
});

// Example 3: keyword search
const results = await scrapeLinkedInProfiles(page, {
  keywords: 'machine learning',
  company: 'OpenAI',
  maxProfiles: 10
});

console.log(JSON.stringify(results, null, 2));

Configuration Options

OptionDefaultDescription
company''Target company name
country''Target country
industry''Target industry
keywords''Additional search keywords
maxProfiles10Max profiles to scrape
maxPages5Max search result pages to scan
delayMin2000Min delay between requests (ms)
delayMax4000Max delay between requests (ms)

Output Example

[
  {
    "status": "success",
    "name": "Jane Smith",
    "headline": "Senior Software Engineer at Google",
    "location": "San Francisco, California",
    "currentCompany": "Google",
    "currentTitle": "Senior Software Engineer",
    "industry": "Software Development",
    "workHistory": [
      {
        "title": "Senior Software Engineer",
        "company": "Google",
        "dateRange": "Jan 2021 – Present · 3 yrs",
        "location": "San Francisco, CA"
      },
      {
        "title": "Software Engineer",
        "company": "Meta",
        "dateRange": "Jun 2018 – Dec 2020 · 2 yrs 6 mos",
        "location": "Menlo Park, CA"
      }
    ],
    "profileUrl": "https://www.linkedin.com/in/janesmith/"
  }
]

How to Run (Step by Step)

  1. Log into LinkedIn in the browser controlled by Playwright
  2. Test single profile first to confirm scraping works:
    const p = await scrapeSingleProfile(page, 'https://www.linkedin.com/in/satyanadella/');
    console.log(p.name, p.currentCompany, p.location);
    
  3. Run batch scrape with your filters
  4. Export results — paste this to save as JSON:
    copy(JSON.stringify(results, null, 2));  // copies to clipboard in browser console
    

Limitations

  • LinkedIn may paginate or limit search results
  • Some profile fields may be private or hidden
  • Excessive scraping may trigger LinkedIn rate limits — keep maxProfiles ≤ 50 per session
  • Work history accuracy depends on LinkedIn's DOM structure (update selectors if LinkedIn redesigns)
  • Recommended session limit: 30–50 profiles, with breaks between sessions

Troubleshooting

Empty workHistory: LinkedIn lazy-loads experience sections. The script scrolls to trigger them, but if the page is slow, increase delayMin to 3000.

name is null: LinkedIn may show a login wall. Ensure you are logged in before running.

No profiles found in search: Try broader keywords or remove some filters. LinkedIn search ranking can be unpredictable.

Keep looking

Skills are one crate of 609,856. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.