Upwork scrape apply

Upwork Job Scrape & Apply Pipeline

Scrape Upwork jobs matching AI/automation keywords, generate personalized cover letters and proposals, and output to a Google Sheet with one-click apply links.

Inputs

• Keywords: List of search terms (default: automation, ai agent, n8n, gpt, workflow, api integration, scraping, ai consultant)
• Limit: Max jobs to fetch (default: 50)
• Days: Only jobs from last N days (default: 1 = last 24 hours)
• Filters:
  • --verified-payment: Only clients with verified payment
  • --min-spent N: Minimum client spend history
  • --experience: Experience levels (entry, intermediate, expert)

Execution Tools

1. Scrape Jobs

python execution/upwork_apify_scraper.py \
  --limit 50 \
  --days 1 \
  --verified-payment \
  -o .tmp/upwork_jobs_batch.json

Uses upwork-vibe~upwork-job-scraper Apify actor (PPE $0/event, free tier). Always add to Google Sheet! Do not ask user, just assume.

2. Filter & Generate Proposals

python execution/upwork_proposal_generator.py \
  --input .tmp/upwork_jobs_batch.json \
  --workers 5 \
  --output .tmp/upwork_jobs_with_proposals.json

Options:

• --workers N: Parallel Opus 4.5 calls (default: 5)
• --sheet-id ID: Use existing sheet (creates new if omitted)
• --filter-keywords "ai,automation": Only process jobs matching keywords

For each job:

• Generates Apply Now link (/nx/proposals/job/{id}/apply/)
• Creates personalized cover letter using Opus 4.5 with extended thinking
• Creates project proposal Google Doc (with retry + exponential backoff)
• Outputs to new Google Sheet with all columns

Output

Google Sheet with columns:

Column	Description
Keyword	Search term that found this job
Title	Job title
URL	Job listing URL
Budget	Fixed price or hourly range
Experience	Required level
Skills	Top 5 required skills
Client Country	Client location
Client Spent	Total $ spent on platform
Client Hires	Total past hires
Connects	Cost to apply
Posted	Date posted
Contact Name	Discovered first name (if found)
Contact Confidence	high/medium/low - how certain we are
Apply Link	One-click apply URL
Cover Letter	Personalized pitch
Proposal Doc	Google Doc with full proposal

Cover Letter Format

Must stay above the fold (~35 words max). Uses short paraphrases:

Hi. I work with [2-4 word paraphrase] daily & just built a [2-5 word thing]. Free walkthrough: [PROPOSAL_DOC_LINK]

Example:

Hi. I work with n8n automations daily & just built an AI lead scoring pipeline. Free walkthrough: https://docs.google.com/document/d/...

Proposal Format

Conversational, first-person format written as Nick:

Hey [name if available].

I spent ~15 minutes putting this together for you. In short, it's how I would create your [paraphrasedThing] system end to end.

I've worked with $MM companies like Anthropic (yes—that Anthropic) and I have a lot of experience designing/building similar workflows.

Here's a step-by-step, along with my reasoning at every point:

My proposed approach

[4-6 numbered steps with reasoning for each]

What you'll get

[2-3 concrete deliverables]

Timeline

[Realistic estimate, conversational tone]

Tone: Direct, confident, peer-to-peer. Not salesy or formal.

Keywords for AI/Automation

Keyword	Target Jobs
automation	Workflow automation, Zapier/Make
ai agent	AI assistants, autonomous systems
n8n	Self-hosted automation
gpt	OpenAI API, LLM apps
workflow	Business process, systems
api integration	Connect services, middleware
scraping	Data extraction, lead gen
ai consultant	Strategy, implementation

Edge Cases

• No jobs found: Increase limit or broaden keywords
• Anthropic rate limit: Reduce --workers to 2-3
• Google Doc creation fails: Script retries 4x with exponential backoff (1.5s, 3s, 6s, 12s)
• Google API quota: Max ~100 doc creates/day on free tier
• Sheet already has columns: Use --sheet-id to append, or omit for fresh sheet

Contact Name Discovery

The system uses Opus 4.5 to discover the likely contact name from each job posting:

1. From description (high confidence): Signatures like "Thanks, John" or "I'm Sarah"
2. From company research (medium confidence): If a company name is mentioned and AI recognizes it, infers founder/CEO
3. Hedged greeting: For medium/low confidence names, proposal uses "Hey [Name] (if I have the right person)"

Contact info is stored in output and displayed in the Google Sheet.

Learnings

• Apify free tier only filters: limit, fromDate, toDate - all other filters are post-scrape
• Job URL format: https://www.upwork.com/jobs/~{id} → Apply: https://www.upwork.com/nx/proposals/job/~{id}/apply/
• Opus 4.5 model ID: claude-opus-4-5-20251101
• Extended thinking budget: 5000 tokens for cover letters, 8000 for proposals
• Parallel Opus calls work well (5 workers), but Google Docs API needs serialization (semaphore)
• Doc creation uses threading.Semaphore(1) + retry with exponential backoff to avoid SSL errors
• 10 jobs with 5 workers: ~2 min (vs ~20 min sequential)
• Contact name discovery uses Opus 4.5 before proposal generation
• Don't use regex for name extraction - AI handles edge cases much better