Apify local dev loop

Apify Local Dev Loop

Overview

Build and test Apify Actors on your local machine before deploying to the platform. The Apify CLI (apify run) emulates the platform environment locally — creating storage directories for datasets, key-value stores, and request queues — giving you a tight edit → run → inspect loop with no cloud round-trip.

Prerequisites

• npm install -g apify-cli (global CLI)
• apify login completed with valid token
• Node.js 18+

Authentication

The CLI authenticates with your Apify API token. Run apify login once (it stores the token under ~/.apify/), or export APIFY_TOKEN in the shell for non-interactive use. Local runs (apify run) do not require auth — only apify push / apify call reach the platform. Never commit the token or a plaintext .env containing it.

Actor Project Structure

my-actor/
├── .actor/
│   ├── actor.json          # Actor metadata and config
│   └── INPUT_SCHEMA.json   # Input schema (auto-generates UI on platform)
├── src/
│   └── main.ts             # Entry point
├── storage/                # Created by apify run (git-ignored)
│   ├── datasets/default/
│   ├── key_value_stores/default/
│   └── request_queues/default/
├── package.json
└── tsconfig.json

Instructions

Full config files and Actor source live in implementation.md; the high-level loop is:

Step 1: Create a New Actor Project

# Create from template (interactive)
apify create my-actor

# Or create from specific template
apify create my-actor --template project_cheerio_crawler_ts
# Templates: project_empty, project_cheerio_crawler_ts,
#   project_playwright_crawler_ts, project_puppeteer_crawler_ts

Step 2: Configure and code

Read and Edit the scaffolded .actor/actor.json (metadata + optional dataset view), define .actor/INPUT_SCHEMA.json (validates input and auto-generates the platform UI), and write your crawler in src/main.ts. See implementation.md for the complete actor.json, input schema, and a Cheerio-based main.ts that reads validated input and pushes structured rows via Actor.pushData().

Step 3: Run Locally

# Run with default input from storage/key_value_stores/default/INPUT.json
apify run

# Run with input from command line
apify run --input='{"startUrls":[{"url":"https://example.com"}],"maxPages":5}'

# View results
cat storage/datasets/default/*.json | jq '.'

Step 4: Provide Local Input

Create storage/key_value_stores/default/INPUT.json so repeated apify run invocations reuse the same input:

{
  "startUrls": [{ "url": "https://example.com" }],
  "maxPages": 5
}

For the fastest inner loop, run the entry point directly with tsx watch instead of apify run — wiring and platform-emulating env vars are in implementation.md § Hot Reload Development. Unit tests that mock the SDK boundary are in that same file.

Local Storage Emulation

apify run creates a storage/ directory that mirrors platform storage:

Platform Storage	Local Path	Access via SDK
Default dataset	storage/datasets/default/	Actor.pushData()
Default KV store	storage/key_value_stores/default/	Actor.setValue() / Actor.getValue()
Default request queue	storage/request_queues/default/	Managed by crawler

Output

• A runnable Actor project scaffolded from a template (.actor/, src/, package.json)
• A typed input schema that validates locally and generates the platform UI
• Scraped rows written to storage/datasets/default/ as JSON files
• A local storage/ tree mirroring platform datasets, KV stores, and request queues
• A watch-mode dev loop (tsx watch) and a Vitest test that mocks the SDK boundary

Error Handling

Error	Cause	Solution
apify: command not found	CLI not installed	npm i -g apify-cli
INPUT.json not found	No input provided	Create storage/key_value_stores/default/INPUT.json
Cannot find module 'apify'	SDK not installed	npm install apify crawlee
Dockerfile not found	Missing actor config	Run apify create or create .actor/actor.json

Examples

A quick end-to-end run — seed a local input, run the Actor, and inspect results:

mkdir -p storage/key_value_stores/default
echo '{"startUrls":[{"url":"https://example.com"}],"maxPages":5}' \
  > storage/key_value_stores/default/INPUT.json
apify run
cat storage/datasets/default/*.json | jq '.'

Three fuller worked scenarios live in examples.md:

• Scaffold a new Actor and run it locally — apify create from a template through the first apify run.
• Provide a local input file and inspect results — persistent INPUT.json, plus the exact dataset row shape.
• One-shot run with inline input — throwaway --input runs while iterating on selectors.

Resources

• Local Actor Development
• Apify CLI Reference
• Actor Templates
• Full implementation walkthrough — complete actor.json, input schema, main.ts, hot reload, and tests
• Worked examples — three end-to-end run scenarios

Next Steps

Once the local loop is producing clean data, move on to production-ready Actor code patterns — routing, proxy configuration, retries, and dataset shaping — covered in apify-sdk-patterns.