Open source octopus investigation

Octopus Investigation

A systematic method for discovering relevant open-source projects and contributors by crawling social graphs on GitHub. You start from a single origin (a person or topic), extend tentacles outward through their network, and follow human curation — stars, follows, contributions — instead of keyword search. This surfaces projects that are hard to find through search because you're leveraging the taste of people already in the space.

Reference guide (the canonical description): docs/guides/octopus-investigation.md in the theophile.world repo.

Prerequisites

• gh CLI must be authenticated (gh auth status). If not, stop and ask the user to run gh auth login.
• All crawling uses gh api (no scraping). Be mindful of rate limits — check with gh api rate_limit -q '.rate.remaining' if calls start failing.

Before you start: scope it

An octopus search expands indefinitely. Always confirm scope with the user first unless they already gave it:

• Origin(s): one well-chosen GitHub profile or topic. One strong origin beats five mediocre ones.
• Depth: how many levels to crawl (default: 2 levels deep).
• Budget: time or call limit (default: ~60 API calls / breadth-first, don't tunnel into one tentacle).
• Output: tracker only, or also file GitHub issues? Into which repo?

State the scope back in one line, then proceed.

The process

1. Pick an origin

Profile a single origin deeply embedded in the target ecosystem. Get the lay of the land:

gh api users/ORIGIN -q '{login, name, bio, followers, following, public_repos, blog}'

If the user gave a topic not a person, find an origin first:

gh search repos "TOPIC" --sort stars --limit 10 --json fullName,stargazersCount,owner
gh search users "TOPIC" --limit 10

2. Crawl the social graph

Pull the origin's four signal sources. Stars and following are the highest-signal edges.

# What they build
gh api users/ORIGIN/repos --paginate -q '.[] | "\(.full_name)\t\(.stargazers_count)\t\(.description // "")"'
# What they find interesting (highest signal)
gh api users/ORIGIN/starred --paginate -q '.[] | "\(.full_name)\t\(.stargazers_count)\t\(.description // "")"'
# Who they follow (curation of people)
gh api users/ORIGIN/following --paginate -q '.[].login'
# Who follows them (lower signal, use sparingly)
gh api users/ORIGIN/followers --paginate -q '.[].login'

3. Extend tentacles

Cluster the discoveries by theme (what they do), not by who led you to them. Name each cluster a tentacle. For each promising profile/repo, do a deeper dive (its repos, its stars, its following) — but breadth-first: survey everything at one level before diving into any single tentacle.

Example tentacles from one origin:

• Tentacle A: Privacy-first networking (Tor/Arti, Reticulum)
• Tentacle B: Mesh protocols + anti-forensics (Yggdrasil)
• Tentacle C: Local/on-device inference engines

4. Log everything (dedup tracker)

Maintain a markdown tracker so you never re-crawl the same network. Default location: octopus-tracker.md in the working directory (ask if the user wants it elsewhere). Append every profile and repo you touch:

| Profile / Repo  | Source                           | Tentacle   | Stars | Status   |
| --------------- | -------------------------------- | ---------- | ----- | -------- |
| `user/repo`     | via `origin`'s stars             | A          | 1.2k  | Explored |
| `other-user`    | via `user`'s following           | B          | -     | Queued   |

Status values: Queued → Explored → Issue filed / Cold (went nowhere). Before crawling any profile, grep the tracker — if it's already there, skip it.

5. Report findings (and optionally file issues)

Synthesize the discoveries into a summary grouped by tentacle: what each find is, why it matters, how it could be integrated. If the user opted into issues, file one per significant discovery (or one thematic issue per tentacle when several findings cluster):

gh issue create --repo OWNER/REPO --title "TITLE" --body "What it is / Why it matters / How to integrate" --label discovery

6. Track unexplored leads

Leave queued (unexplored) profiles in the tracker. These seed future octopus searches — the method compounds: each run enriches the network map for the next.

Principles

• Graph-based, not keyword-based. Follow stars and follows; surface what practitioners value but haven't optimized for discoverability.
• Stars are signal. What someone stars reveals their interests more honestly than their bio.
• Breadth-first. Don't tunnel. Survey each level fully before going deeper.
• Time-box it. Set scope before starting (e.g., "2 levels deep, 60 calls max"). Respect the budget and report when you hit it.
• Dedup or waste time. The tracker is what makes the method efficient — always check it before crawling.
• Tentacles are thematic. Organize by what things do, not who led you there — that makes the output actionable.

Output template

When done, report:

1. Scope run — origin(s), depth, calls used.
2. Tentacles — named clusters, each with its top finds (repo, stars, one-line why-it-matters).
3. Tracker — path to the updated tracker file; count of explored vs. queued.
4. Issues — links to any filed (if applicable).
5. Next leads — the most promising queued profiles to seed the next search.