Update macos sandbox testing skill

Objective

Keep the macos-sandbox-testing skill accurate, current, and standards-compliant as macOS and toolchains evolve.

This internal skill is intended to be used by future agentic sessions working inside this repository.

Quick checks (no macOS required)

Run the bundled repo checks:

python3 .agents/skills/update-macos-sandbox-testing-skill/scripts/run_repo_checks.py

This validates:

• required skill structure (SKILL.md + optional scripts/, references/, assets/)
• YAML frontmatter name/dir-name match
• Python scripts compile (compileall)
• no ChatGPT-only citation tokens accidentally committed (e.g. ChatGPT-only inline citation markers)
• template invariants (presence of required env vars, deny/allow SBPL ordering)

Maintainer references

These notes are repo-internal (not required to apply the macos-sandbox-testing skill to a target repo):

• references/upstream_examples.md (Chromium + SBPL + dyld interpose examples)
• references/network_research.md (why Seatbelt SBPL network rules were chosen)
• references/integration-review-2026.md (integration tradeoffs and modern macOS constraints)

macOS verification (recommended on each meaningful update)

1) Smoke-test against a throwaway SwiftPM package

On a macOS machine with Xcode CLT:

rm -rf /tmp/spm-sandbox-smoke && mkdir -p /tmp/spm-sandbox-smoke
cd /tmp/spm-sandbox-smoke

# Note: `swift package init --type executable` does not necessarily create a test target
# on modern SwiftPM toolchains; `verify.py` runs `swift test`, so create a package that
# includes *both* an executable and tests.
cat > Package.swift <<'SWIFT'
// swift-tools-version: 6.0
import PackageDescription

let package = Package(
    name: "SandboxSmoke",
    products: [
        .library(name: "SandboxSmoke", targets: ["SandboxSmoke"]),
        .executable(name: "SandboxSmokeRunner", targets: ["SandboxSmokeRunner"]),
    ],
    targets: [
        .target(name: "SandboxSmoke"),
        .executableTarget(name: "SandboxSmokeRunner", dependencies: ["SandboxSmoke"]),
        .testTarget(name: "SandboxSmokeTests", dependencies: ["SandboxSmoke"]),
    ]
)
SWIFT

mkdir -p Sources/SandboxSmoke Sources/SandboxSmokeRunner Tests/SandboxSmokeTests
cat > Sources/SandboxSmoke/SandboxSmoke.swift <<'SWIFT'
public enum SandboxSmoke { public static func hello() -> String { "hello" } }
SWIFT
cat > Sources/SandboxSmokeRunner/main.swift <<'SWIFT'
import SandboxSmoke
print(SandboxSmoke.hello())
SWIFT
cat > Tests/SandboxSmokeTests/SandboxSmokeTests.swift <<'SWIFT'
import XCTest
@testable import SandboxSmoke

final class SandboxSmokeTests: XCTestCase {
    func testHello() { XCTAssertEqual(SandboxSmoke.hello(), "hello") }
}
SWIFT

swift test

python3 <path-to-this-repo>/macos-sandbox-testing/scripts/swiftpm_install.py --package-root .
python3 <path-to-this-repo>/macos-sandbox-testing/scripts/swiftpm_verify.py --package-root .

# Also sanity-check the `swift run` path:
SEATBELT_SANDBOX_SELFTEST=1 swift run SandboxSmokeRunner

If swiftpm_verify.py fails due to denied operations, triage with:

• macos-sandbox-testing/references/debugging.md

2) Re-validate SBPL assumptions against upstream examples

This repo relies on behaviors that Apple does not treat as “public API stable”. Re-validate periodically:

• sandbox_init_with_parameters usage and parameter array shape
• SBPL rule ordering behavior
• sandbox_check() return semantics
• dyld interposing mechanics (__DATA,__interpose)

Start from:

• references/upstream_examples.md

Web research checklist (update sources and examples)

Perform broad, up-to-date checks (prefer primary sources):

1. Chromium Seatbelt code (seatbelt.cc) and SBPL design doc:

   • confirm signatures and examples still match current Chromium
   • note any behavioral footnotes for recent macOS releases

2. Mark Rowe’s “Sandboxing on macOS”:

   • confirm high-level statements used by this skill remain correct

3. OpenAI Codex seatbelt policies:

   • compare patterns for /dev/null, process-exec, etc.

4. Any macOS release notes / security discussions that indicate Seatbelt behavior changes (especially around path canonicalization and action modifiers).

When changes are required:

• update the template at macos-sandbox-testing/assets/templates/SandboxTestingBootstrap.c
• update docs under:
  • macos-sandbox-testing/references/ (skill-user-facing)
  • references/ (repo-maintainer research notes, this internal skill)
• bump metadata.version in macos-sandbox-testing/SKILL.md

Package.swift patching pitfalls (installer/uninstaller)

When touching macos-sandbox-testing/scripts/swiftpm_install.py or macos-sandbox-testing/scripts/swiftpm_uninstall.py:

• Do not search/replace targets: blindly: product declarations like .library(..., targets: [...]) also contain targets:.
• Scope modifications to the package-level targets: [...] argument inside the Package(...) call.
• Prefer marker-based edits and remove only what you inserted.
• Treat the bootstrap target name as potentially suffixed (name conflicts); discover it from the installer’s marker block in Package.swift.

Standard compliance

After edits:

• re-run run_repo_checks.py
• ensure macos-sandbox-testing/SKILL.md name: equals directory name
• ensure scripts remain deterministic and do not require network access during basic validation