Hcppipeline tool

HCP Pipeline Tool

Overview

The HCP Pipelines are the official, highly optimized preprocessing pipelines developed by the Human Connectome Project. They provide state-of-the-art processing for structural (T1w/T2w), functional (task/resting-state fMRI with ICA-FIX), and diffusion MRI (topup + eddy + bedpostx + probtrackx + MSMAll surface alignment).

This skill serves as the NeuroClaw interface-layer wrapper for the HCP Pipelines and strictly follows the hierarchical design:

1. Check whether HCP Pipelines and dependencies are installed.
2. If missing → invoke dependency-planner to generate a safe installation plan.
3. Detect input data (preferably BIDS or HCP-style organized) and confirm processing stages.
4. Generate a clear, numbered execution plan with exact commands, flags, estimated runtime, and risks.
5. Present the plan and wait for explicit user confirmation (“YES” / “execute” / “proceed”).
6. On confirmation → delegate all pipeline stages to claw-shell.
7. After completion, summarize outputs and suggest next steps (e.g., connectivity analysis via fmri-skill or fsl-tool).

Research use only.

Quick Reference

Task	Recommended Pipeline Stage	Typical Runtime (per subject)
Structural preprocessing	PreFreeSurfer + FreeSurfer + PostFreeSurfer	4–12 hours
Functional preprocessing	fMRIVolume + fMRISurface + ICA-FIX	2–6 hours
Diffusion preprocessing	DiffusionPreprocessing + bedpostx + probtrackx	6–24 hours
Surface-based registration	MSMAll	2–4 hours
Full HCP-style multimodal pipeline	All stages combined	12–36+ hours

Common Shell Command Examples

# Structural pipeline (benchmark-safe baseline: BIDS-aware discovery + validation first)
SUBJECT="sub-001"
SESSION=""
BIDS_DIR="/data/bids"
OUTDIR="/data/hcp_output"

if [[ -n "${SESSION}" ]]; then
    ANAT_DIR="${BIDS_DIR}/${SUBJECT}/${SESSION}/anat"
    HCP_SUBJECT="${SUBJECT}_${SESSION}"
else
    ANAT_DIR="${BIDS_DIR}/${SUBJECT}/anat"
    HCP_SUBJECT="${SUBJECT}"
fi

T1W="$(find "${ANAT_DIR}" -maxdepth 1 -type f -name '*_T1w.nii.gz' | sort | head -n 1)"
T2W="$(find "${ANAT_DIR}" -maxdepth 1 -type f -name '*_T2w.nii.gz' | sort | head -n 1)"

[[ -f "${T1W}" ]] || { echo "Missing required input: T1w"; exit 1; }
[[ -f "${T2W}" ]] || { echo "Missing required input: T2w"; exit 1; }
[[ -x "${HCPPIPEDIR}/PreFreeSurfer/PreFreeSurferPipeline.sh" ]] || { echo "Missing required HCP resource: PreFreeSurferPipeline.sh"; exit 1; }

${HCPPIPEDIR}/PreFreeSurfer/PreFreeSurferPipeline.sh \
    --path="${OUTDIR}" \
    --subject="${HCP_SUBJECT}" \
    --t1="${T1W}" \
    --t2="${T2W}" \
    --SEPhaseNeg=NONE \
    --SEPhasePos=NONE \
    --gdcoeffs=NONE

# Functional pipeline with ICA-FIX
${HCPPIPEDIR}/fMRIVolume/fMRIVolumePipeline.sh \
  --path=/data/hcp_output \
  --subject=sub-001 \
  --fmriname=rfMRI_REST1 \
  --fmritcs=/data/bids/sub-001/func/sub-001_task-rest_bold.nii.gz

# Diffusion pipeline
${HCPPIPEDIR}/DiffusionPreprocessing/DiffusionPreprocessing.sh \
  --path=/data/hcp_output \
  --subject=sub-001

Installation (Handled by dependency-planner)

Use dependency-planner with one of the following requests:

• “Install official HCP Pipelines from GitHub”
• “Install HCP Pipelines and dependencies (FSL, FreeSurfer, CUDA if needed)”

After installation, verify with:

echo $HCPPIPEDIR

Prerequisites:

• FSL, FreeSurfer (with valid license)
• MATLAB (for some legacy parts) or Octave
• Sufficient disk space (20–100 GB per subject) and RAM (≥32 GB recommended)

NeuroClaw recommended wrapper script

Use this only as a last-resort orchestration reference. For benchmark-style tasks, prefer a direct task-level shell plan that first discovers BIDS inputs, checks required HCP resources, and only then calls the official stage script.

# hcppipeline_wrapper.py (placed inside the skill folder for reference)
import subprocess
import argparse

def run_hcp_stage(stage, subject, bids_dir, output_dir):
    env = {"HCPPIPEDIR": "/opt/HCP-Pipelines"}
    cmd = [
        f"{env['HCPPIPEDIR']}/{stage}/{stage}Pipeline.sh",
        "--path", output_dir,
        "--subject", subject
    ]
    print("Running HCP stage:", stage)
    subprocess.run(cmd, env=env, check=True)

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("--stage", required=True, help="PreFreeSurfer, fMRIVolume, DiffusionPreprocessing, etc.")
    parser.add_argument("--subject", required=True)
    parser.add_argument("--bids-dir", required=True)
    parser.add_argument("--output-dir", required=True)
    args = parser.parse_args()
    
    run_hcp_stage(args.stage, args.subject, args.bids_dir, args.output_dir)

Important Notes & Limitations

• All actual pipeline execution is routed through claw-shell due to extremely long runtimes.
• HCP Pipelines are very resource-intensive and disk-heavy.
• Requires well-organized input data (preferably BIDS via bids-organizer first).
• For benchmark or contract-sensitive tasks, do not stop at the bare PreFreeSurferPipeline.sh --path --subject --t1 --t2 surface. First resolve BIDS subject/session inputs, validate required template/config resources, and make missing-input or no-fieldmap/no-gdc decisions explicit.
• ICA-FIX denoising is one of the strongest features of HCP functional pipeline.
• Surface-based MSMAll registration provides superior alignment compared to volume-based methods.

When to Call This Skill

• User wants the highest-quality, HCP-style preprocessing for multimodal data.
• When research requires accurate surface-based alignment, ICA-FIX cleaned resting-state fMRI, or advanced diffusion modeling.
• After bids-organizer when preparing data for connectomics or high-precision analysis.

Complementary / Related Skills

• bids-organizer → organize raw data into BIDS before running HCP
• fmriprep-tool → lighter and faster alternative to HCP preprocessing
• fsl-tool → post-HCP connectivity and ROI analysis
• dependency-planner → install HCP Pipelines and dependencies
• claw-shell → safe execution of long-running pipelines
• multi-search-engine / academic-research-hub → retrieve latest HCP best practices

Reference

Official HCP Pipelines integration for NeuroClaw high-precision multimodal preprocessing.
Official repository: https://github.com/Washington-University/HCPpipelines

Post-Execution Verification (Harness Integration)

After HCP Pipeline processing completes, this skill automatically invokes harness-core's VerificationRunner to validate output integrity:

Integrated verification checks:

from skills.harness_core import VerificationRunner, AuditLogger

verifier = VerificationRunner(task_type="hcp_preprocessing")

# 1. Structural preprocessing completion (PreFreeSurfer/FreeSurfer/PostFreeSurfer)
verifier.add_check("structural_pipeline",
    checker=lambda: verify_structural_outputs(output_dir),
    severity="error"
)

# 2. Functional preprocessing (fMRIVolume/fMRISurface completion)
verifier.add_check("functional_pipeline",
    checker=lambda: verify_functional_outputs(output_dir),
    severity="error"
)

# 3. Diffusion preprocessing (topup/eddy/bedpostx completion)
verifier.add_check("diffusion_pipeline",
    checker=lambda: verify_diffusion_outputs(output_dir),
    severity="error"
)

# 4. Surface registration quality (MSMAll)
verifier.add_check("surface_registration",
    checker=lambda: verify_surface_registration_quality(output_dir),
    severity="warning"
)

# 5. ICA-FIX denoising success (if applied)
verifier.add_check("ica_fix_denoising",
    checker=lambda: verify_ica_fix_completion(output_dir),
    severity="warning"
)

# 6. Output BIDS compliance
verifier.add_check("bids_compliance",
    checker=lambda: verify_bids_structure(output_dir),
    severity="warning"
)

report = verifier.run(output_dir)

# Log verification results
logger = AuditLogger(log_file=f"{output_dir}/hcp_verification.jsonl")
logger.log_validation(
    task_name="hcp_preprocessing",
    checks_passed=len([r for r in report.results if r.passed]),
    total_checks=len(report.results),
    output_path=output_dir
)

Output: {output_dir}/hcp_verification.jsonl (structured audit log with JSONL format)

---

Created At: 2026-03-25 19:30 HKT
Last Updated At: 2026-04-05 02:03 HKT
Author: chengwang96