Tribe v2 neuroscience

TRIBE v2 Neuroscience

In-silico neuroscience using Meta FAIR's TRIBE v2 — predict fMRI cortical responses to video, audio, or text using a single pretrained transformer. Run experiments on any hardware, without a scanner.

What TRIBE v2 Is

TRansformer for In-silico Brain Experiments (v2) is a brain encoding model released by Meta FAIR on March 26, 2026. It is not a language model — it predicts fMRI BOLD responses on the fsaverage5 cortical surface (~20,000 vertices per hemisphere) given multimodal sensory input.

Architecture:

Video  → V-JEPA2 (video encoder)
Audio  → Wav2Vec-BERT 2.0 (audio encoder)
Text   → LLaMA 3.2-3B (text encoder)
         ↓
   Unified Transformer
         ↓
   fsaverage5 mesh (~20k vertices)
   (n_timesteps × n_vertices)

Key properties:

• 70x resolution improvement over TRIBE v1
• 2-3x accuracy improvement, zero-shot generalization to new subjects
• 5-second temporal offset built in — accounts for hemodynamic lag
• Log-linear scaling with fMRI training data (like LLMs with tokens)
• License: CC BY-NC 4.0 (non-commercial research only)
• HuggingFace: facebook/tribev2
• Demo: https://aidemos.atmeta.com/tribev2

---

Quick Start

1. Install TRIBE v2

# Requires Python 3.11+
git clone https://github.com/facebookresearch/tribev2
cd tribev2
pip install -e .

2. Load the Model

from tribev2 import TribeModel

model = TribeModel.from_pretrained("facebook/tribev2", cache_folder="./cache")

The first load downloads model weights (~7GB). Subsequent loads use the cache.

3. Run Your First Prediction

# Video input (returns DataFrame of events)
df = model.get_events_dataframe(video_path="stimulus.mp4")

# Text input
df = model.get_events_dataframe(text_path="transcript.txt")

# Audio input
df = model.get_events_dataframe(audio_path="audio.wav")

# Predict cortical responses
preds, segments = model.predict(events=df)

# Output shape: (n_timesteps, n_vertices)
# n_vertices ≈ 20,484 on fsaverage5
print(preds.shape)   # e.g., (142, 20484)
print(segments)      # list of segment boundaries in seconds

---

Workflow A: Single Stimulus Prediction

Trigger: You have one video, audio clip, or transcript and want to know which brain regions activate.

from tribev2 import TribeModel
import numpy as np
import pandas as pd

# Load model
model = TribeModel.from_pretrained("facebook/tribev2", cache_folder="./cache")

# Load stimulus (choose one modality)
df = model.get_events_dataframe(video_path="face_stimulus.mp4")
# df = model.get_events_dataframe(audio_path="speech.wav")
# df = model.get_events_dataframe(text_path="story.txt")

# Predict
preds, segments = model.predict(events=df)
# preds: numpy array (n_timesteps, n_vertices)

# Find peak activation timestep
peak_t = np.argmax(preds.mean(axis=1))
peak_activations = preds[peak_t, :]

# Top-10 most activated vertices at peak
top_verts = np.argsort(peak_activations)[-10:][::-1]
print(f"Peak timestep: {peak_t} (~{peak_t * 1.5:.1f}s)")
print(f"Top vertices: {top_verts}")
print(f"Peak activation values: {peak_activations[top_verts]}")

# Regional mean activations (using known fsaverage5 ranges)
REGIONS = {
    "V1_left":  (0, 1500),
    "FFA_right": (9900, 10400),
    "A1_left":  (3500, 4200),
    "Broca_left": (6200, 6800),
    "DMN_mPFC": (14000, 15000),
}

for region, (v_start, v_end) in REGIONS.items():
    region_mean = preds[:, v_start:v_end].mean()
    print(f"  {region}: mean activation = {region_mean:.4f}")

Use the Cortical Atlas to interpret which vertices correspond to which regions.

---

Workflow B: Virtual Experiment

Trigger: You want to compare brain responses across multiple stimuli — e.g., faces vs. objects vs. scenes.

from tribev2 import TribeModel
import numpy as np
import pandas as pd
from pathlib import Path

model = TribeModel.from_pretrained("facebook/tribev2", cache_folder="./cache")

# Define your stimulus set
stimuli = {
    "faces":   "stimuli/faces.mp4",
    "scenes":  "stimuli/scenes.mp4",
    "objects": "stimuli/objects.mp4",
    "baseline": "stimuli/scrambled.mp4",
}

# Define regions of interest (vertex ranges on fsaverage5)
ROIS = {
    "FFA_right": (9900, 10400),    # Fusiform Face Area
    "PPA_right": (10400, 11000),   # Parahippocampal Place Area
    "LOC_right": (8800, 9500),     # Lateral Occipital Complex
    "EBA_right": (9500, 9900),     # Extrastriate Body Area
}

results = []

for condition, path in stimuli.items():
    df = model.get_events_dataframe(video_path=path)
    preds, _ = model.predict(events=df)
    
    for roi_name, (v_start, v_end) in ROIS.items():
        roi_activation = preds[:, v_start:v_end].mean()
        results.append({
            "condition": condition,
            "roi": roi_name,
            "mean_activation": roi_activation,
            "peak_activation": preds[:, v_start:v_end].max(),
        })

# Analyze
df_results = pd.DataFrame(results)
pivot = df_results.pivot(index="condition", columns="roi", values="mean_activation")
print(pivot)

# Expected result for face selectivity:
# FFA_right should be highest for "faces" condition
# PPA_right should be highest for "scenes" condition

Use scripts/run_experiment.py to automate this over a directory of stimuli.

---

Workflow C: Paradigm Replication

Trigger: You want to replicate a classic neuroscience finding in-silico before running a real fMRI study.

Example: Replicating face selectivity in the Fusiform Face Area (Kanwisher 1997).

from tribev2 import TribeModel
import numpy as np

model = TribeModel.from_pretrained("facebook/tribev2", cache_folder="./cache")

# Kanwisher 1997: faces >> objects in FFA
# FFA is right-lateralized, ~vertices 9900-10400 (fsaverage5)
FFA_RIGHT = (9900, 10400)
FFA_LEFT  = (1100, 1600)   # smaller response expected

face_df   = model.get_events_dataframe(video_path="faces_stimulus.mp4")
object_df = model.get_events_dataframe(video_path="objects_stimulus.mp4")

face_preds,   _ = model.predict(events=face_df)
object_preds, _ = model.predict(events=object_df)

# Compute selectivity index
def roi_mean(preds, vertex_range):
    v_start, v_end = vertex_range
    return preds[:, v_start:v_end].mean()

face_ffa_r   = roi_mean(face_preds, FFA_RIGHT)
object_ffa_r = roi_mean(object_preds, FFA_RIGHT)
face_ffa_l   = roi_mean(face_preds, FFA_LEFT)

selectivity_index = (face_ffa_r - object_ffa_r) / (face_ffa_r + object_ffa_r + 1e-8)

print(f"FFA-right (faces):   {face_ffa_r:.4f}")
print(f"FFA-right (objects): {object_ffa_r:.4f}")
print(f"FFA-left  (faces):   {face_ffa_l:.4f}")
print(f"Selectivity index:   {selectivity_index:.4f}")

# Positive selectivity_index = FFA prefers faces
# Right > Left = expected right lateralization
# This replicates Kanwisher 1997 in-silico

See paradigm-library.md for 8 fully documented paradigms with expected results and vertex ranges.

---

Workflow D: Hypothesis Testing

Trigger: You have a hypothesis like "visual cortex responds to motion but not to static images" and want to test it computationally.

Step 1: Define your hypothesis formally

H0: mean_activation(MT_right, motion_video) == mean_activation(MT_right, static_image_video)
H1: mean_activation(MT_right, motion_video) > mean_activation(MT_right, static_image_video)
Region: MT/V5 right hemisphere, vertices 7800-8200 (approx fsaverage5)

Step 2: Generate contrasting stimuli

The stimuli must differ only on the dimension you're testing. For motion vs. static:

• Motion: videos with global optic flow (dot fields, moving gratings)
• Static: same scene photographed repeatedly (no temporal change)

Step 3: Run predictions and compute the contrast

from tribev2 import TribeModel
import numpy as np
from scipy import stats

model = TribeModel.from_pretrained("facebook/tribev2", cache_folder="./cache")

MT_RIGHT = (7800, 8200)

# Multiple clips per condition for effect size estimation
motion_clips  = ["motion_01.mp4", "motion_02.mp4", "motion_03.mp4"]
static_clips  = ["static_01.mp4", "static_02.mp4", "static_03.mp4"]

def get_roi_activation(clips, roi):
    activations = []
    for clip in clips:
        df = model.get_events_dataframe(video_path=clip)
        preds, _ = model.predict(events=df)
        v_start, v_end = roi
        activations.append(preds[:, v_start:v_end].mean())
    return np.array(activations)

motion_acts = get_roi_activation(motion_clips, MT_RIGHT)
static_acts = get_roi_activation(static_clips, MT_RIGHT)

# One-tailed t-test
t_stat, p_val = stats.ttest_ind(motion_acts, static_acts, alternative='greater')
effect_size   = (motion_acts.mean() - static_acts.mean()) / np.std(np.concatenate([motion_acts, static_acts]))

print(f"Motion MT activation:  {motion_acts.mean():.4f} ± {motion_acts.std():.4f}")
print(f"Static MT activation:  {static_acts.mean():.4f} ± {static_acts.std():.4f}")
print(f"t = {t_stat:.3f}, p = {p_val:.4f}")
print(f"Cohen's d = {effect_size:.3f}")
print(f"H1 supported: {p_val < 0.05 and t_stat > 0}")

Step 4: Interpret and decide whether to proceed to real fMRI

If the in-silico result supports H1 with d > 0.5, the effect size is large enough to power a real study. Use TRIBE v2 output to:

• Estimate required sample size (TRIBE v2 predictions correlate with real fMRI at r~0.6-0.8)
• Identify best ROIs to measure in-scanner
• Pre-register your analysis plan

---

Output Interpretation

Shape: (n_timesteps, n_vertices)

preds, segments = model.predict(events=df)

# preds.shape[0] = number of TRs (fMRI volumes)
#   Each TR ≈ 1.5 seconds (typical fMRI repetition time)
#   Total duration covered = n_timesteps × 1.5s

# preds.shape[1] = 20,484 vertices (fsaverage5 surface)
#   Vertices 0–10,241      = Left hemisphere
#   Vertices 10,242–20,483 = Right hemisphere

What the values mean

TRIBE v2 outputs z-scored BOLD signal predictions in arbitrary units:

• 0.0 = mean response for this brain region (no activation above baseline)
• > 0 = above-average activation
• < 0 = below-average (suppression or deactivation)
• Typical range: [-3.0, 3.0]

The 5-second temporal offset

TRIBE v2 automatically applies a +5s hemodynamic lag. The prediction at timestep t reflects neural processing that occurred at t - 5s in the stimulus. You do not need to manually shift; the model handles this.

Vertex-to-region mapping

fsaverage5 has 10,242 vertices per hemisphere. Key landmarks:

Left hemisphere (vertices 0–10,241):
  V1/V2 primary visual:    0–1,500
  V3/V4 ventral stream:   1,500–3,000
  MT/V5 motion:           2,800–3,200
  A1 primary auditory:    3,500–4,200
  Broca's area (44/45):   6,200–6,800
  VWFA (word form):       7,100–7,500

Right hemisphere (vertices 10,242–20,483):
  V1/V2 primary visual:  10,242–11,742
  FFA (face area):        9,900–10,400  ← note: RH vertex numbers
  PPA (place area):      10,400–11,000
  A1 primary auditory:   13,742–14,442

See references/brain-regions.md for the full atlas with all regions, hemispheres, and vertex ranges.

Segments

preds, segments = model.predict(events=df)
# segments: list of (start_sec, end_sec) tuples
# Corresponds to natural scene/speech boundaries the model detected
# Useful for aligning predictions to stimulus timing

---

Using the Companion Scripts

Single Prediction

python scripts/predict_brain.py \
  --input stimulus.mp4 \
  --modality video \
  --output results/predictions.csv \
  --cache-dir ./model-cache

Output CSV columns: timestep, vertex_id, predicted_activation

Also prints top-5 vertices at peak timestep to stdout.

Batch Experiment

python scripts/run_experiment.py \
  --stimuli-dir stimuli/faces/ \
  --modality video \
  --output-dir results/face_experiment/ \
  --region FFA_right

Output CSV: stimulus_file, region, mean_activation, peak_timestep

---

Common Pitfalls

Issue	Cause	Fix
preds.shape[1] != 20484	Wrong surface resolution	Verify facebook/tribev2 loaded, not v1
All activations near 0	Stimulus too short	Use clips > 10 seconds; TRIBE v2 needs sufficient temporal context
Right-hemisphere FFA vertex range seems off	Vertex indexing	RH vertices start at 10,242; FFA_right is still ~9,900-10,400 in the combined array
Memory error on GPU	Long video, full batch	Pass --chunk-duration 30 to process in 30s windows
get_events_dataframe fails on audio	Wrong sample rate	Convert to 16kHz mono WAV first: ffmpeg -i input.mp4 -ar 16000 -ac 1 audio.wav

---

Detailed References

• references/brain-regions.md — Full cortical atlas: every region, hemisphere, fsaverage5 vertex range, and what activates it
• references/paradigm-library.md — 8 classic paradigms with TRIBE v2 replication protocols and expected results
• scripts/predict_brain.py — CLI for single-stimulus prediction with CSV output
• scripts/run_experiment.py — Batch multi-stimulus experiment runner with region-averaged output