Post connection data scope filtering

Choosing a Data-Scope Filtering Strategy

Customers almost never want every record their connected system has. An HRIS customer might want only active employees in specific departments. An ATS customer might want only candidates past a certain pipeline stage. A CRM customer might want only accounts owned by their team. A ticketing customer might want only tickets from one project or above a priority threshold.

Decide the filtering approach before a live customer forces the decision, because changing it later usually requires a re-sync.

Prerequisites

• Initial sync logic working (sync-implement-webhooks and/or sync-implement-polling)

Before Proceeding

Three pieces of information are needed before writing any filter logic.

If invoked from implementing-post-connection, the first two were answered in Step 1 — use that context. Otherwise, gather them now:

• Categories: Which Merge categories need filtering? (hris, ats, crm, accounting, ticketing) — drives which filter parameters apply.
• Existing sync logic location: Where is the sync code (webhook handler / polling job)? Filters apply inside it. Search for files that already call /{category}/v1/{model} or use modified_after.
• Filtering strategy preference: pre-storage (Strategy 1) or post-storage (Strategy 2) — see the two-strategies section below. The recommendation is post-storage unless storage cost is a concern.

Ask the user before continuing:

"Strategy 1 (pre-storage) keeps your DB smaller but requires a full re-sync if filter criteria change later. Strategy 2 (post-storage) is more flexible but stores all records. Which would you like — and which categories should I scope the filtering to?"

Wait for the answer before proceeding.

The two strategies

The same trade-off applies regardless of category. Pick once per integration; document the choice.

Strategy 1: Pre-storage filtering (Merge API filters)

Apply filters inside your sync logic before writing records to your DB. Each record is evaluated at ingestion; non-matching records are discarded.

• Pros: smaller DB, cleaner data, fewer records to query downstream.
• Cons: changing filter criteria requires a full re-sync to backfill previously excluded records; more complex sync logic; you can't show customers "what they would have synced" if they widen filters.

Strategy 2: Post-storage filtering (product filters)

Store ALL records from Merge in a staging table (or with an included flag), and apply filters dynamically at query time or via a separate filtering pass.

• Pros: filter criteria can change without a re-sync; easier to iterate in UI; "preview what changes if I widen this filter" becomes feasible.
• Cons: more data stored; DB queries must always carry filter predicates.

Recommendation

Start with post-storage filtering (Strategy 2) unless storage cost is a meaningful concern. It's easier to tighten filters later than to backfill excluded records.

Filtering by category — Merge API parameters

The Merge query params differ per category. The decision framework above is the same; only the filters themselves change. Use these as starting points; consult Merge's category-specific common-model docs for the full set.

HRIS — employee filtering

Parameter	Values	Notes
employment_status	ACTIVE, INACTIVE, PENDING	Most common filter
employment_type	FULL_TIME, PART_TIME, etc.	Filter by work arrangement
groups	Group UUIDs	Filter by department, location, subsidiary
expand	employment, locations, manager	Include nested objects in response

Example: GET /hris/v1/employees?employment_status=ACTIVE

ATS — candidate / application filtering

Parameter	Values	Notes
current_stage	Stage UUIDs	Filter applications by pipeline stage
credited_to	User UUID	Filter by recruiter / hiring manager
job_id	Job UUID	Filter to candidates for one job
expand	applications, jobs, current_stage, etc.	Include nested objects

Example: GET /ats/v1/candidates?expand=applications, then post-filter by applications[].current_stage if pre-storage by stage.

CRM — account / contact / opportunity filtering

Parameter	Values	Notes
owner_id	User UUID	Filter to records owned by a specific user
account_id	Account UUID	Filter contacts/opportunities to one account
stage	Stage UUID	Filter opportunities by sales stage
expand	account, owner	Include nested objects

Example: GET /crm/v1/accounts?owner_id={uuid}

Ticketing — ticket / project filtering

Parameter	Values	Notes
project_id	Project UUID	Filter tickets to a specific project
status	Status UUID	Filter by ticket status
priority	Priority value	Filter by priority threshold
assignee_ids	User UUIDs (list)	Filter to tickets assigned to specific users
expand	assignees, account, collections	Include nested objects

Example: GET /ticketing/v1/tickets?project_id={uuid}&priority=HIGH

Accounting — record filtering

Parameter	Values	Notes
account_type	ASSET, LIABILITY, etc.	Filter chart-of-accounts entries
transaction_date_after	ISO date	Filter transactions by date range
company_id	Company UUID	Filter to one company in a multi-entity setup

Example: GET /accounting/v1/accounts?account_type=ASSET

If you're building a filter UI for customers

The skeleton is identical regardless of category — substitute "employees", "candidates", "accounts", "tickets", or "transactions" as appropriate.

1. Populate filter options — internal API fetching the available filter values (unique departments, stages, owners, projects, statuses, etc.) from your stored data or Merge's API.
2. Store filter selections — backend table keyed by linked account ID.
3. Save filter selections — internal API endpoint to write selections.
4. Apply filters in sync — either as Merge API query params (Strategy 1) or as predicates on your staging table (Strategy 2).
5. Define filter-change behavior — when a customer updates filters:
   • Re-sync newly included records? (Required if Strategy 1.)
   • Mark previously synced but newly excluded records as inactive / hidden?
   • Notify downstream systems of scope change?

Implementation checklist

• Filtering strategy chosen (pre-storage or post-storage) and documented per integration
• Internal API built to populate filter options in product UI
• Backend table stores filter selections per linked account
• Internal API saves filter selections
• Sync logic incorporates stored filters (in webhook handler and/or polling job)
• Filter-change behavior defined (re-sync? mark inactive? notify?)
• Category-appropriate Merge API filters identified and tested

Troubleshooting

SYMPTOM: Customer changed filter criteria; old records that should now be included are missing CAUSE: Pre-storage filtering was used; previously excluded records were never written to your DB. FIX: Trigger a re-sync that uses the updated filter criteria. Or migrate to post-storage filtering so future filter changes don't require re-syncs.

SYMPTOM: DB queries are slow because every query has filter predicates CAUSE: Post-storage filtering is in use without supporting indexes. FIX: Index the columns used in filter predicates (e.g., employment_status, current_stage, owner_id). For high-cardinality filters, consider materialized views.

SYMPTOM: "Active employees" filter syncs employees that the customer's HRIS shows as terminated CAUSE: Merge's employment_status reflects the provider's status field at sync time; some providers update statuses on a delay or have intermediate states (e.g., PENDING_TERMINATION). FIX: Cross-check Merge's employment_status with termination_date (if present) or a provider-specific custom field, then re-evaluate the filter rule.