Openevidence performance tuning

OpenEvidence Performance Tuning

Overview

OpenEvidence's clinical API handles evidence query response times, citation batch retrieval, and complex multi-condition query optimization. Clinical evidence queries can take 2-5 seconds as the system searches across thousands of medical studies and synthesizes responses. Citation batch retrieval for systematic reviews generates heavy load when fetching 50-200 references per query. Caching evidence responses, batching citation fetches, and optimizing query specificity reduces clinician wait times by 50-70% and keeps complex queries within acceptable latency bounds.

Caching Strategy

const cache = new Map<string, { data: any; expiry: number }>();
const TTL = { evidence: 1_800_000, citations: 3_600_000, queries: 300_000 };

async function cached(key: string, ttlKey: keyof typeof TTL, fn: () => Promise<any>) {
  const entry = cache.get(key);
  if (entry && entry.expiry > Date.now()) return entry.data;
  const data = await fn();
  cache.set(key, { data, expiry: Date.now() + TTL[ttlKey] });
  return data;
}
// Citations are stable (1hr). Evidence summaries update with new studies (30 min).

Batch Operations

async function fetchCitationsBatch(client: any, citationIds: string[], batchSize = 25) {
  const results = [];
  for (let i = 0; i < citationIds.length; i += batchSize) {
    const batch = citationIds.slice(i, i + batchSize);
    const res = await Promise.all(batch.map(id => client.getCitation(id)));
    results.push(...res);
    if (i + batchSize < citationIds.length) await new Promise(r => setTimeout(r, 500));
  }
  return results;
}

Connection Pooling

import { Agent } from 'https';
const agent = new Agent({ keepAlive: true, maxSockets: 6, maxFreeSockets: 3, timeout: 30_000 });
// Moderate socket count — evidence queries are sequential, citations parallel

Rate Limit Management

async function withRateLimit(fn: () => Promise<any>): Promise<any> {
  try { return await fn(); }
  catch (err: any) {
    if (err.status === 429) {
      const retryMs = parseInt(err.headers?.['retry-after'] || '10') * 1000;
      await new Promise(r => setTimeout(r, retryMs));
      return fn();
    }
    throw err;
  }
}

Monitoring

const metrics = { queries: 0, citationFetches: 0, cacheHits: 0, avgLatencyMs: 0, errors: 0 };
function track(op: 'query' | 'citation', startMs: number, cached: boolean) {
  metrics[op === 'query' ? 'queries' : 'citationFetches']++;
  const lat = Date.now() - startMs;
  metrics.avgLatencyMs = (metrics.avgLatencyMs * (metrics.queries - 1) + lat) / metrics.queries;
  if (cached) metrics.cacheHits++;
}

Performance Checklist

• Cache evidence responses with 30-min TTL (studies update periodically)
• Cache citation metadata with 1-hour TTL (stable once published)
• Batch citation retrieval in groups of 25 with 500ms pauses
• Use specific condition + intervention queries instead of broad searches
• Prefetch commonly queried drug interaction evidence
• Enable HTTP keep-alive for persistent API connections
• Monitor average query latency (target < 3s for simple queries)
• Set client timeout to 30s for complex multi-condition queries

Error Handling

Issue	Cause	Fix
Slow evidence query (> 5s)	Broad multi-condition search	Narrow query to specific condition + intervention
429 on citation batch	Too many parallel citation fetches	Batch to 25, add 500ms delay between groups
Stale evidence summary	Cache too long for rapidly evolving topic	Reduce TTL for high-churn topics (e.g., COVID)
Timeout on complex query	Multi-study synthesis exceeding limit	Increase timeout to 30s, simplify query scope
Missing citations	Study not yet indexed	Retry after 24h, check study publication date

Resources

• OpenEvidence Platform
• OpenEvidence API Docs

Next Steps

See openevidence-reference-architecture.