Argument logic audit

Argument Logic Audit

Test whether the stated premises and evidence support the stated conclusion. Preserve scope, definitions, uncertainty, and alternative explanations; do not infer author intent from a reasoning error.

Required workflow

1. Identify the central claims before searching for flaws. Record the exact source location, claim type, population, intervention or exposure, outcome, comparison, time, geography, estimand, modality, and stated uncertainty.

2. Build a claim-premise-evidence-inference-conclusion graph. Use typed nodes for:

   • claim or intermediate proposition;
   • premise or assumption;
   • evidence, including data, table, figure, citation, or theorem;
   • inference_rule, estimator, logical step, or mathematical operation;
   • conclusion.

   Use explicit edges such as supports, depends_on, defines, qualifies, contradicts, and generalizes. Attach source locations to nodes and state unstated premises as inferred rather than reported.

3. Normalize each proposition without strengthening it. Preserve quantifiers, comparison groups, signs, thresholds, probability language, population, time, and causal versus associational wording. Resolve pronouns and notation, but retain ambiguity when the text does not resolve it.

4. Test each inference against the challenge patterns below. Search the paper, supplement, cited source, and relevant table or equation for both supporting and opposing evidence.

5. Trace the impact of a broken edge. State whether it affects one sentence, a secondary interpretation, a mechanism, an identification claim, or a central conclusion.

Required challenge patterns

• Internal contradiction: Compare definitions, hypotheses, sample descriptions, signs, values, method statements, results, abstract, and conclusion under the same scope. Do not combine different versions, populations, models, or conditional statements into a false contradiction.
• Necessary versus sufficient: Check whether a necessary condition is treated as sufficient, whether the converse or inverse is assumed, and whether failure of one sufficient route is mistaken for impossibility.
• Circular reasoning: Check whether the conclusion, a relabeling of it, or an estimate defined from it is used as its own premise or validation criterion.
• Non-rejection versus equivalence: Treat failure to reject a difference as inconclusive. Require a prespecified equivalence margin and an appropriate interval or equivalence test before accepting an equivalence claim.
• Association versus causation: Require a design, estimand, and identifying assumptions that support the causal language. Load $causal-claims-audit when causal identification is central; statistical adjustment or temporal ordering alone does not establish causality.
• Proxy versus truth: Distinguish a construct from a proxy, model prediction, imputed value, list estimate, diagnostic score, or self-report. Require independent validation before treating a proxy as ground truth.
• Local versus general: Check transport from the analyzed sample, site, period, treatment version, or complier group to the claimed target population. Precision inside the study does not establish external validity.
• Average versus subgroup: Do not infer subgroup effects, individual effects, sign consistency, or absence of heterogeneity from an average effect. Require the relevant interaction, subgroup estimand, uncertainty, and multiplicity treatment.
• Mechanism: Separate evidence that an effect exists from evidence identifying a pathway. Check mediator timing, measurement, alternative pathways, post-treatment conditioning, and assumptions needed to exclude rival mechanisms.
• Temporal reasoning: Verify that the proposed cause precedes the effect and that eligibility, exposure, mediator, outcome, and follow-up windows are compatible. Distinguish measurement time from event time and account for anticipation, reverse causation, and immortal-time or look-ahead bias where relevant.
• Citation support: Inspect the cited source itself. Check that it supports the precise proposition, population, direction, magnitude, and conditions attributed to it. Citation existence, metadata, abstract similarity, or a citation chain is not substantive support.
• Mathematical domain and branches: Record definitions, domain, range, units, sign convention, denominator restrictions, boundary cases, monotonicity, and every piecewise branch. Verify that transformations preserve direction and that a derivation for one sign, tail, reporting direction, or parameter region is not silently reused in another.

Classify the result

Use deterministic_contradiction only when all of the following hold:

• the relevant text, table, or equation is verified in the source;
• the propositions use the same definitions, scope, conditions, and version;
• two or more normalized propositions cannot all be true;
• feasible rounding, notation, OCR, conditional branches, and documented alternatives do not reconcile them.

Use formal_inference_failure when verified premises do not entail the stated conclusion under an explicit logical, mathematical, or statistical rule. This status identifies a broken edge; it does not establish that the conclusion is false in the world.

Judge the documented argument separately from the truth of the conclusion. If the audited paper and its available cited support assert the conclusion but omit a premise required for entailment, the documented edge may be a verified formal_inference_failure while the conclusion's real-world truth remains unresolved. If an identified citation, appendix, definition, or other claimed support is unavailable for inspection, use unresolved until that source is checked.

Use judgment_required when the relevant materials are available but the concern depends on contestable assumptions about evidence sufficiency, construct validity, causal identification, mechanisms, transportability, or citation interpretation. State the disputed premise and evaluate the strongest reasonable alternative explanation.

Use unresolved instead of judgment_required when a required source, definition, branch, model detail, or legible rendering is unavailable. Record what evidence would decide the issue rather than filling the gap with a substantive judgment.

Map statuses into the enclosing audit explicitly:

Reasoning status	Evidence class	Verification status
deterministic_contradiction	logic_or_argument_issue	verified only after all source and scope conditions above pass
formal_inference_failure	logic_or_argument_issue	verified only for the broken entailment, not for the conclusion's truth
judgment_required	logic_or_argument_issue or identification_or_method_issue	unresolved
unresolved	unresolved_anomaly	unresolved or blocked, according to artifact availability

Use identification_or_method_issue for a contestable design, estimator, variance, or identifying-assumption judgment. Use logic_or_argument_issue for evidence sufficiency, construct substitution, citation entailment, mechanism, scope, or generalization. Create separate findings when both occur rather than assigning two causes to one vague concern.

Use verification status blocked only when a named artifact, access restriction, or failed required step prevents the planned check after a documented attempt. Use unresolved when the available evidence is inconclusive or clarification could resolve it.

If a deterministic numerical relationship should be run through $statistical-consistency-audit but required metadata or a writable case-file location is unavailable, label the check not_executed, preserve any conditional hand calculation separately, and do not present it as checker output.

Rules

• Distinguish inconsistency in the document from falsity in the world.
• Distinguish a missing justification from a proven false conclusion.
• Do not use automated language-model agreement, citation counts, or a plausibility score as proof of logical validity.
• Do not infer deception, fabrication, recklessness, or misconduct. Describe the exact broken or unsupported graph edge.
• Report counterevidence and benign interpretations before escalating impact.
• A clean logic audit means only that the inspected claims survived the listed checks within scope.

Output

Return:

1. scope, artifact versions, and central-claim inventory;
2. the claim graph as a compact node-and-edge table;
3. findings with exact locations, normalized propositions, the challenged edge, reasoning status, counterevidence, and alternative explanations;
4. deterministic contradictions, formal inference failures, judgment-required issues, and unresolved questions in separate sections;
5. local and central-conclusion impact, checks that passed, and the evidence needed to resolve each open issue.