Abstract

Automated fact-checking systems still fall short of producing explanations that mirror the depth and structure of expert human reasoning. In this work, we propose a multi-agent framework that integrates five specialized linguistic agents covering polarization, linguistic style, argumentation, plausibility, and contextual framing with web-based evidence retrieval, synthesized by a supervisor agent into structured reports resembling professional fact-checking outputs. We evaluate the framework on a dataset of fact-checked Brazilian news through a classification benchmark and two further quantitative studies of explanation quality, addressing: (1) Do the generated reports elicit reader confidence comparable to reports written by professional fact-checkers? and (2) Which explanatory dimensions most influence reader confidence? The classification benchmark shows the framework performs competitively with strong baselines. A blinded within-subjects study with 95 participants, analyzed via Linear Mixed Models, shows that post-verification confidence reaches levels statistically indistinguishable from expert-written reports, with plausibility and analytical depth as the strongest predictors of confidence gain and depth being especially important for implausible claims. Complementary LLM-as-a-judge experiments corroborate these findings, showing the framework's explanations are consistently preferred for depth, persuasion, and plausibility.

1 Introduction

The rapid spread of misinformation is one of the defining challenges of the information age, ranked among the most severe global threats by the World Economic Forum's 2026 Global Risks Report,1 with documented consequences for electoral integrity [18], climate action [5, 24], and public health [2, 21]. Brazil is a particularly relevant context for studying this problem: growing reliance on social media as a primary news source2 coincides with widespread distrust of professional journalism—nearly half of Brazilian internet users report being consistently skeptical of it, a higher share than for content from friends, family, or messaging apps3—while most automated fact-checking systems in the literature remain evaluated on English datasets, leaving this landscape underrepresented.

Fact-checking also faces a structural asymmetry: producing misinformation is trivial, whereas debunking it requires substantial effort and expertise [23], and misinformation itself exploits psychological mechanisms such as fear, anger, and confirmation bias to spread [19]. This creates an explainability gap: fact-checking reports are persuasive artifacts, not merely informational ones, and explanations that fail to engage readers’ prior beliefs or simply restate the claim can backfire, reinforcing the misinformation they aim to correct [14]. Effective fact-checking therefore requires explanations that are linguistically and cognitively aligned with how humans interpret and evaluate information.

Multi-agent architectures offer a promising paradigm for this problem, decomposing verification into specialized, interacting components that enable modular reasoning and inspection of intermediate outputs. However, existing systems primarily use this paradigm to improve retrieval quality or verdict consistency, treating explanation as a secondary, post-hoc outcome rather than a first-class objective—falling short of the linguistic, rhetorical, and contextual richness of human fact-checking reports.

A substantial body of psycholinguistic research shows that deceptive content leaves identifiable linguistic traces—reduced narrative specificity [6], emotional manipulation [19], detectable stylistic patterns [32], and argumentation fallacies [27, 28]—yet existing fact-checking systems treat these signals in isolation rather than integrating them into a unified, interpretable framework.

In this work, we address this gap with a linguistically grounded, multi-agent architecture that operationalizes diverse strands of linguistic theory into specialized, auditable agents producing structured explanations alongside verification outcomes. We evaluate the system through a quantitative classification benchmark and two further quantitative studies of explanation quality: a blinded human study and an LLM-as-a-judge comparison against a baseline. Concretely, we ask: RQ1—do the linguistic agents produce explanation reports comparable in quality to those written by professional fact-checkers? and RQ2—which linguistic dimensions most strongly drive reader confidence in a verification report?

Our main contributions are:

A holistic multi-agent framework for fact-checking that produces tri-class verdicts (True, False, Inconclusive) alongside structured explanatory reports;\

A blinded within-subjects user study (N = 95) comparing AI-generated and expert-written explanations;\

Five linguistically grounded agents—polarization, linguistic style, argumentation, plausibility, and contextual framing—each producing interpretable, auditable explanations grounded in established literature.\

2 Related Work

The earliest automated fact-checking systems addressed the problem by extracting features and training supervised classifiers [12]. The emergence of large language models has shifted the paradigm toward end-to-end pipelines capable of producing explainable textual reports [4, 7, 30]. FOLK [30] translates claims into first-order logic predicates and verifies each sub-claim, while FACTOOL [4] operates across multiple domains by chaining claim extraction, query generation, and evidence retrieval. MiniCheck [25] achieves near GPT-4 performance at a fraction of the cost by distilling a small verifier from synthetic data. These systems share a common pattern: decompose the claim, retrieve evidence, classify, and optionally justify. None of them dedicates specialized components to the internal linguistic properties of the text being checked.

ARG [9] uses an LLM as a ‘counselor’ that generates rationales to guide a smaller supervised classifier; L-Defense [29] proposes a defense-based framework that uses wisdom within sampled confronting evidence to form two competing parties that provide rationales to mitigate majority-vote bias. Both require fine-tuning on labeled data and do not decompose linguistic analysis into interpretable dimensions.

Bo Wang et al. [29] and QRAFT [22] generate full-length verification articles, and QRAFT is the first to evaluate AI reports against human-written ones, though with expert annotators rather than the ordinary readers who are the actual audience for such reports. We replicate and extend this finding with a randomized, blinded, within-subjects design targeting ordinary readers, and additionally identify which specific report characteristics drive their confidence (Section 5).

Our work fills these gaps by proposing a framework that explains each agent's output with reason and grounds the report's reasoning in linguistic evidence, informed by theoretical frameworks and a web-first, iterative evidence-retrieval approach. This linguistic grounding draws on a substantial body of psycholinguistic research on deception cues: Bella M. DePaulo et al. [6] landmark meta-analysis of 1,338 cue estimates across 158 behavioral indicators establishes that deceivers produce less spontaneous, less detailed, and less convincing narratives, while Lina Zhou et al. [32] formalize nine linguistic constructs—quantity, diversity, complexity, specificity, expressivity, informality, affect, uncertainty, and non-immediacy—into the Linguistics-Based Cues (LBC) framework, enabling computational operationalization of deception signals.

Sarah Ita Levitanm, Angel Maredia & Julia Hirschberg [13] demonstrate that automated systems capture subtle linguistic patterns that humans miss in deceptive dialogs, including complexity markers associated with fabrication. Armen E. Allahverdyan & Aram Galstyan [1] model how confirmation bias drives opinion polarization in social networks, motivating our polarization agent. Santos et al. [11] show that LLMs can identify and mitigate polarizing language in social media text without altering factual content, validating the feasibility of polarization detection in our context. Jean H. M. Wagemans [27, 28] systematizes argumentation analysis through a ‘Periodic Table of Arguments’, enabling computational identification of formal and informal fallacies — the theoretical backbone of our argumentation agent. In Section 3, we ground each linguistic agent in the related works that inspired it; the architecture that hosts them follows a separate, complementary line of work on multi-agent fact-checking systems.

Recent work has explored multi-agent architectures as a means to decompose fact-checking into coordinated reasoning steps. FactAgent [15] models the verification process as an agentic workflow that emulates human experts. Trinh et al. [26] propose a sequential pipeline for claim decomposition, query generation, retrieval, and verdict classification. LoCal [16] emphasizes logical and causal consistency across sub-claims to mitigate hallucinations. Tool-MAD [10] introduces a multi-agent debate setting with heterogeneous tool use, while MERMAID [3] incorporates long-term memory to improve efficiency and consistency across related claims. A common pattern across these approaches is centralized orchestration, where a supervisor agent coordinates specialized sub-agents [10, 26], which we adopt as an implementation backbone. As discussed in Section 1, however, these systems treat explanation as a secondary, post-hoc outcome; our linguistically grounded agents make it a first-class objective, and we complement the largely automated-metric evaluations in this literature with a human-centered study of which explanatory dimensions drive reader confidence.

3 Framework Architecture

The proposed architecture was developed using the Python library LangGraph 4 and follows a centralized orchestration pattern [3, 10, 16, 26], enabling finer control of information flow, a modular-first design, and per-agent explainability. Figure 1 displays the proposed architecture and its components.

A supervisor agent receives a news article, decomposes it into atomic, verifiable claims, and distributes both the full article and its extracted claims to two groups of agents operating in parallel. The linguistic group (Section 4) generates internal artifacts for each agent, including both the results and their corresponding explanations. The web group executes an iterative Deep Research protocol—formulating research plans, decomposing them into subtasks, performing cyclic retrieval–reflection loops based on intermediate results, and synthesizing external evidence with source citations. Both sets of artifacts are returned to the supervisor, who generates the final verdict and report following a human-based reporting style.

Figure 1: Proposed framework architecture and its components: (i) supervisor agent; (ii) linguistic agents; and (iii) web agents.

During final report generation, all previously produced artifacts are aggregated by the supervisor and synthesized into a structured verification report following a Lupa-inspired5 fact-checking methodology: (i) the claim under verification; (ii) the impact of each finding; (iii) identified inconsistencies and gaps; (iv) a conclusion with a three-class verdict (True/Inconclusive/False), where the Inconclusive category aggregates multiple Lupa labels that do not correspond to clearly true or false claims; and (v) an enumerated list of sources. Linguistic findings are integrated directly into the narrative, making the explanation accessible to a general audience. This methodology was developed based on manual analysis of the types of content addressed in Lupa's verification articles, translated into a zero-shot instruction prompt.

3.1 Linguistic Agents

The linguistic component comprises five agents, each targeting a distinct analytical dimension of the text (Table 1). All agents receive the full news article and its extracted claims as input, and produce structured JSON artifacts containing both categorical labels and passage-level justifications, enabling auditors or journalists to inspect why the system flagged specific text as polarizing, fallacious, or implausible. The full implementation, including all prompts, is available in the project repository.

Table 1: The five linguistic agents: theoretical grounding and detected signals.

Agent	Theoretical basis	What it detects
Polarization	Santos et al. [11]; Shu et al. [23]; deceptive news exploits anger, fear, and group identity to spread [19]	Toxic/hate language, emotional appeals, conspiracy narratives, dehumanization, false criminal imputation, in-/out-group framing.
Linguistic style	LBC framework [32]; complexity as a deception cue [13]	Objectivity, formality, persuasiveness, complexity, readability (Flesch-Kincaid, SMOG).
Argumentation	Wagemans’ argumentation typology [27, 28]	Premises, conclusions, formal/informal fallacies (ad hominem, strawman, false cause, slippery slope, etc.).
Plausibility	Confirmation-bias model [1]; implausibility as a fabrication cue [6]	Plausible/implausible classification via common-sense reasoning, logical coherence, social norms.
Context	HeteroSGT [31]; LESS4FD[17]	Named entities, up to 18 thematic domains, feeding the plausibility and supervisor agents.

3.2 Web agents

The web agents complement the linguistic agents by gathering external evidence online and analyzing how it can be used to support or refute the claims identified in the news article.

For each atomic claim extracted by the supervisor, the web agent executes a four-stage iterative Deep Research protocol: (i) the claim is reformulated into a structured, semantically grounded research plan that reduces search ambiguity; (ii) a research supervisor sub-agent decomposes it into independent investigative sub-tasks, each targeting a distinct informational gap; (iii) dedicated researcher sub-agents iteratively generate queries, retrieve and scrape pages via Tavily6, and reflect on evidential sufficiency, bounded by a query budget of 5 iterations per question; and (iv) the evidence is synthesized into a structured artifact with the original claim, research question, narrative synthesis, and an enumerated list of sources with URLs.

4 Experimental Setup

The framework uses GPT-4.1-mini (temperature = 0) for deterministic extraction tasks and GPT-4.1-mini/GPT-5.27 for verification under two experimental regimes: with knowledge cutoff (or KC, using GPT-4.1-mini), restricting web retrieval to content published before the news date and restricting access to news published after the model training cutoff date, simulating first-time verification; and without KC (or no-KC, using GPT-5.2), simulating re-verification of recirculating news where there are no restrictions. The web agents use Tavily for page scraping, and the full pipeline is implemented in Python with LangGraph.

4.1 Dataset

As a case study, we collected 4,522 verified news items from Agência Lupa, one of the most influential fact-checking agencies in Brazil, spanning July 2016 to August 2025. All collected news items are publicly available on the web4. After filtering for post-cutoff date (≥ June 2024), content length (≤ 7,000 characters), and stratified sampling, we obtained 284 instances with verdicts aggregated into three classes: False (244), Inconclusive (40), and True (0). The absence of True instances reflects the natural distribution of journalistic verification datasets. For the human study, we used the 213 instances where the proposed framework's verdict matched Lupa's reference verdict, controlling for verdict-level differences and isolating report quality as the variable of interest.

4.2 Quantitative Evaluation

We compare four configurations to assess whether the proposed framework maintains competitive verification performance while enabling richer explanations: (i) a simple prompt-based baseline equipped with web search; (ii) LoCal [16], a multi-agent system based on logical-causal analysis, adapted to our tri-class setting; (iii) our proposed solution; and (iv) a variant of our approach, focused exclusively on linguistic agents (i.e., excluding web-search capabilities). To ensure fair comparison, all methods use the same Tavily retrieval mechanism and are evaluated under both knowledge-complete (with-KC) and knowledge-constrained (no-KC) regimes.

The baseline configuration employs a concise prompt designed to emulate how a typical end-user would query a general-purpose chatbot (e.g., ChatGPT, Claude or Grok) for news verification with as little contextual input provided as possible: only the news article to be analyzed and the deadline for web search. It uses the same underlying model as the proposed framework in each regime (GPT-4.1-mini under with-KC, GPT-5.2 under no-KC). The prompt asks the model to return a JSON object with a three-class verdict (true/false/inconclusive) and a justification; the full prompt text is available in the project repository.4

The LoCal method, as originally proposed, classifies news claims as either True or False through an iterative process of logical and causal evaluation, without an intermediary class. To allow proper comparison with the proposed framework across all three verdict classes (True/Inconclusive/False), the method was adapted to output Inconclusive when it has exhausted its three iterations — consistent with the original implementation — without reaching a consensus through its logical and causal mechanism.

Table 2 reports macro F1, weighted F1, macro precision, and macro recall. Against LoCal, the proposed framework matches or exceeds it on most metrics in both regimes, with macro precision the only consistent gap. Against the simple prompt-based baseline, the framework trails on macro F1 and precision but achieves higher macro recall, most notably in the no-KC regime; its weighted F1 stays close to or above the baseline under with-KC. The linguistic-only variant—which excludes web retrieval entirely—performs nearly on par with the complete framework and matches the baseline on macro F1 and recall, indicating that the linguistic layer alone captures substantial signal at lower computational cost.

Summary. Taken together, these results place the proposed framework broadly in line with a state-of-the-art multi-agent system and a strong prompt-based baseline, rather than establishing a clear advantage in raw classification accuracy—which is expected, given that modern LLM-based systems augmented with web search are already highly effective at this task [20]. Our contribution, therefore, is not to replace these systems in terms of accuracy, but to extend them with a principled and effective explanation layer. As we show in the following section, this additional layer translates into measurable gains in human-centered evaluation, improving users’ perception, trust, and interpretation of fact-checking outputs.

This raises a natural architectural question: would it be more effective to decouple classification from explanation, delegating verdict prediction to a stronger external classifier and restricting our agents to explanation generation? Our current design is integrated by choice—agents such as plausibility and argumentation do not merely narrate a given verdict, they contribute evidence toward it, and decoupling them would require re-deriving these linguistic signals conditioned on an externally supplied verdict. Our human and LLM-as-a-judge evaluations assess explanation quality conditioned on the framework's own verdict (or on verdict agreement with the baseline); testing explanation quality under a fully decoupled, externally-classified pipeline is a concrete and promising direction for future work.

Table 2: Evaluation of methods and configurations (with-KC and no-KC) across the metrics macro F1, weighted F1, macro precision, and macro recall.

Method	Conf.	F1-M	F1-W	P-M	R-M
Baseline	wKC	0.39	0.72	0.42	0.60
Baseline	noKC	0.44	0.85	0.50	0.62
LoCal [16]	wKC	0.29	0.76	0.44	0.45
LoCal [16]	noKC	0.31	0.73	0.35	0.43
Proposed (Complete)	wKC	0.34	0.76	0.36	0.48
Proposed (Complete)	noKC	0.42	0.79	0.41	0.68
Proposed (Ling.-only)	wKC	0.39	0.75	0.42	0.59

5 Human Study: Evaluating Linguistic Explainability

We conducted a blinded, anonymous within-subjects study to evaluate the impact of linguistically grounded explanations on users’ confidence and perception of fact-checking reports. Each of the 95 participants evaluated four news items—two accompanied by expert reports from Agência Lupa and two by reports generated by our framework—presented in random order without source attribution. Participants were recruited via Prolific (n = 50, compensated at £2.50 for ∼ 15 minutes) and voluntary academic channels (n = 45). Inclusion criteria included Brazilian nationality, Portuguese as a native language, a 90–100% Prolific approval rate, and willingness to engage with potentially sensitive content.

Participants first completed a profile-mapping step, reporting political orientation (8-point scale from far-right to far-left, plus “no answer”) and their level of news consumption across topics. News items were then selected to minimize prior exposure.

Each evaluation followed two stages. In the Pre-verification stage, participants read the news item and its verdict, rated their confidence on a 5-point Likert scale, and reported prior exposure. In the Post-verification stage, they read the full verification report and re-rated their confidence, in addition to evaluating four report characteristics—Depth, Persuasion, Coherence, and Plausibility—also on 5-point Likert scales.

5.1 Hypotheses

We tested six hypotheses (HP1–HP6). HP1 examines whether verification increases confidence. HP2 evaluates whether confidence gains differ between expert and generated reports. HP3 assesses the effect of prior exposure. HP4 tests whether perceived report characteristics predict changes in confidence. HP5 evaluates the interaction between depth and plausibility. HP6 tests robustness across political orientation and recruitment method.

HP1–HP2 are analyzed using paired t-tests and a Linear Mixed Model (LMM) with confidence as the dependent variable. HP3–HP6 are analyzed using a second LMM with confidence change (post − pre) as the outcome.

5.2 Confidence Change (HP1–HP2)

Both expert and framework-generated reports produced highly significant increases in reader confidence (p < .001 in paired t-tests). The first LMM (Table 3) confirms HP1: the verification moment yields a substantial effect ($hat{beta }1 = 0.632$, SE = 0.090, p < .001), corresponding to an average increase of 0.63 points on the 5-point scale. Random intercepts for participants (σ = 0.440) and items capture individual variability and item difficulty, supporting the use of an LMM.

HP2 is supported by the absence of significant differences between expert and generated reports. The effect of report source is negligible ($hat{beta }2 = 0.005$, p = .955), and the interaction between verification stage and source is not significant ($hat{beta }3 = -0.089$, p = .480). In practice, this indicates that participants do not distinguish between expert-written and framework-generated reports in terms of the confidence they elicit. This result remains robust after controlling for prior exposure, political orientation, and recruitment method (all p > .30).

Table 3: Linear Mixed Model results (LMM 1: confidence post; LMM 2: confidence change).

Predictor	$boldsymbol {hat{beta }}$	SE	p	Res.

LMM 1 — Confidence post
Moment (post vs pre)	0.632	0.090	< .001	HP1 ✓
Source (Lupa vs Prop.)	0.005	0.093	.955	HP2 ✓
Moment × Source	− 0.089	0.125	.480	HP2 ✓

LMM 2 — Confidence change
Previous exposure	0.228	0.109	.037	HP3 ✓
Depth	0.294	0.065	< .001	HP4 ✓
Plausibility	0.166	0.057	.004	HP4 ✓
Persuasion	0.098	0.045	.030	HP4 ✓
Coherence	0.091	0.047	.053	HP4 ∼
Depth × Plausib.	− 0.105	0.041	.011	HP5 ✓
Pol. Orientation	− 0.009	0.023	.699	HP6 ✓
Recruit. Method	− 0.095	0.095	.317	HP6 ✓

5.3 Predictors of Confidence Change (HP3–HP6)

The second LMM (Table 3) models confidence change as the dependent variable. HP3 is supported: prior exposure significantly increases confidence gains ($hat{beta } = 0.228$, p = .037), suggesting that verification is particularly effective when readers are already familiar with the claim.

Regarding HP4, Depth emerges as the strongest predictor of confidence increase ($hat{beta } = 0.294$, p < .001), followed by Plausibility ($hat{beta } = 0.166$, p = .004) and Persuasion ($hat{beta } = 0.098$, p = .030). Coherence shows only a marginal effect ($hat{beta } = 0.091$, p = .053). These findings indicate that readers place greater value on analytical depth and factual grounding than on rhetorical fluency.

HP5 reveals a significant compensatory interaction between Depth and Plausibility ($hat{beta } = -0.105$, p = .011). Simple slope analysis shows that depth is particularly impactful when claims are perceived as implausible, whereas plausibility becomes more influential when depth is limited. This suggests that effective fact-checking explanations adapt to the claim's epistemic difficulty.

Finally, HP6 confirms that the results are robust across political orientation ($hat{beta } = -0.009$, p = .699) and recruitment method ($hat{beta } = -0.095$, p = .317).

6 LLM-as-a-Judge Evaluation: Comparing Report Quality

To complement the human study, we conduct an LLM-as-a-judge [8] evaluation to compare the quality of reports generated by the proposed framework against those produced by the baseline. This experiment uses the same evaluation dimensions as the human study—Depth, Persuasion, Coherence, and Plausibility—but enables a broader, more controlled comparison across a larger set of instances.

Baseline reports were not included in the human study due to respondent fatigue, as participants were already required to evaluate both expert-written reports and those generated by the proposed framework. The LLM-based evaluation, therefore, serves as a complementary analysis, allowing us to directly compare baseline and framework-generated reports under the same conditions.

Additionally, the LoCal method was not submitted to this evaluation because it was not designed to generate an explainable final report for end-users, but rather an interpretable solution that justifies the logical equivalence of its sub-tasks to the original claim and assesses whether this solution survives a counterfactual challenge.

We evaluate 148 report pairs in which both methods produced the same final verdict, thereby controlling for differences in label assignment and focusing the comparison on explanatory quality.

Evaluation protocol. We adopt a single-blind pairwise-comparison design using the Gemma 4 31B model (an LLM from a different family than the GPT models we used for fact-checking).4 Each evaluation instance consists of two reports—denoted as Report A and Report B—presented in random order, with the baseline and framework outputs assigned randomly to each position.

For each of the four evaluation dimensions, the model is asked to select: (i) A, if Report A better expresses the characteristic; (ii) B, if Report B is superior; or (iii) T (tie), if both reports perform similarly. Results are aggregated into a comparison matrix, in which wins are counted per characteristic, and ties contribute equally to both methods.

Table 4 summarizes the outcomes. The proposed framework consistently outperforms the baseline in Depth, Persuasion, and Plausibility across both regimes, by roughly 56–61 percentage points under with-KC and 22–35 points under no-KC. Coherence is the exception, remaining close to balanced (with-KC) or slightly favoring the baseline (no-KC)—likely because coherent, fluent text is already a strength of modern LLMs regardless of the generation pipeline, making it less sensitive to architectural differences than depth or plausibility.

Table 4: LLM-as-a-Judge evaluation results. Proposed Framework vs. Baseline win-rate (%) across characteristics, with and without knowledge context (KC). The gain of the Proposed Framework over the baseline is represented via Δ.

	With KC			No KC
Char.	Prop.	Base.	Δ	Prop.	Base.	Δ
Depth	80.41	19.59	+60.82	67.42	32.58	+34.84
Persuasion	80.41	19.59	+60.82	64.39	35.61	+28.78
Coherence	50.68	49.32	+1.36	44.70	55.30	− 10.60
Plausibility	77.85	22.15	+55.70	60.90	39.10	+21.80

7 Discussion

Returning to RQ1 (report quality) and RQ2 (which dimensions drive confidence), our results suggest that the linguistic analysis layer addresses a genuine gap in the explainability of automated fact-checking. Prior systems either skip internal text analysis entirely [16, 26] or treat it as a secondary, post-hoc annotation [15]. By contrast, our five agents produce structured, passage-level evidence that flows directly into the final report a reader sees: the human study (Section 5) answers RQ1 directly—AI reports are perceptually equivalent to expert reports—while depth and plausibility answer RQ2 as the strongest, mutually compensatory predictors of confidence gain (Section 5.3). This has a concrete design implication: systems optimized purely for classification accuracy may score well on easy, plausible claims but fail on implausible ones that challenge the reader's worldview, whereas a deep report is precisely what sustains confidence when a claim is not plausible on its face. Our argumentation and plausibility agents target exactly these cases, surfacing the fallacies and incoherence that make a claim suspicious. The LLM-as-a-Judge evaluation (Section 6) reinforces this reading: the proposed framework's reports outperform the baseline's on Depth, Persuasion, and Plausibility, and match it on Coherence—so the framework's explanations are not merely on par with experts, but ahead of naive LLM-generated reports as well.

Explanation quality under incorrect verdicts. Both the human study and the LLM-as-a-judge comparison assess explanation quality conditional on a correct verdict—213 instances where the framework agreed with Lupa's ground truth, and 148 where it agreed with the baseline. This isolates report quality from verdict-level disagreement, but leaves an important question open: how do readers respond to a fluent, well-argued explanation attached to an incorrect verdict? Since HP4 shows that depth and persuasion increase reader confidence, the very properties that make our explanations effective when the verdict is right could make a wrong verdict more convincing, and therefore more harmful. Evaluating explanation quality and reader response under incorrect verdicts is an important direction for future work.

8 Conclusion

We introduced a multi-agent fact-checking framework whose central contribution is a linguistically grounded explainability layer built on five specialized agents rooted in psycholinguistic theory. Unlike traditional pipelines that prioritize verdict prediction over explanation, our approach treats explanation as a first-class component of verification, and our quantitative, human-subjects, and LLM-as-a-judge evaluations (Section 4–Section 6) support this design: framework-generated reports are statistically indistinguishable from expert-written ones in the confidence they confer to readers, with depth and plausibility as the strongest, mutually compensatory predictors of that gain. These findings position linguistic explainability as an under-explored, high-impact frontier for interpretable, trustworthy fact-checking systems.

Such systems also hold promise as tools to help professional fact-checkers scale production of high-quality reports in underrepresented languages such as Brazilian Portuguese.

Limitations. This work has the following main limitations. First, the dataset is drawn exclusively from Agência Lupa and, after filtering, contains no True-verdict instances, restricting evaluation across the full tri-class spectrum. Second, the user study was conducted in a controlled survey setting using already verified claims; responses may differ in naturalistic environments where users encounter misinformation organically and under time or attention constraints. Third, the system relies on proprietary large language models (GPT-4.1-mini/GPT-5.2), which introduce reproducibility and stability concerns due to potential undocumented model updates over time.

Future Work. Future research should explore multimodal extensions incorporating visual and audiovisual cues, and longitudinal, in-the-wild deployment studies of how linguistically grounded explanations influence belief formation over time.

Acknowledgments

This work was supported by CNPq's National Institute of Science and Technology in Responsible AI for Computational Linguistics, Information Treatment, and Dissemination (INCT-TILDIAR), grant no. 408490/2024-1, and by the Kunumi Institute through the Agents4Good project.

Notes

1Retrieved April 10, 2026 from https://www.weforum.org/publications/global-risks-report-2026/

2Retrieved April 10, 2026 from https://reutersinstitute.politics.ox.ac.uk/digital-news-report/2025/dnr-executive-summary

3Retrieved April 10, 2026 from https://www1.folha.uol.com.br/cotidiano/2026/04/brasileiros-desconfiam-mais-de-noticias-de-veiculos-jornalisticos-que-de-conteudo-de-amigos-em-redes-sociais-e-mensagens.shtml

4Retrieved April 10, 2026 from https://www.langchain.com/langgraph

5Retrieved April 10, 2026 from https://www.agencialupa.org/acontecendo-na-lupa/2015/10/15/entenda-nossas-etiquetas/

6Retrieved April 10, 2026 from https://www.tavily.com/

7Retrieved July 19, 2026 from https://developers.openai.com/api/docs/models/gpt-5.2

4Code repository: https://github.com/pedrohos/CheckAIPaper.

4Retrieved April 10, 2026 from https://deepmind.google/models/gemma/gemma-4/

Source

ACM full-text HTML

ACM Digital Library

Imported from ACM’s structured HTML source. ACM Reference Format: Pedro Henrique de Oliveira Silva, Lucas Raniére Juvino Santos, Leandro Balby Marinho, and Claudio Elizio Calazans Campelo. 2026. Beyond Verdicts: Explainable Fact-Checking via a Linguistically-Grounded Multi-Agent Framework. In 37th ACM Conference on Hypertext (HT '26), September 14--18, 2026, London, United Kingdom. ACM, New York, NY, USA 7 Pages. https://doi.org/10.1145/3800935.3830873

References

[1] Armen E. Allahverdyan and Aram Galstyan. 2014. Opinion Dynamics with Confirmation Bias. PLoS ONE 9, 7 (jul 2014), e99557. https://doi.org/10.1371/journal.pone.0099557

[2] John Bollenbacher, Filippo Menczer, and John Bryden. 2026. Effects of Antivaccine Tweets on COVID-19 Vaccinations, Cases, and Deaths. EPJ Data Science 15, 1 (2026), 12. https://doi.org/10.1140/epjds/s13688-025-00606-1

[3] Yupeng Cao, Chengyang He, Yangyang Yu, Ping Wang, and K. P. Subbalakshmi. 2026. MERMAID: Memory-Enhanced Retrieval and Reasoning with Multi-Agent Iterative Knowledge Grounding for Veracity Assessment. arxiv:2601.22361 [cs.CL] https://arxiv.org/abs/2601.22361

[4] I Chern, Steffi Chern, Shiqi Chen, Weizhe Yuan, Kehua Feng, Chunting Zhou, Junxian He, Graham Neubig, and Pengfei Liu. 2023. FacTool: Factuality Detection in Generative AI–A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios. arxiv:2307.13528 [cs.CL] https://arxiv.org/abs/2307.13528

[5] Ramit Debnath, Danny Ebanks, Kamiar Mohaddes, Thomas Roulet, and R. Michael Alvarez. 2023. Do Fossil Fuel Firms Reframe Online Climate and Sustainability Communication? A Data-Driven Analysis. npj Climate Action 2, 1 (2023), 47. https://doi.org/10.1038/s44168-023-00086-x

[6] Bella M. DePaulo, James J. Lindsay, Brian E. Malone, Laura Muhlenbruck, Kelly Charlton, and Harris Cooper. 2003. Cues to Deception. Psychological Bulletin 129, 1 (2003), 74–118. https://doi.org/10.1037/0033-2909.129.1.74

[7] Matthew R DeVerna, Kai-Cheng Yang, Harry Yaojun Yan, and Filippo Menczer. 2025. Large Language Models Require Curated Context for Reliable Political Fact-Checking–Even with Reasoning and Web Search. arxiv:2511.18749 [cs.CL] https://arxiv.org/abs/2511.18749

[8] Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Saizhuo Wang, Kun Zhang, Yuanzhuo Wang, Wen Gao, Lionel Ni, and Jian Guo. 2025. A Survey on LLM-as-a-Judge. arxiv:2411.15594 [cs.CL] https://arxiv.org/abs/2411.15594

[9] Beizhe Hu, Qiang Sheng, Juan Cao, Yuhui Shi, Yang Li, Danding Wang, and Peng Qi. 2024. Bad Actor, Good Advisor: Exploring the Role of Large Language Models in Fake News Detection. Proceedings of the AAAI Conference on Artificial Intelligence 38, 20 (Mar. 2024), 22105–22113. https://doi.org/10.1609/aaai.v38i20.30214

[10] Seyeon Jeong, Yeonjun Choi, JongWook Kim, and Beakcheol Jang. 2026. Tool-MAD: A Multi-Agent Debate Framework for Fact Verification with Diverse Tool Augmentation and Adaptive Retrieval. arxiv:2601.04742 [cs.CL] https://arxiv.org/abs/2601.04742

[11] Lucas Raniére Juvino Santos, Leandro Balby Marinho, Claudio Elizio Calazans Campelo, Filippo Menczer, and Alessandro Flammini. 2025. Can Large Language Models Effectively Mitigate Polarization in Social Media Text?. In Proceedings of the 17th ACM Web Science Conference 2025(Websci ’25). Association for Computing Machinery, New York, NY, USA, 348–357. https://doi.org/10.1145/3717867.3717904

[12] Hamid Karimi, Proteek Roy, Sari Saba-Sadiya, and Jiliang Tang. 2018. Multi-Source Multi-Class Fake News Detection. In Proceedings of the 27th International Conference on Computational Linguistics, Emily M. Bender, Leon Derczynski, and Pierre Isabelle (Eds.). Association for Computational Linguistics, Santa Fe, New Mexico, USA, 1546–1557. https://aclanthology.org/C18-1131/

[13] Sarah Ita Levitan, Angel Maredia, and Julia Hirschberg. 2018. Linguistic Cues to Deception and Perceived Deception in Interview Dialogues. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), Marilyn Walker, Heng Ji, and Amanda Stent (Eds.). Association for Computational Linguistics, New Orleans, Louisiana, 1941–1950. https://doi.org/10.18653/v1/N18-1176

[14] Stephan Lewandowsky, Ullrich K. H. Ecker, Colleen M. Seifert, Norbert Schwarz, and John Cook. 2012. Misinformation and Its Correction: Continued Influence and Successful Debiasing. Psychological Science in the Public Interest 13, 3 (2012), 106–131. https://doi.org/10.1177/1529100612451018

[15] Xinyi Li, Yongfeng Zhang, and Edward C. Malthouse. 2024. Large Language Model Agent for Fake News Detection. arxiv:2405.01593 [cs.CL] https://arxiv.org/abs/2405.01593

[16] Jiatong Ma, Linmei Hu, Rang Li, and Wenbo Fu. 2025. LoCal: Logical and Causal Fact-Checking with LLM-Based Multi-Agents. In Proceedings of the ACM on Web Conference 2025 (Sydney NSW, Australia) (WWW ’25). Association for Computing Machinery, New York, NY, USA, 1614–1625. https://doi.org/10.1145/3696410.3714748

[17] Xiaoxiao Ma, Yuchen Zhang, Kaize Ding, Jian Yang, Jia Wu, and Hao Fan. 2024. On Fake News Detection with LLM Enhanced Semantics Mining. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 508–521. https://doi.org/10.18653/v1/2024.emnlp-main.31

[18] Marlene Mauk and Max Grömping. 2024. Online disinformation predicts inaccurate beliefs about election fairness among both winners and losers. Comparative Political Studies 57, 6 (2024), 965–998. https://doi.org/10.1177/00104140231193008

[19] Shalini Munusamy, Kalaivanan Syasyila, Azahah Abu Hassan Shaari, Muhammad Adnan Pitchan, Mohammad Rahim Kamaluddin, and Ratna Jatnika. 2024. Psychological Factors Contributing to the Creation and Dissemination of Fake News Among Social Media Users: A Systematic Review. BMC Psychology 12, 1 (2024), 673. https://doi.org/10.1186/s40359-024-02129-2

[20] Thomas Renault, Mohsen Mosleh, and David G Rand. 2026. @Grok Is This True? LLM-Powered Fact-Checking on Social Media. https://doi.org/10.31234/osf.io/85quwv2

[21] Jon Roozenbeek, Claudia R. Schneider, Sarah Dryhurst, John Kerr, Alexandra L. J. Freeman, Gabriel Recchia, Anne Marthe van der Bles, and Sander van der Linden. 2020. Susceptibility to Misinformation about COVID-19 Around the World. Royal Society Open Science 7, 10 (2020), 201199. https://doi.org/10.1098/rsos.201199

[22] Dhruv Sahnan, David Corney, Irene Larraz, Giovanni Zagni, Ruben Miguez, Zhuohan Xie, Iryna Gurevych, Elizabeth Churchill, Tanmoy Chakraborty, and Preslav Nakov. 2026. Can LLMs Automate Fact-Checking Article Writing?arxiv:2503.17684 [cs.CL] https://arxiv.org/abs/2503.17684

[23] Kai Shu, Amy Sliva, Suhang Wang, Jiliang Tang, and Huan Liu. 2017. Fake News Detection on Social Media: A Data Mining Perspective. SIGKDD Explor. Newsl. 19, 1 (sep 2017), 22–36. https://doi.org/10.1145/3137597.3137600

[24] Saverio Storani, Max Falkenberg, Walter Quattrociocchi, and Matteo Cinelli. 2025. Relative Engagement with Sources of Climate Misinformation Is Growing Across Social Media Platforms. Scientific Reports 15, 1 (2025), 18629. https://doi.org/10.1038/s41598-025-03082-9

[25] Liyan Tang, Philippe Laban, and Greg Durrett. 2024. MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 8818–8847. https://doi.org/10.18653/v1/2024.emnlp-main.499

[26] Tam Trinh, Manh Nguyen, and Truong-Son Hy. 2025. Towards Robust Fact-Checking: A Multi-Agent System with Advanced Evidence Retrieval. arxiv:2506.17878 [cs.AI] https://arxiv.org/abs/2506.17878

[27] Jean Wagemans. 2016. Constructing a Periodic Table of Arguments. In Argumentation, Objectivity, and Bias: Proceedings of the 11th International Conference of the Ontario Society for the Study of Argumentation (OSSA) (2016-03-21), Patrick Bondy and Laura Benacquista (Eds.). OSSA, Windsor, ON, Canada, 1–12. Available at SSRN. https://doi.org/10.2139/ssrn.2769833

[28] Jean H.M. Wagemans. 2023. How to identify an argument type? On the hermeneutics of persuasive discourse. Journal of Pragmatics 203 (2023), 117–129. https://doi.org/10.1016/j.pragma.2022.11.015

[29] Bo Wang, Jing Ma, Hongzhan Lin, Zhiwei Yang, Ruichao Yang, Yuan Tian, and Yi Chang. 2025. Explainable Fake News Detection With Large Language Model via Defense Among Competing Wisdom. arxiv:2405.03371 [cs.CL] https://arxiv.org/abs/2405.03371

[30] Haoran Wang and Kai Shu. 2023. Explainable Claim Verification via Knowledge-Grounded Reasoning with Large Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2023, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguistics, Singapore, 6288–6304. https://doi.org/10.18653/v1/2023.findings-emnlp.416

[31] Yuchen Zhang, Xiaoxiao Ma, Jia Wu, Jian Yang, and Hao Fan. 2024. Heterogeneous Subgraph Transformer for Fake News Detection. arxiv:2404.13192 [cs.CL] https://arxiv.org/abs/2404.13192

[32] Lina Zhou, Judee K. Burgoon, Jay F. Nunamaker, and Doug Twitchell. 2004. Automating Linguistics-Based Cues for Detecting Deception in Text-Based Asynchronous Computer-Mediated Communications. Group Decision and Negotiation 13, 1 (2004), 81–106. https://doi.org/10.1023/B:GRUP.0000011944.62889.6f