Int J Med Inform. 2026 May 29;218:106491. doi: 10.1016/j.ijmedinf.2026.106491. Online ahead of print.

ABSTRACT

PURPOSE: Reproducibility in rehabilitation evidence synthesis is influenced not only by search strategy and adjudication architecture but also by the structural clarity of operational taxonomy. This study evaluated whether shared operational definitions support classification stability across AI-assisted adjudication architectures.

METHODS: A previously established deduplicated rehabilitation corpus was analyzed using a fixed multi-adjudicator architecture under standardized operational constraints. Inter-architecture concordance (agreement, Cohen’s κ, and Gwet’s AC1) was assessed. Corpus expansion was modeled through staged database inclusion, and stability bounds were estimated under best- and worst-case perturbation scenarios without re-adjudication of newly identified records. Risk of bias assessment was not performed, as the objective was classification concordance rather than therapeutic effect estimation.

RESULTS: High inter-architecture concordance was observed under fixed operational definitions. Sensitivity-envelope modeling identified both stability-preserving and stability-failing boundary conditions. Under worst-case forced-discordance assumptions, κ declined substantially as modeled corpus expansion increased, indicating that robustness was conditional rather than unconditional.

CONCLUSIONS: Explicit operational taxonomy may constrain classification variability across AI-assisted adjudication architectures when citation-level metadata are incomplete. AI systems cannot recover procedural specificity that is not encoded within bibliographic records. Because taxonomy was held constant rather than experimentally varied, the relative contribution of taxonomy versus architecture remains an empirical question for future work. Although evaluated within a dry needling corpus, the underlying metadata-signal problem may extend to other intervention domains, with implications for AI-assisted evidence workflows in healthcare decision-making.

PMID:42235437 | DOI:10.1016/j.ijmedinf.2026.106491