Quality Assurance Tool System for the Entire Process of Real-World Studies: Classification and Practical Guidelines

doi:10.19546/j.issn.1674-3830.2026.5.007

China Health Insurance ›› 2026, Vol. 0 ›› Issue (5) : 54-64. DOI: 10.19546/j.issn.1674-3830.2026.5.007

Topics in Focus

Quality Assurance Tool System for the Entire Process of Real-World Studies: Classification and Practical Guidelines

Author information +

History +

Abstract

Objective: In light of the current situation where the methodological standardization of real-world studies (RWS) needs improvement and the actual research implementation pathways in China are still in the exploratory stage, the paper constructs a standardized tool application framework centered on the entire research process. Methods: The RWS process was deconstructed into five stages—study design, data preparation, analysis implementation, report disclosure, and evidence appraisal, and the core quality risks at each stage were identified. Mainstream international standardized tools were systematically searched and screened, then grouped by function into four categories: risk-of-bias assessment, data suitability assessment, design and reporting standards, and evidence quality grading. A mapping between the tools and the risk nodes was established. Results: A full-lifecycle quality risk map of RWS was constructed, clarifying the risk manifestations at each stage and the appropriate timing for tool application. A three-dimensional tool selection logic based on “research objective-data source-evidence use” was derived, and specific tool combination schemes were provided for three typical scenarios in China. Conclusion: Integrating full-lifecycle risk awareness, stage-specific tool allocation, and scenario-based combination strategies helps improve the methodological rigor and evidence credibility of RWS in China, providing solid support for health technology assessment and healthcare security decision-making.

Key words

real-world study / research tools / quality assurance / bias control / reporting standards

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations

Quality Assurance Tool System for the Entire Process of Real-World Studies: Classification and Practical Guidelines[J]. China Health Insurance. 2026, 0(5): 54-64 https://doi.org/10.19546/j.issn.1674-3830.2026.5.007

References

[1] 国家药品监督管理局.真实世界证据支持药物研发与审评的指导原则(试行)[EB/OL].(2020-04-02)[2026-05-15].https://amr.hainan.gov.cn/himpa/xxgk/zcfg/ypl/202004/t20200402_2769970.html.
[2] 吕兰婷,夏雨婕.真实世界证据如何赋能医保决策?——英国NICE框架的方法学创新与启示[J].中国医疗保险,2026(01):122-127.
[3] FRANKLIN JM, SCHNEEWEISS S.When and how can real world data analyses substitute for randomized controlled trials?[J]. Clinical pharmacology & therapeutics, 2017,102(6):924-933.
[4] 孙鑫,谭婧,王雯,等.建立真实世界数据与研究技术规范,促进中国真实世界证据的生产与使用[J].中国循证医学杂志,2019,19(07):755-762.
[5] 姚振阁,周佳薇,陆梦依,等.基于德尔菲法构建真实世界证据评价体系[J].中国循证医学杂志,2024,24(10):1156-1161.
[6] 张强,卢存存,赵文霞,等.系统评价与Meta分析中真实世界研究质量评价工具(QATSM-RWS)的解读与应用[J].中国循证医学杂志,2024,24(12):1451-1457.
[7] WILLKE R, COTTU P, BRIGGS A, et al.Finding the right tool for the specific task: navigating RWE tools and checklists[J]. Future oncology, 2025,21(23):3075-3089.
[8] CAPKUN G, CORRY S, DOWLING O, et al.Can we use existing guidance to support the development of robust real-world evidence for health technology assessment/payer decision-making?[J]. International journal of technology assessment in health care, 2022,38(1):e79.
[9] FLEURENCE RL, KENT S, ADAMSON B, et al.Assessing real-world data from electronic health records for health technology assessment: the suitability checklist: a good practices report of an ISPOR task force[J]. Value health, 2024, 27(6):692-701.
[10] MORGAN RL, THAYER KA, SANTESSO N, et al.Evaluation of the risk of bias in non-randomized studies of interventions (ROBINS-I) and the ‘target experiment’ concept in studies of exposures: rationale and preliminary instrument development[J]. Environment international, 2018, 120:382-387.
[11] WANG SV, PINHEIRO S, HUA W, et al.STaRT-RWE: structured template for planning and reporting on the implementation of real world evidence studies[J]. BMJ, 2021,372:m4856.
[12] GUYATT G, AGORITSAS T, BRIGNARDELLO-PETERSEN R, et al.Core GRADE 1: overview of the Core GRADE approach[J]. BMJ, 2025,389:e081903.
[13] Katsiaryna Bykov, Ashley Jaksa, Jennifer L.Lund, et al, APPRAISE: A Tool for Appraising Potential for Bias in Real-World Evidence Studies on Medication Effectiveness or Safety[J]. Value in Health, 2025, 28(12):1849-1856.
[14] STERNE JA, HERNAN MA, REEVES BC, et al.ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions[J]. BMJ, 2016,355:i4919.
[15] CONOVER MM, RYAN PB, CHEN Y, et al.Objective study validity diagnostics: a framework requiring pre-specified, empirical verification to increase trust in the reliability of real-world evidence[J]. Journal of the American Medical Informatics Association, 2025,32(3):518-525.
[16] National Institute for Health and Care Excellence. NICE real-world evidence framework [EB/OL]. (2022-06-23)[2026-05-08]. https://www.nice.org.uk/corporate/ecd9.
[17] CANDORE G, MARTIN C, MILLS MJ, et al.FRAME: framework for real-world evidence assessment to mitigate evidence uncertainties for efficacy/effectiveness - an evaluation of regulatory and health technology assessment decision making[J]. Clinical pharmacology & therapeutics, 2025,118(3):649-661.
[18] U.S. Food & Drug Administration. Use of real-world evidence to support regulatory decision-making for medical devices[EB/OL]. (2022-06-23)[2026-05-08]. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/use-real-world-evidence-support-regulatory-decision-making-medical-devices.
[19] WANG SV, POTTEGARD A, CROWN W, et al.Harmonized protocol template to enhance reproducibility of hypothesis evaluating real-world evidence studies on treatment effects: a good practices report of a joint ISPE/ISPOR task force[J]. Value health, 2022,25(10):1663-1672.
[20] VON ELM E, ALTMAN DG, EGGER M, et al.The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies[J]. Lancet, 2007,370(9596):1453-1457.
[21] BENCHIMOL EI, SMEETH L, GUTTMANN A, et al.The reporting of studies conducted using observational routinely-collected health data (RECORD) statement[J]. PLoS med, 2015,12(10):e1001885.
[22] LANGAN SM, SCHMIDT SA, WING K, et al.The reporting of studies conducted using observational routinely collected health data statement for pharmacoepidemiology (RECORD-PE)[J]. BMJ, 2018,363:k3532.
[23] HERNAN MA, ROBINS JM.Using big data to emulate a target trial when a randomized trial is not available[J]. American journal of epidemiology, 2016,183(8):758-764.
[24] CASHIN AG, HANSFORD HJ, HERNAN MA, et al.Transparent Reporting of observational studies emulating a target trial-The TARGET Statement[J]. JAMA, 2025,334(12):1084-1093.
[25] LIU J, YAO M, WANG M, et al.Design, conduct, and analysis of externally controlled trials[J]. JAMA network open, 2025,8(9):e2530277.
[26] JAHANSHAHI M, GREGG K, DAVIS G, et al.The use of external controls in FDA regulatory decision making[J]. Therapeutic innovation & regulatory science, 2021,55(5):1019-1035.
[27] SCHUEMIE MJ, RYAN PB, PRATT N, et al.Large-scale evidence generation and evaluation across a network of databases (LEGEND): assessing validity using hypertension as a case study[J]. Journal of the American Medical Informatics Association,2020,27(8):1268-1277.
[28] SRIVASTAVA T, SHARMA R, GAUTAM R, et al.MSR195 The AI-in-RWE Transparency (AIRT) checklist: essential and desirable standards for AI-enhanced real-world evidence[J]. Value in health, 2025, 28(12): S534.