Article
Multilevel network meta-regression: A step forward in HTA evidence synthesis?
Multilevel network meta-regression is increasingly recognized by HTA agencies as a method to overcome concerns that arise when comparing trials with different study designs for the same intervention and indication. The method is particularly appealing when trial data are limited, such as for treatments for rare diseases
Figure 1. Treatment network
Problem
Figure 1. Treatment network
Many treatments are tested only against a placebo or standard of care in their registrational trials and therefore lack head-to-head comparisons of efficacy against competitors. Indirect treatment comparisons (ITCs) are common in health technology assessments (HTAs) where head-to-head trials (particularly across the full range, or network, of treatments approved for a given condition) are not available.
The trial data available for ITCs can be at the individual patient-level data (IPD), or as aggregated data (AD), which are typically at the treatment-arm level. Network meta-analysis (NMA) is commonly used as an ITC technique to simultaneously compare multiple trials that share a common comparator, producing relative treatment effects, treatment rankings, and estimates of uncertainty. Figure 1 shows how both direct and indirect evidence can be looped into a network for an intervention (where the trial for treatment A was directly against treatment B, but trials for B and C are indirectly related as they were both against a placebo).
The trial data available for ITCs can be at the individual patient-level data (IPD), or as aggregated data (AD), which are typically at the treatment-arm level. Network meta-analysis (NMA) is commonly used as an ITC technique to simultaneously compare multiple trials that share a common comparator, producing relative treatment effects, treatment rankings, and estimates of uncertainty. Figure 1 shows how both direct and indirect evidence can be looped into a network for an intervention (where the trial for treatment A was directly against treatment B, but trials for B and C are indirectly related as they were both against a placebo).
Figure 1. Treatment network
However, NMA uses only AD and requires the assumption that the distribution of covariates from all individual trial populations in the network is homogenous (i.e. similarities that will allow for a fair comparison and not lead to highly uncertain or biased conclusions); bias will arise if there is heterogeneity between trials’ patient populations, study designs, treatment pathways, and distribution of treatment effect modifiers, such as age, prior treatment(s), or severity of disease. Network meta-regression (NMR) builds on NMA and allows for between-study differences in treatment effect modifiers (variables that change the strength or direction of the relationship between treatment and outcome) to be adjusted for a regression modeling framework.
By using NMR, analysts can explore whether treatment effects vary across the network. However, NMR still requires AD, and such aggregation methods have been criticized as being prone to misrepresentations where relationships observed in the aggregated study do not reflect those at the individual level (aggregation/ecological bias). Augmenting NMA with population adjustment techniques — for example, matching-adjusted indirect comparison (MAIC) and simulated treatment comparison (STC) — can help to overcome aggregation bias and covariate imbalances. MAIC reweights IPD to match the distribution of AD for known treatment effect-modifying covariates, while STC fits a regression model to the IPD to estimate the outcome in the AD trial. Both of these methods are limited to pairwise comparisons that cannot make relative evaluations across a network of interventions and have limitations in inferring to target populations when these differ meaningfully from those of the index trial (NICE TSD 18). Multilevel network meta-regression (ML-NMR) is an emerging ITC approach that extends the NMA framework while maintaining population adjustment benefits by allowing IPD (where available) and AD levels of data to be used simultaneously. This article examines the merits of ML-NMR and the situations in which the technique could be recommended for ITCs used in HTA appraisals.
Benefits
ML-NMR supports adjustment for effect modifiers across studies, even those where IPD are not available, by modeling IPD treatment effects and integrating over the covariate distribution to create a probabilistic network model. By incorporating all available data across the network and leveraging IPD, ML-NMR has the potential for greater confidence in decision-making by reducing bias from treatment effect modifiers and allowing a broad view across multiple indicated treatments simultaneously. ML-NMR allows analysts to generate treatment effect estimates that are applicable to a clearly defined target population, such as National Health Service (NHS) patients, rather than being limited to a population defined by a single comparator trial. As a result, ML-NMR is recognized by agencies in HTA processes and is increasingly supported by the United Kingdom’s National Institute for Health and Care Excellence (NICE) and the EU HTA Coordination Group, particularly as a means to reduce between-trial variation, improving evidence transparency in economic model parameters and alignment with scoped PICOs.
For diseases where trial data are limited (e.g. rare cancers or pediatric conditions), ML-NMR allows for more flexible and inclusive use of all available data, supporting fairer access decisions and strengthening evidence.
Challenges
Despite the advantages of ML-NMR, the method is not without limitations. Conducting ML-NMR requires detailed covariate information from all studies in the network in the form of IPD or published summary statistics from AD trials. Gathering and standardizing these data can be time-consuming and resource-intensive as these should ideally originate from a systematic literature review; in addition, producing ML-NMR models with large amounts of IPD can be computationally demanding. Advanced statistical expertise (e.g. Bayesian frameworks and Markov Chain Monte Carlo [MCMC] simulation) and software are required to perform this analysis. Furthermore, ML-NMR assumes that the effect of a covariate on treatment response is consistent across studies, and results may be biased where this assumption does not hold.
Although ML-NMR is in the early stages of development, the underlying code is freely available, and it has been built with extensibility in mind. The NICE Decision Support Unit (DSU) is keen to adapt to new challenges, such as informal review of the changes to the code framework to manage negative binomial functions needed to model overdispersion in a hemophilia attack rate endpoint, as discussed in the committee papers for the ongoing NICE HTA ID6394. Standardized reporting guidelines also lack consensus, and unlike MAIC and STC, which are supported by growing literature and technical support documents (TSDs), ML-NMR currently lacks best-practice guidance and reporting support. The novelty of ML-NMR is also challenging, as stakeholders, particularly those unfamiliar with Bayesian methods or advanced modeling, may require significant support to interpret ML-NMR outputs correctly or even be open to committing resources to investigate using the technique.
Although ML-NMR is in the early stages of development, the underlying code is freely available, and it has been built with extensibility in mind. The NICE Decision Support Unit (DSU) is keen to adapt to new challenges, such as informal review of the changes to the code framework to manage negative binomial functions needed to model overdispersion in a hemophilia attack rate endpoint, as discussed in the committee papers for the ongoing NICE HTA ID6394. Standardized reporting guidelines also lack consensus, and unlike MAIC and STC, which are supported by growing literature and technical support documents (TSDs), ML-NMR currently lacks best-practice guidance and reporting support. The novelty of ML-NMR is also challenging, as stakeholders, particularly those unfamiliar with Bayesian methods or advanced modeling, may require significant support to interpret ML-NMR outputs correctly or even be open to committing resources to investigate using the technique.
Implementation
As noted, ML-NMR is still an emergent technique: the seminal paper was published in 2020, and only three HTAs using ML-NMR have been submitted to date, the first of which led to a publication of a validated model in 2023 by academics in the NICE DSU who developed the technique. Where ML-NMR has been used, statisticians have aimed to estimate treatment effects that are applicable to a specified target population, rather than limited to the population characteristics of a single comparator trial. In an assessment of treatment for acute myeloid leukaemia (TA1013), the external advisory group (EAG) engaged by the NICE committee recommended that the company produce an ML-NMR in place of the company’s original MAIC-based ITC. The EAG’s rationale was that ML-NMR provided estimates more relevant to the NHS target since it adjusted treatment effect estimates to a predefined population, not just to the population of a comparator trial.
The ML-NMR produced by the company in response generated treatment effect estimates in the target population, which the committee accepted for decision-making. Additionally, in the validation publication based on TA1013, the EAG provided detailed commentary on the methodology, assumptions (e.g. shared effect modification assumption), and correct integration of ML-NMR outputs into survival-based cost-effectiveness models; this detailed validation acts as an initial standard for ML-NMR use in the absence of published guidance.
Data requirements and analytic complexity are a challenge in using ML-NMR. Even where data can be gathered, feasibility studies may show that the technique is unable to overcome limitations of the evidence base. In a second NICE HTA, evaluating chemotherapy treatment for non-small cell lung cancer, the manufacturer was also advised to conduct an ML-NMR for relevant population alignment. After conducting a feasibility study, the manufacturer concluded that, because of the nature of the network of trials, the model would have to rely on the shared-effect modifier assumption, which was not supported by the network (TA1030). These findings were agreed upon by the EAG and accepted by the committee; in this instance, the need for a full ML-NMR was avoided.
In the third HTA (NICE reference ID6394, still in evaluation), ML-NMR was deemed the most comprehensive and least biased approach. With too many comparators to sensibly conduct and synthesize pairwise MAICs, and interpretation difficulties due to multiple target populations, ML-NMR was chosen to reduce bias from heterogeneity between studies in the NMA and provided a sensitivity analysis to the network analysis.
Conclusion
The TA1013 case sets a precedent of NICE's willingness to accept ML-NMR and suggests that the method may feature more regularly in future submissions, especially in oncology, rare diseases, and high-uncertainty appraisals.
However, broader uptake will depend on formal inclusion of ML-NMR in NICE’s evolving methodological guidance, such as specification in the TSDs, and further clear articulation of the evidentiary standards expected when such methods are used. ML-NMR is particularly valuable in comparisons where:
- The target population for reimbursement differs materially from the trial populations.
- IPD is available for one or more trials but not for all trials containing interventions or comparators of interest. Trials report heterogeneous baseline characteristics and covariates known to modify treatment effects.
- Conventional matching methods like MAIC introduce instability due to overfitting or lack of overlap, or would require too many pairwise comparisons.
Manufacturing companies preparing for HTA submissions should consider whether ML-NMR can strengthen their value proposition, particularly when ITCs are unavoidable.
This article summarises Cencora’s understanding of the topic based on publicly available information at the time of writing (see listed sources) and the authors’ expertise in this area. Any recommendations provided in the article may not be applicable to all situations and do not constitute legal advice; readers should not rely on the article in making decisions related to the topics discussed.
Connect with our team
Our team of leading value experts is dedicated to transforming evidence, policy insights, and market intelligence into effective global market access strategies. Let us help you navigate today’s complex healthcare landscape with confidence. Reach out to discover how we can support your goals.
Sources
- European Union Health Technology Assessment Coordination Group. Methodological guidance for quantitative evidence synthesis. https://health.ec.europa.eu/document/download/4ec8288e-6d15-49c5-a490-d8ad7748578f_en?filename=hta_methodological-guideline_direct-indirect-comparisons_en.pdf
- Nevitt SJ, Phillippo DM, Hodgson R, et al. Application of multilevel network meta-regression in the NICE technology appraisal of quizartinib for induction, consolidation and maintenance treatment of newly diagnosed FLT3-ITD-positive acute myeloid leukaemia: an external assessment group perspective. PharmacoEconomics. 2025;43:243-247. https://doi.org/10.1007/s40273-024-01460-1
- NICE Decision Support Unit. Evidence synthesis technical support documents. https://sheffield.ac.uk/nice-dsu/tsds/evidence-synthesis
- Phillippo DM, Dias S, Ades AE, et al. Multilevel network meta-regression for population-adjusted treatment comparisons. J R Stat Soc Ser A Stat Soc. 2020;183(3):1189-1210. doi:10.1111/rssa.12579
Cencora.com is providing automated translations to assist in reading the website in languages other than English. For these translations, reasonable efforts have been made to provide an accurate translation, however, no automated translation is perfect nor is it intended to replace human translators. These translations are provided as a service to users of Cencora.com and are provided "as is." No warranty of any kind, either expressed or implied, is made as to the accuracy, reliability, or correctness of any of these translations made from English into any other language. Some content (such as images, videos, Flash, etc.) may not be accurately translated due to the limitations of the translation software.
Any discrepancies or differences created in translating this content from English into another language are not binding and have no legal effect for compliance, enforcement, or any other purpose. If any errors are identified, please contact us. If any questions arise related to the accuracy of the information contained in these translations, please refer to the English version of the page.