Covid Analysis, January 16, 2022, DRAFT
https://c19early.com/mfmeta.html
•Statistically significant improvements are seen for mortality and hospitalization. 16 studies from 16 independent teams in 6 different countries show statistically significant
improvements in isolation (13 for the most serious outcome).
•Meta analysis using the most serious outcome reported shows
17% [11‑22%] improvement. Results are worse for Randomized Controlled Trials, similar after exclusions, and similar for peer-reviewed studies.
•Results are robust — in exclusion sensitivity analysis 17 of 25
studies must be excluded to avoid finding statistically significant efficacy
in pooled analysis.
•Most studies analyze existing use with diabetic patients. Many results
are subject to confounding by indication — metformin is typically
used early in the progression of type 2 diabetes.
•While many treatments have some level
of efficacy, they do not replace vaccines and other measures to avoid
infection.
None of the metformin studies show zero events in the treatment arm.
•Multiple treatments are typically used
in combination, and other treatments
are significantly more effective.
•Elimination of COVID-19 is a race
against viral evolution. No treatment, vaccine, or intervention is 100%
available and effective for all variants. All practical, effective, and safe
means should be used, including treatments, as supported by Pfizer [Pfizer].
Denying the efficacy of treatments increases mortality, morbidity, collateral
damage, and endemic risk.
•All data to reproduce this paper and
sources are in the appendix.
| Studies | Early treatment | Prophylaxis | Patients | Authors | ||
| All studies | 25 | 6% [-61‑45%] | 17% [11‑23%] | 104,505 | 348 | |
| With exclusions | 23 | 6% [-61‑45%] | 18% [12‑23%] | 88,912 | 300 | |
| Peer-reviewed | 24 | 6% [-61‑45%] | 16% [10‑22%] | 82,381 | 337 | |
| Randomized Controlled TrialsRCTs | 1 | 6% [-61‑45%] | 418 | 23 | ||
| Percentage improvement with metformin treatment | ||||||
Figure 1. A. Random effects
meta-analysis. This plot shows pooled effects, discussion can be found in the heterogeneity section,
and results for specific outcomes can be found in the individual outcome analyses.
Effect extraction is pre-specified, using the most serious outcome reported.
For details of effect extraction see the appendix.
B. Scatter plot showing the
distribution of effects reported in studies. C. History of all reported
effects (chronological within treatment stages).
Introduction
We analyze all significant studies concerning the use of
metformin for COVID-19. Search methods, inclusion criteria, effect
extraction criteria (more serious outcomes have priority), all individual
study data, PRISMA answers, and statistical methods are detailed in
Appendix 1. We present random effects meta-analysis results for all
studies, for studies within each treatment stage, for individual outcomes, for
peer-reviewed studies, for Randomized Controlled Trials (RCTs), and after
exclusions.
Figure 2 shows stages of possible treatment for
COVID-19. Prophylaxis refers to regularly taking medication before
becoming sick, in order to prevent or minimize infection. Early
Treatment refers to treatment immediately or soon after symptoms appear,
while Late Treatment refers to more delayed treatment.
Figure 2. Treatment stages.
Results
Figure 3, 4, 5, 6, 7, 8, 9, 10, and 11
show forest plots for a random effects meta-analysis of
all studies with pooled effects, mortality results, ventilation, ICU admission, hospitalization, progression, cases, viral clearance, and peer reviewed studies.
Table 1 summarizes the results by treatment stage.
| Treatment time | Number of studies reporting positive effects | Total number of studies | Percentage of studies reporting positive effects | Random effects meta-analysis results |
| Early treatment | 1 | 1 | 100% |
6% improvement RR 0.94 [0.55‑1.61] p = 0.84 |
| Prophylaxis | 19 | 24 | 79.2% |
17% improvement RR 0.83 [0.77‑0.89] p < 0.0001 |
| All studies | 20 | 25 | 80.0% |
17% improvement RR 0.83 [0.78‑0.89] p < 0.0001 |
Table 1. Results by treatment stage.
Figure 3. Random effects meta-analysis for all studies with pooled effects.
This plot shows pooled effects, discussion can be found in the heterogeneity section,
and results for specific outcomes can be found in the individual outcome analyses.
Effect extraction is pre-specified, using the most serious outcome reported.
For details of effect extraction see the appendix.
Figure 4. Random effects meta-analysis for mortality results.
Figure 5. Random effects meta-analysis for ventilation.
Figure 6. Random effects meta-analysis for ICU admission.
Figure 7. Random effects meta-analysis for hospitalization.
Figure 8. Random effects meta-analysis for progression.
Figure 9. Random effects meta-analysis for cases.
Figure 10. Random effects meta-analysis for viral clearance.
Figure 11. Random effects meta-analysis for peer reviewed studies.
Effect extraction is pre-specified, using the most serious outcome reported,
see the appendix for details.
Exclusions
To avoid bias in the selection of studies, we analyze all
non-retracted studies. Here we show the results after excluding studies with
major issues likely to alter results, non-standard studies, and studies where
very minimal detail is currently available. Our bias evaluation is based on
analysis of each study and identifying when there is a significant chance that
limitations will substantially change the outcome of the study. We believe
this can be more valuable than checklist-based approaches such as Cochrane
GRADE, which may underemphasize serious issues not captured in the checklists,
overemphasize issues unlikely to alter outcomes in specific cases (for
example, lack of blinding for an objective mortality outcome, or certain
specifics of randomization with a very large effect size), or be easily
influenced by potential bias. However, they can also be very high
quality.
The studies excluded are as below.
Figure 12 shows a forest plot for random
effects meta-analysis of all studies after exclusions.
[Holt], significant unadjusted confounding possible.
[Ravindra], minimal details provided.
Figure 12. Random effects meta-analysis for all studies after exclusions.
This plot shows pooled effects, discussion can be found in the heterogeneity section,
and results for specific outcomes can be found in the individual outcome analyses.
Effect extraction is pre-specified, using the most serious outcome reported.
For details of effect extraction see the appendix.
Randomized Controlled Trials (RCTs)
Figure 13 shows a forest plot for random
effects meta-analysis of all Randomized Controlled Trials.
Table 2 summarizes the results.
Currently there is only one RCT.
RCTs help to make study groups more similar, however they are
subject to many biases, including age bias, treatment delay bias, severity of
illness bias, regulation bias, recruitment bias, trial design bias, followup
time bias, selective reporting bias, fraud bias, hidden agenda bias, vested
interest bias, publication bias, and publication delay bias [Jadad],
all of which have been observed with COVID-19 RCTs.
RCTs have a bias against finding an effect for interventions
that are widely available — patients that believe they need the
intervention are more likely to decline participation and take the
intervention. This is illustrated with the extreme example of an RCT showing
no significant differences for use of a parachute when jumping from a plane
[Yeh]. RCTs for metformin are more likely to enroll low-risk
participants that do not need treatment to recover, making the results less
applicable to clinical practice. This bias is likely to be greater for widely
known treatments.
Note that this bias does
not apply to the typical pharmaceutical trial of a new drug that is otherwise
unavailable.
Evidence shows that non-RCT trials can also provide reliable
results. [Concato] find that well-designed observational studies do
not systematically overestimate the magnitude of the effects of treatment
compared to RCTs. [Anglemyer] summarized reviews comparing RCTs to
observational studies and found little evidence for significant differences
in effect estimates.
[Lee] shows that only 14% of the guidelines
of the Infectious Diseases Society of America were based on RCTs. Evaluation
of studies relies on an understanding of the study and potential biases.
Limitations in an RCT can outweigh the benefits, for example excessive
dosages, excessive treatment delays, or Internet survey bias could have a
greater effect on results. Ethical issues may also prevent running RCTs for
known effective treatments. For more on issues with RCTs see [Deaton, Nichol].
Figure 13. Random effects meta-analysis for all Randomized Controlled Trials.
This plot shows pooled effects, discussion can be found in the heterogeneity section,
and results for specific outcomes can be found in the individual outcome analyses.
Effect extraction is pre-specified, using the most serious outcome reported.
For details of effect extraction see the appendix.
| Treatment time | Number of studies reporting positive effects | Total number of studies | Percentage of studies reporting positive effects | Random effects meta-analysis results |
| Randomized Controlled Trials | 1 | 1 | 100% |
6% improvement RR 0.94 [0.55‑1.61] p = 0.84 |
Table 2. Randomized Controlled Trial results.
Heterogeneity
Heterogeneity in COVID-19 studies arises from many factors including:
Treatment delay.
The time between infection
or the onset of symptoms and treatment may critically affect how well a
treatment works. For example an antiviral may be very effective when used
early but may not be effective in late stage disease, and may even be harmful.
Oseltamivir, for example, is generally only considered effective for influenza
when used within 0-36 or 0-48 hours [McLean, Treanor]. Other
medications might be beneficial for late stage complications, while early use
may not be effective or may even be harmful. Figure 14 shows
an example where efficacy declines as a function of treatment delay.
Figure 14. Effectiveness may depend critically
on treatment delay.
Patient demographics.
Details of the
patient population including age and comorbidities may critically affect how
well a treatment works. For example, many COVID-19 studies with relatively
young low-comorbidity patients show all patients recovering quickly with or
without treatment. In such cases, there is little room for an effective
treatment to improve results (as in [López-Medina]).Effect measured.
Efficacy may differ
significantly depending on the effect measured, for example a treatment may be
very effective at reducing mortality, but less effective at minimizing cases
or hospitalization. Or a treatment may have no effect on viral clearance while
still being effective at reducing mortality.Variants.
There are many different
variants of SARS-CoV-2 and efficacy may depend critically on the distribution
of variants encountered by the patients in a study. For example, the Gamma
variant shows significantly different characteristics
[Faria, Karita, Nonaka, Zavascki]. Different mechanisms of action may be
more or less effective depending on variants, for example the viral entry
process for the omicron variant has moved towards TMPRSS2-independent fusion,
suggesting that TMPRSS2 inhibitors may be less effective
[Peacock, Willett].Regimen.
Effectiveness may depend
strongly on the dosage and treatment regimen.Treatments.
The use of other
treatments may significantly affect outcomes, including anything from
supplements, other medications, or other kinds of treatment such as prone
positioning.The distribution of studies will alter the outcome of a meta
analysis. Consider a simplified example where everything is equal except for
the treatment delay, and effectiveness decreases to zero or below with
increasing delay. If there are many studies using very late treatment, the
outcome may be negative, even though the treatment may be very effective when
used earlier.
In general, by combining heterogeneous studies, as all meta
analyses do, we run the risk of obscuring an effect by including studies where
the treatment is less effective, not effective, or harmful.
When including studies where a treatment is less effective we
expect the estimated effect size to be lower than that for the optimal case.
We do not a priori expect that pooling all studies will create a
positive result for an effective treatment. Looking at all studies is valuable
for providing an overview of all research, important to avoid cherry-picking,
and informative when a positive result is found despite combining less-optimal
situations. However, the resulting estimate does not apply to specific cases
such as early treatment in high-risk populations.
Discussion
Publication bias.
Publishing is often biased
towards positive results, however evidence suggests that there may be a negative bias for
inexpensive treatments for COVID-19. Both negative and positive results are
very important for COVID-19, media in many countries prioritizes negative
results for inexpensive treatments (inverting the typical incentive for
scientists that value media recognition), and there are many reports of
difficulty publishing positive results
[Boulware, Meeus, Meneguesso].
For metformin, there is currently not
enough data to evaluate publication bias with high confidence.Conflicts of interest.
Pharmaceutical drug
trials often have conflicts of interest whereby sponsors or trial staff have a
financial interest in the outcome being positive. Metformin for COVID-19
lacks this because it is
off-patent, has multiple manufacturers, and is very low cost.
In contrast, most COVID-19 metformin trials have been run by
physicians on the front lines with the primary goal of finding the best
methods to save human lives and minimize the collateral damage caused by
COVID-19. While pharmaceutical companies are careful to run trials under
optimal conditions (for example, restricting patients to those most likely to
benefit, only including patients that can be treated soon after onset when
necessary, and ensuring accurate dosing), not all metformin trials
represent the optimal conditions for efficacy.Early/late vs. mild/moderate/severe.
Some analyses classify treatment based on early/late administration (as we do
here), while others distinguish between mild/moderate/severe cases. We note
that viral load does not indicate degree of symptoms — for example
patients may have a high viral load while being asymptomatic. With regard to
treatments that have antiviral properties, timing of treatment is
critical — late administration may be less helpful regardless of
severity.Conclusion
Statistically significant improvements are seen for mortality and hospitalization. 16 studies from 16 independent teams in 6 different countries show statistically significant
improvements in isolation (13 for the most serious outcome).
Meta analysis using the most serious outcome reported shows
17% [11‑22%] improvement. Results are worse for Randomized Controlled Trials, similar after exclusions, and similar for peer-reviewed studies.
Results are robust — in exclusion sensitivity analysis 17 of 25
studies must be excluded to avoid finding statistically significant efficacy
in pooled analysis.
Most studies analyze existing use with diabetic patients. Many results
are subject to confounding by indication — metformin is typically
used early in the progression of type 2 diabetes.
Study Notes
[Alamgir]
In Silico study followed by PSM analysis of the National COVID Cohort Collaborative data in the USA, showing 27% lower mortality with metformin use.
ABSTRACT Background Drug repositioning is a key component of COVID-19 pandemic response, through identification of existing drugs that can effectively disrupt COVID-19 disease processes, contributing valuable insights into disease pathways. Traditional non in silico drug repositioning approaches take substantial time and cost to discover effect and, crucially, to validate repositioned effects. Methods Using a novel in-silico quasi-quantum molecular simulation platform that analyzes energies and electron densities of both target proteins and candidate interruption compounds on High Performance Computing (HPC), we identified a list of FDA-approved compounds with potential to interrupt specific SARS-CoV-2 proteins. Subsequently we used 1.5M patient records from the National COVID Cohort Collaborative to create matched cohorts to refine our in-silico hits to those candidates that show statistically significant clinical effect. Results We identified four drugs, Metformin, Triamcinolone, Amoxicillin and Hydrochlorothiazide, that were associated with reduced mortality by 27%, 26%, 26%, and 23%, respectively, in COVID-19 patients. Conclusions Together, these findings provide support to our hypothesis that in-silico simulation of active compounds against SARS-CoV-2 proteins followed by statistical analysis of electronic health data results in effective therapeutics identification.
[Boye]
Retrospective 9531 COVID+ diabetes patients in the USA, showing lower risk of hospitalization with existing biguanides treatment (defined as mainly metformin in the abstract and entirely metformin in the text).
[Bramante]
Retrospective 6,256 COVID-19+ diabetes patients in the USA, showing lower mortality with existing metformin treatment, statistically significant only for women.