Covid Analysis, October 3, 2022, DRAFT
https://c19early.com/vameta.html
•Statistically significant improvements are seen for recovery, cases, and viral clearance. 5 studies from 4 independent teams in 3 different countries show statistically significant
improvements in isolation (2 for the most serious outcome).
•Meta analysis using the most serious outcome reported shows
46% [-6‑72%] improvement, without reaching statistical significance. Results are similar for Randomized Controlled Trials, similar after exclusions, and similar for peer-reviewed studies.
•Sufficiency studies, analyzing outcomes based on serum levels,
show 73% [51‑85%] improvement for patients with higher vitamin A levels
(3 studies).
•No treatment, vaccine, or
intervention is 100% effective and available. All practical, effective, and
safe means should be used based on risk/benefit analysis.
Multiple treatments are typically used
in combination, and other treatments
may be more effective.
Only 22% of vitamin A
studies show zero events with treatment.
The quality of non-prescription supplements can vary widely [Crawford, Crighton].
•All data to reproduce this paper and
sources are in the appendix.
Highlights
Vitamin A reduces
risk for COVID-19 with very high confidence for recovery, low confidence for cases, viral clearance, and in pooled analysis, and very low confidence for ICU admission, hospitalization, and progression.
We show traditional outcome specific analyses and combined
evidence from all studies, incorporating treatment delay, a primary
confounding factor in COVID-19 studies.
Real-time updates and corrections,
transparent analysis with all results in the same format, consistent protocol
for 47
treatments.
Figure 1. A. Random effects
meta-analysis. This plot shows pooled effects, discussion can be found in the heterogeneity section,
and results for specific outcomes can be found in the individual outcome analyses.
Effect extraction is pre-specified, using the most serious outcome reported.
For details of effect extraction see the appendix.
B. Scatter plot showing the
distribution of effects reported in studies. C. History of all reported
effects (chronological within treatment stages).
Introduction
We analyze all significant studies
concerning the use of
vitamin A
for COVID-19.
Search methods, inclusion criteria, effect
extraction criteria (more serious outcomes have priority), all individual
study data, PRISMA answers, and statistical methods are detailed in
Appendix 1. We present random effects meta-analysis results for all
studies, for studies within each treatment stage, for individual outcomes, for
peer-reviewed studies, for Randomized Controlled Trials (RCTs), and after
exclusions.
Figure 2 shows stages of possible treatment for
COVID-19. Prophylaxis refers to regularly taking medication before
becoming sick, in order to prevent or minimize infection. Early
Treatment refers to treatment immediately or soon after symptoms appear,
while Late Treatment refers to more delayed treatment.
Figure 2. Treatment stages.
Preclinical Research
3 In Silico studies support the efficacy of vitamin A [Chakraborty, Li, Pandya].
Preclinical research is an important part of the development of
treatments, however results may be very different in clinical trials.
Preclinical results are not used in this paper.
Results
Figure 3 shows a visual overview of the results, with details in
Table 1 and Table 2.
Figure 4, 5, 6, 7, 8, 9, 10, 11, 12, and 13
show forest plots for a random effects meta-analysis of
all studies with pooled effects, mortality results, ICU admission, hospitalization, progression, recovery, cases, viral clearance, sufficiency studies, and peer reviewed studies.
Figure 3. Overview of results.
| Treatment time | Number of studies reporting positive effects | Total number of studies | Percentage of studies reporting positive effects | Random effects meta-analysis results |
| Early treatment | 3 | 3 | 100% |
62% improvement RR 0.38 [0.14‑1.03] p = 0.058 |
| Late treatment | 2 | 3 | 66.7% |
48% improvement RR 0.52 [0.04‑7.80] p = 0.65 |
| Prophylaxis | 3 | 3 | 100% |
49% improvement RR 0.51 [0.27‑0.99] p = 0.046 |
| All studies | 8 | 9 | 88.9% |
46% improvement RR 0.54 [0.28‑1.06] p = 0.072 |
Table 1. Results by treatment stage.
| Studies | Early treatment | Late treatment | Prophylaxis | Patients | Authors | |
| All studies | 9 | 62% [-3‑86%] | 48% [-680‑96%] | 49% [1‑73%] | 18,131 | 67 |
| With exclusions | 6 | 38% [-31‑71%] | 87% [3‑98%] | 48% [-14‑76%] | 2,737 | 26 |
| Peer-reviewed | 5 | 64% [-80‑93%] | 89% [-98‑99%] | 24% [-23‑53%] | 17,755 | 53 |
| Randomized Controlled TrialsRCTs | 3 | 26% [-76‑69%] | 87% [3‑98%] | 280 | 16 |
Table 2. Results by treatment stage for all studies and with different exclusions.
Figure 4. Random effects meta-analysis for all studies with pooled effects.
This plot shows pooled effects, discussion can be found in the heterogeneity section,
and results for specific outcomes can be found in the individual outcome analyses.
Effect extraction is pre-specified, using the most serious outcome reported.
For details of effect extraction see the appendix.
Figure 5. Random effects meta-analysis for mortality results.
Figure 6. Random effects meta-analysis for ICU admission.
Figure 7. Random effects meta-analysis for hospitalization.
Figure 8. Random effects meta-analysis for progression.
Figure 9. Random effects meta-analysis for recovery.
Figure 10. Random effects meta-analysis for cases.
Figure 11. Random effects meta-analysis for viral clearance.
Figure 12. Random effects meta-analysis for sufficiency studies.
Effect extraction is pre-specified, using the most serious outcome reported,
see the appendix for details.
Figure 13. Random effects meta-analysis for peer reviewed studies.
[Zeraatkar] analyze 356 COVID-19 trials, finding no
significant evidence that peer-reviewed studies are more trustworthy.
They also show extremely slow review times during a pandemic. Authors
recommend using preprint evidence, with appropriate checks for potential
falsified data, which provides higher certainty much earlier.
Effect extraction is pre-specified, using the most serious outcome reported,
see the appendix for details.
Exclusions
To avoid bias in the selection of studies, we analyze all
non-retracted studies. Here we show the results after excluding studies with
major issues likely to alter results, non-standard studies, and studies where
very minimal detail is currently available. Our bias evaluation is based on
analysis of each study and identifying when there is a significant chance that
limitations will substantially change the outcome of the study. We believe
this can be more valuable than checklist-based approaches such as Cochrane
GRADE, which may underemphasize serious issues not captured in the checklists,
overemphasize issues unlikely to alter outcomes in specific cases (for
example, lack of blinding for an objective mortality outcome, or certain
specifics of randomization with a very large effect size), or be easily
influenced by potential bias. However, they can also be very high
quality.
The studies excluded are as below.
Figure 14 shows a forest plot for random
effects meta-analysis of all studies after exclusions.
[Al-Sumiadai], minimal details of groups provided.
[Holt], significant unadjusted confounding possible.
[Sarohan], unadjusted results with no group details.
Figure 14. Random effects meta-analysis for all studies after exclusions.
This plot shows pooled effects, discussion can be found in the heterogeneity section,
and results for specific outcomes can be found in the individual outcome analyses.
Effect extraction is pre-specified, using the most serious outcome reported.
For details of effect extraction see the appendix.
Randomized Controlled Trials (RCTs)
Figure 15 shows the distribution of results for Randomized Controlled Trials and other studies, and
a chronological history of results.
The median effect size for
RCTs is 86% improvement,
compared to 67% for other studies.
Figure 16 and 17
show forest plots for a random effects meta-analysis of
all Randomized Controlled Trials and RCT mortality results.
Table 3 summarizes the results.
Figure 15. The distribution of results for Randomized Controlled Trials and other studies, and
a chronological history of results.
Figure 16. Random effects meta-analysis for all Randomized Controlled Trials.
This plot shows pooled effects, discussion can be found in the heterogeneity section,
and results for specific outcomes can be found in the individual outcome analyses.
Effect extraction is pre-specified, using the most serious outcome reported.
For details of effect extraction see the appendix.
Figure 17. Random effects meta-analysis for RCT mortality results.
| Treatment time | Number of studies reporting positive effects | Total number of studies | Percentage of studies reporting positive effects | Random effects meta-analysis results |
| Randomized Controlled Trials | 3 | 3 | 100% |
55% improvement RR 0.45 [0.14‑1.43] p = 0.18 |
| RCT mortality results | 2 | 2 | 100% |
87% improvement RR 0.13 [0.02‑0.97] p = 0.046 |
Table 3. Randomized Controlled Trial results.
Heterogeneity
Heterogeneity in COVID-19 studies arises from many factors including:
Treatment delay.
The time between infection
or the onset of symptoms and treatment may critically affect how well a
treatment works. For example an antiviral may be very effective when used
early but may not be effective in late stage disease, and may even be harmful.
Oseltamivir, for example, is generally only considered effective for influenza
when used within 0-36 or 0-48 hours [McLean, Treanor].
Figure 18 shows a mixed-effects meta-regression for efficacy
as a function of treatment delay in COVID-19 studies from 47 treatments, showing
that efficacy declines rapidly with treatment delay. Early treatment is
critical for COVID-19.
Figure 18. Meta-regression
showing efficacy as a function of treatment delay in COVID-19 studies from 47 treatments. Early
treatment is critical.
Patient demographics.
Details of the
patient population including age and comorbidities may critically affect how
well a treatment works. For example, many COVID-19 studies with relatively
young low-comorbidity patients show all patients recovering quickly with or
without treatment. In such cases, there is little room for an effective
treatment to improve results (as in [López-Medina]).Effect measured.
Efficacy may differ
significantly depending on the effect measured, for example a treatment may be
very effective at reducing mortality, but less effective at minimizing cases
or hospitalization. Or a treatment may have no effect on viral clearance while
still being effective at reducing mortality.Variants.
There are many different
variants of SARS-CoV-2 and efficacy may depend critically on the distribution
of variants encountered by the patients in a study. For example, the Gamma
variant shows significantly different characteristics
[Faria, Karita, Nonaka, Zavascki]. Different mechanisms of action may be
more or less effective depending on variants, for example the viral entry
process for the omicron variant has moved towards TMPRSS2-independent fusion,
suggesting that TMPRSS2 inhibitors may be less effective
[Peacock, Willett].Regimen.
Effectiveness may depend strongly on the dosage and treatment regimen.
Other treatments.
The use of other
treatments may significantly affect outcomes, including anything from
supplements, other medications, or other kinds of treatment such as prone
positioning.Medication quality.
The
quality of medications may vary significantly between manufacturers and
production batches, which may significantly affect efficacy and safety.
[Williams] analyze ivermectin from 11 different sources, showing
highly variable antiparasitic efficacy across different manufacturers.
[Xu] analyze a treatment from two different manufacturers, showing 9
different impurities, with significantly different concentrations for each
manufacturer.
Non-prescription supplements may show very wide variations in quality
[Crawford, Crighton].
Meta analysis.
The
distribution of studies will alter the outcome of a meta analysis. Consider a
simplified example where everything is equal except for the treatment delay,
and effectiveness decreases to zero or below with increasing delay. If there
are many studies using very late treatment, the outcome may be negative, even
though the treatment may be very effective when used earlier.In general, by combining heterogeneous studies, as all meta
analyses do, we run the risk of obscuring an effect by including studies where
the treatment is less effective, not effective, or harmful.
When including studies where a treatment is less effective we
expect the estimated effect size to be lower than that for the optimal case.
We do not a priori expect that pooling all studies will create a
positive result for an effective treatment. Looking at all studies is valuable
for providing an overview of all research, important to avoid cherry-picking,
and informative when a positive result is found despite combining less-optimal
situations. However, the resulting estimate does not apply to specific cases
such as
early treatment in high-risk populations.
While we present pooled results for all studies, we also present individual
outcome and treatment time analyses, which are more relevant for specific use
cases.
Discussion
Publication bias.
Publishing is often biased
towards positive results, however evidence suggests that there may be a negative bias for
inexpensive treatments for COVID-19. Both negative and positive results are
very important for COVID-19, media in many countries prioritizes negative
results for inexpensive treatments (inverting the typical incentive for
scientists that value media recognition), and there are many reports of
difficulty publishing positive results
[Boulware, Meeus, Meneguesso].
For vitamin A, there is currently not
enough data to evaluate publication bias with high confidence.
One method to evaluate bias is to compare prospective vs.
retrospective studies. Prospective studies are more likely to be published
regardless of the result, while retrospective studies are more likely to
exhibit bias. For example, researchers may perform preliminary analysis with
minimal effort and the results may influence their decision to continue.
Retrospective studies also provide more opportunities for the specifics of
data extraction and adjustments to influence results.
67% of retrospective studies
report positive effects, compared to
100% of prospective studies, consistent with a bias toward publishing negative results.
The median effect size for
retrospective studies is 21% improvement,
compared to 66% for prospective
studies, suggesting a potential bias towards publishing results showing lower efficacy.
Figure 19 shows a scatter plot of
results for prospective and retrospective treatment studies.
Figure 19. Prospective vs. retrospective studies.
Funnel plot analysis.
Funnel
plots have traditionally been used for analyzing publication bias. This is
invalid for COVID-19 acute treatment trials — the underlying assumptions
are invalid, which we can demonstrate with a simple example. Consider a set of
hypothetical perfect trials with no bias. Figure 20 plot A
shows a funnel plot for a simulation of 80 perfect trials, with random group
sizes, and each patient's outcome randomly sampled (10% control event
probability, and a 30% effect size for treatment). Analysis shows no asymmetry
(p > 0.05). In plot B, we add a single typical variation in COVID-19 treatment
trials — treatment delay. Consider that efficacy varies from 90% for
treatment within 24 hours, reducing to 10% when treatment is delayed 3 days.
In plot B, each trial's treatment delay is randomly selected. Analysis now
shows highly significant asymmetry, p < 0.0001, with six variants of
Egger's test all showing p < 0.05
[Egger, Harbord, Macaskill, Moreno, Peters, Rothstein, Rücker, Stanley].
Note that these tests fail even though treatment delay is uniformly
distributed. In reality treatment delay is more complex — each trial has
a different distribution of delays across patients, and the distribution
across trials may be biased (e.g., late treatment trials may be more common).
Similarly, many other variations in trials may produce asymmetry, including
dose, administration, duration of treatment, differences in SOC,
comorbidities, age, variants, and bias in design, implementation, analysis,
and reporting.
Figure 20. Example funnel plot analysis for
simulated perfect trials.
Conflicts of interest.
Pharmaceutical drug
trials often have conflicts of interest whereby sponsors or trial staff have a
financial interest in the outcome being positive. Vitamin A for COVID-19
lacks this because it is an inexpensive and widely available supplement.
In contrast, most COVID-19 vitamin A trials have been run by
physicians on the front lines with the primary goal of finding the best
methods to save human lives and minimize the collateral damage caused by
COVID-19. While pharmaceutical companies are careful to run trials under
optimal conditions (for example, restricting patients to those most likely to
benefit, only including patients that can be treated soon after onset when
necessary, and ensuring accurate dosing), not all vitamin A trials
represent the optimal conditions for efficacy.
Early/late vs. mild/moderate/severe.
Some analyses classify treatment based on early/late administration (as we do
here), while others distinguish between mild/moderate/severe cases. We note
that viral load does not indicate degree of symptoms — for example
patients may have a high viral load while being asymptomatic. With regard to
treatments that have antiviral properties, timing of treatment is
critical — late administration may be less helpful regardless of
severity.Notes.
1 of 9 studies
combine treatments. The results of
vitamin A
alone may differ.
1 of 3 RCTs use combined treatment.
Conclusion
Statistically significant improvements are seen for recovery, cases, and viral clearance. 5 studies from 4 independent teams in 3 different countries show statistically significant
improvements in isolation (2 for the most serious outcome).
Meta analysis using the most serious outcome reported shows
46% [-6‑72%] improvement, without reaching statistical significance. Results are similar for Randomized Controlled Trials, similar after exclusions, and similar for peer-reviewed studies.
Sufficiency studies, analyzing outcomes based on serum levels,
show 73% [51‑85%] improvement for patients with higher vitamin A levels
(3 studies).
Study Notes
[Al-Sumiadai (C)]
Treatment and prophylaxis studies of vitamin A in Iraq.
The treatment study contained 100 patients, 50 treated with 200,000IU vitamin A for two days, showing lower progression to severe disease, and shorter duration of symptoms.
The prophylaxis study contained 209 contacts of COVID-19 patients, 97 treated with vitamin A, showing significantly lower cases with treatment, and shorter duration of symptoms.
The treatment study contained 100 patients, 50 treated with 200,000IU vitamin A for two days, showing lower progression to severe disease, and shorter duration of symptoms.
The prophylaxis study contained 209 contacts of COVID-19 patients, 97 treated with vitamin A, showing significantly lower cases with treatment, and shorter duration of symptoms.
[Al-Sumiadai (B)]
Treatment and prophylaxis studies of vitamin A in Iraq.
The treatment study contained 100 patients, 50 treated with 200,000IU vitamin A for two days, showing lower progression to severe disease, and shorter duration of symptoms.
The prophylaxis study contained 209 contacts of COVID-19 patients, 97 treated with vitamin A, showing significantly lower cases with treatment, and shorter duration of symptoms.
The treatment study contained 100 patients, 50 treated with 200,000IU vitamin A for two days, showing lower progression to severe disease, and shorter duration of symptoms.
The prophylaxis study contained 209 contacts of COVID-19 patients, 97 treated with vitamin A, showing significantly lower cases with treatment, and shorter duration of symptoms.
[Al-Sumiadai]
Retrospective 70 severe condition patients treated with vitamin A (200,000IU for two days), salbutamol, and budesonide, and 70 patients not treated with vitamin A, showing significantly lower mortality with the addition of vitamin A.
[Beigmohammadi]
Small RCT 60 ICU patients in Iran, 30 treated with vitamins A, B, C, D, and E, showing significant improvement in SOFA score and several inflammatory markers at day 7 with treatment.
5,000 IU vitamin A daily, 600,000 IU vitamin D once, 300 IU of vitamin E twice a day, 500 mg vitamin C four times a day, and one ampule daily of B vitamins [thiamine nitrate 3.1 mg, sodium riboflavin phosphate 4.9 mg (corresponding to vitamin B2 3.6 mg), nicotinamide 40 mg, pyridoxine hydrochloride 4.9 mg (corresponding to vitamin B6 4.0 mg), sodium pantothenate 16.5 mg (corresponding to pantothenic acid 15 mg), sodium ascorbate 113 mg (corresponding to vitamin C 100 mg), biotin 60 μg, folic acid 400 μg, and cyanocobalamin 5 μg]. IRCT20200319046819N [irct.ir].
5,000 IU vitamin A daily, 600,000 IU vitamin D once, 300 IU of vitamin E twice a day, 500 mg vitamin C four times a day, and one ampule daily of B vitamins [thiamine nitrate 3.1 mg, sodium riboflavin phosphate 4.9 mg (corresponding to vitamin B2 3.6 mg), nicotinamide 40 mg, pyridoxine hydrochloride 4.9 mg (corresponding to vitamin B6 4.0 mg), sodium pantothenate 16.5 mg (corresponding to pantothenic acid 15 mg), sodium ascorbate 113 mg (corresponding to vitamin C 100 mg), biotin 60 μg, folic acid 400 μg, and cyanocobalamin 5 μg]. IRCT20200319046819N [irct.ir].
[Elkazzaz]
RCT with 20 13-cis-retinoic acid patients and 20 control patients, showing faster recovery and viral clearance with treatment. Aerosolized 13-cis-retinoic acid with increasing dose from 0.2 mg/kg/day to 4 mg/kg/day for 14 days, plus oral 13-cis-retinoic acid 20 mg/day. 13-cis retinoic acid is a synthetic vitamin A derivative, and is teratogenic. NCT04353180.
[Holt]
Prospective survey-based study with 15,227 people in the UK, showing lower risk of COVID-19 cases with vitamin A, vitamin D, zinc, selenium, probiotics, and inhaled corticosteroids; and higher risk with metformin and vitamin C. Statistical significance was not reached for any of these. Except for vitamin D, the results for treatments we follow were only adjusted for age, sex, duration of participation, and test frequency. NCT04330599. COVIDENCE UK.
[Nimer]
Retrospective survey based analysis of 2,148 COVID-19 recovered patients in Jordan, showing no significant differences in the risk of severity and hospitalization with vitamin A prophylaxis.
