2 In Silico studies support the efficacy of vitamin A [Li, Pandya].

An In Vitro study supports the efficacy of vitamin A [Morita].

Preclinical research is an important part of the development of treatments, however results may be very different in clinical trials. Preclinical results are not used in this paper.

Figure 3 shows a visual overview of the results, with details in Table 1 and Table 2. Figure 4, 5, 6, 7, 8, 9, 10, 11, 12, and 13 show forest plots for a random effects meta-analysis of all studies with pooled effects, mortality results, ICU admission, hospitalization, progression, recovery, cases, viral clearance, sufficiency studies, and peer reviewed studies.

To avoid bias in the selection of studies, we analyze all non-retracted studies. Here we show the results after excluding studies with major issues likely to alter results, non-standard studies, and studies where very minimal detail is currently available. Our bias evaluation is based on analysis of each study and identifying when there is a significant chance that limitations will substantially change the outcome of the study. We believe this can be more valuable than checklist-based approaches such as Cochrane GRADE, which may underemphasize serious issues not captured in the checklists, overemphasize issues unlikely to alter outcomes in specific cases (for example, lack of blinding for an objective mortality outcome, or certain specifics of randomization with a very large effect size), or be easily influenced by potential bias. However, they can also be very high quality.

The studies excluded are as below. Figure 14 shows a forest plot for random effects meta-analysis of all studies after exclusions.

[Al-Sumiadai], minimal details of groups provided.

[Holt], significant unadjusted confounding possible.

[Sarohan], unadjusted results with no group details.

Heterogeneity in COVID-19 studies arises from many factors including:

Treatment delay.

The time between infection or the onset of symptoms and treatment may critically affect how well a treatment works. For example an antiviral may be very effective when used early but may not be effective in late stage disease, and may even be harmful. Oseltamivir, for example, is generally only considered effective for influenza when used within 0-36 or 0-48 hours [McLean, Treanor]. Figure 17 shows a mixed-effects meta-regression for efficacy as a function of treatment delay in COVID-19 studies from 42 treatments, showing that efficacy declines rapidly with treatment delay. Early treatment is critical for COVID-19.

Patient demographics.

Details of the patient population including age and comorbidities may critically affect how well a treatment works. For example, many COVID-19 studies with relatively young low-comorbidity patients show all patients recovering quickly with or without treatment. In such cases, there is little room for an effective treatment to improve results (as in [López-Medina]).

Effect measured.

Efficacy may differ significantly depending on the effect measured, for example a treatment may be very effective at reducing mortality, but less effective at minimizing cases or hospitalization. Or a treatment may have no effect on viral clearance while still being effective at reducing mortality.

Variants.

There are many different variants of SARS-CoV-2 and efficacy may depend critically on the distribution of variants encountered by the patients in a study. For example, the Gamma variant shows significantly different characteristics [Faria, Karita, Nonaka, Zavascki]. Different mechanisms of action may be more or less effective depending on variants, for example the viral entry process for the omicron variant has moved towards TMPRSS2-independent fusion, suggesting that TMPRSS2 inhibitors may be less effective [Peacock, Willett].

Regimen.

Effectiveness may depend strongly on the dosage and treatment regimen.

Treatments.

The use of other treatments may significantly affect outcomes, including anything from supplements, other medications, or other kinds of treatment such as prone positioning.

The distribution of studies will alter the outcome of a meta analysis. Consider a simplified example where everything is equal except for the treatment delay, and effectiveness decreases to zero or below with increasing delay. If there are many studies using very late treatment, the outcome may be negative, even though the treatment may be very effective when used earlier.

In general, by combining heterogeneous studies, as all meta analyses do, we run the risk of obscuring an effect by including studies where the treatment is less effective, not effective, or harmful.

When including studies where a treatment is less effective we expect the estimated effect size to be lower than that for the optimal case. We do not a priori expect that pooling all studies will create a positive result for an effective treatment. Looking at all studies is valuable for providing an overview of all research, important to avoid cherry-picking, and informative when a positive result is found despite combining less-optimal situations. However, the resulting estimate does not apply to specific cases such as early treatment in high-risk populations.

Publication bias.

Publishing is often biased towards positive results, however evidence suggests that there may be a negative bias for inexpensive treatments for COVID-19. Both negative and positive results are very important for COVID-19, media in many countries prioritizes negative results for inexpensive treatments (inverting the typical incentive for scientists that value media recognition), and there are many reports of difficulty publishing positive results [Boulware, Meeus, Meneguesso]. For vitamin A, there is currently not enough data to evaluate publication bias with high confidence.

One method to evaluate bias is to compare prospective vs. retrospective studies. Prospective studies are more likely to be published regardless of the result, while retrospective studies are more likely to exhibit bias. For example, researchers may perform preliminary analysis with minimal effort and the results may influence their decision to continue. Retrospective studies also provide more opportunities for the specifics of data extraction and adjustments to influence results.

67% of retrospective studies report a statistically significant positive effect for one or more outcomes, compared to 100% of prospective studies, consistent with a bias toward publishing negative results. The median effect size for retrospective studies is 21% improvement, compared to 67% for prospective studies, suggesting a potential bias towards publishing results showing lower efficacy. Figure 18 shows a scatter plot of results for prospective and retrospective treatment studies.

Funnel plot analysis.

Funnel plots have traditionally been used for analyzing publication bias. This is invalid for COVID-19 acute treatment trials — the underlying assumptions are invalid, which we can demonstrate with a simple example. Consider a set of hypothetical perfect trials with no bias. Figure 19 plot A shows a funnel plot for a simulation of 80 perfect trials, with random group sizes, and each patient's outcome randomly sampled (10% control event probability, and a 30% effect size for treatment). Analysis shows no asymmetry (p > 0.05). In plot B, we add a single typical variation in COVID-19 treatment trials — treatment delay. Consider that efficacy varies from 90% for treatment within 24 hours, reducing to 10% when treatment is delayed 3 days. In plot B, each trial's treatment delay is randomly selected. Analysis now shows highly significant asymmetry, p < 0.0001, with six variants of Egger's test all showing p < 0.05 [Egger, Harbord, Macaskill, Moreno, Peters, Rothstein, Rücker, Stanley]. Note that these tests fail even though treatment delay is uniformly distributed. In reality treatment delay is more complex — each trial has a different distribution of delays across patients, and the distribution across trials may be biased (e.g., late treatment trials may be more common). Similarly, many other variations in trials may produce asymmetry, including dose, administration, duration of treatment, differences in SOC, comorbidities, age, variants, and bias in design, implementation, analysis, and reporting.

Conflicts of interest.

Pharmaceutical drug trials often have conflicts of interest whereby sponsors or trial staff have a financial interest in the outcome being positive. Vitamin A for COVID-19 lacks this because it is an inexpensive and widely available supplement. In contrast, most COVID-19 vitamin A trials have been run by physicians on the front lines with the primary goal of finding the best methods to save human lives and minimize the collateral damage caused by COVID-19. While pharmaceutical companies are careful to run trials under optimal conditions (for example, restricting patients to those most likely to benefit, only including patients that can be treated soon after onset when necessary, and ensuring accurate dosing), not all vitamin A trials represent the optimal conditions for efficacy.

Early/late vs. mild/moderate/severe.

Some analyses classify treatment based on early/late administration (as we do here), while others distinguish between mild/moderate/severe cases. We note that viral load does not indicate degree of symptoms — for example patients may have a high viral load while being asymptomatic. With regard to treatments that have antiviral properties, timing of treatment is critical — late administration may be less helpful regardless of severity.

Notes.

1 of 8 studies combine treatments. The results of vitamin A alone may differ. 1 of 2 RCTs use combined treatment.

Statistically significant improvements are seen for recovery, cases, and viral clearance. 4 studies from 3 independent teams in 3 different countries show statistically significant improvements in isolation (2 for the most serious outcome). Meta analysis using the most serious outcome reported shows 50% [-9‑77%] improvement, without reaching statistical significance. Results are better for Randomized Controlled Trials, similar after exclusions, and better for peer-reviewed studies. Results are consistent with early treatment being more effective than late treatment. Sufficiency studies, analyzing outcomes based on serum levels, show 73% [51‑85%] improvement for patients with higher vitamin A levels (3 studies).

Al-Sumiadai

[Al-Sumiadai (C)] Treatment and prophylaxis studies of vitamin A in Iraq.

The treatment study contained 100 patients, 50 treated with 200,000IU vitamin A for two days, showing lower progression to severe disease, and shorter duration of symptoms.

The prophylaxis study contained 209 contacts of COVID-19 patients, 97 treated with vitamin A, showing significantly lower cases with treatment, and shorter duration of symptoms.

Submit Corrections or Updates.

Al-Sumiadai

[Al-Sumiadai (B)] Treatment and prophylaxis studies of vitamin A in Iraq.

The treatment study contained 100 patients, 50 treated with 200,000IU vitamin A for two days, showing lower progression to severe disease, and shorter duration of symptoms.

The prophylaxis study contained 209 contacts of COVID-19 patients, 97 treated with vitamin A, showing significantly lower cases with treatment, and shorter duration of symptoms.

Submit Corrections or Updates.

Al-Sumiadai

[Al-Sumiadai] Retrospective 70 severe condition patients treated with vitamin A (200,000IU for two days), salbutamol, and budesonide, and 70 patients not treated with vitamin A, showing significantly lower mortality with the addition of vitamin A.

Submit Corrections or Updates.

Beigmohammadi

[Beigmohammadi] Small RCT 60 ICU patients in Iran, 30 treated with vitamins A, B, C, D, and E, showing significant improvement in SOFA score and several inflammatory markers at day 7 with treatment.

5,000 IU vitamin A daily, 600,000 IU vitamin D once, 300 IU of vitamin E twice a day, 500 mg vitamin C four times a day, and one ampule daily of B vitamins [thiamine nitrate 3.1 mg, sodium riboflavin phosphate 4.9 mg (corresponding to vitamin B2 3.6 mg), nicotinamide 40 mg, pyridoxine hydrochloride 4.9 mg (corresponding to vitamin B6 4.0 mg), sodium pantothenate 16.5 mg (corresponding to pantothenic acid 15 mg), sodium ascorbate 113 mg (corresponding to vitamin C 100 mg), biotin 60 μg, folic acid 400 μg, and cyanocobalamin 5 μg]. IRCT20200319046819N [irct.ir].

Submit Corrections or Updates.

Elkazzaz

[Elkazzaz] RCT with 20 13-cis-retinoic acid patients and 20 control patients, showing faster recovery and viral clearance with treatment. Aerosolized 13-cis-retinoic acid with increasing dose from 0.2 mg/kg/day to 4 mg/kg/day for 14 days, plus oral 13-cis-retinoic acid 20 mg/day. 13-cis retinoic acid is a synthetic vitamin A derivative, and is teratogenic. NCT04353180.

Submit Corrections or Updates.

Holt

[Holt] Prospective survey-based study with 15,227 people in the UK, showing lower risk of COVID-19 cases with vitamin A, vitamin D, zinc, selenium, probiotics, and inhaled corticosteroids; and higher risk with metformin and vitamin C. Statistical significance was not reached for any of these. Except for vitamin D, the results for treatments we follow were only adjusted for age, sex, duration of participation, and test frequency. NCT04330599. COVIDENCE UK.

Submit Corrections or Updates.

Nimer