More details on the CASCADE trial:

Table 2 of the 2015 Annals of Internal Medicine paper lists 5 “primary” and 10 “secondary” outcomes that are involved in two comparisons: white wine vs. water and red wine vs. water. This already represents 30 statistical tests. We don’t know how many additional tests were run, but it is likely to be large because the authors engaged in outcome switching. Outcome switching removes constraints on the number of analyses run. There is good evidence that the authors ran additional analyses that don’t appear in the paper, such as looking at cardiovascular disease status and total cholesterol and comparing outcomes at 6 months.

The Annals of Internal Medicine paper highlights five results measured at 2 years in which p≤.05:

  1.  White wine versus water on fasting plasma glucose (p=.004)
  2. White wine versus water on homeostatic model assessment of insulin resistance, HOMA-IA (p=.019)
  3. Red wine versus water on HDL cholesterol (p<.001)
  4. Red wine versus water on cholesterol:HDL ratio (p=.039)
  5. Red wine versus water on apolipoprotein a (p=.05)

All five of these outcome variables are labeled as the primary outcomes in the paper. However, this appears to be a case of outcome switching. In the original preregistered protocol on ClinicalTrials.gov, the authors list “glycemic control” at 6 months as the sole primary outcome. Glycemic control is not defined, but this is typically measured as HbA1c levels. In contrast, the primary outcomes reported in the paper were 2-year measurements of: high-density lipoprotein cholesterol, apolipoprotein a, total cholesterol: HDL ratio, fasting plasma glucose, and homeostatic model assessment of insulin resistance. Only one secondary outcome is listed in the original protocol: cardiovascular disease status at 6 months. This variable is never reported in the paper. Rather, 10 other variables are reported as secondary outcomes: apolipoprotein b: apopliprotein a ratio, triglyceride level, apolipoprotein b, LDL cholesterol, HbA1c, fasting insulin level, systolic blood pressure, diastolic blood pressure, waist circumference, and weight.

Of all these comparisons, only the five comparisons above plus triglyceride level achieved (or nearly achieved) statistical significance at p<.05. Given the large number of tests run in the background, these few significant results are consistent with chance findings. Since the five “primary” outcome variables were not prespecified in the protocol, this appears to be a case of cherry-picking after the fact.

A Letter to the Editor by the COMPare Team also points out this outcome switching (Slade E, Drysdale H, Goldacre B, et al. Discrepancies between prespecified and reported outcomes. Ann Intern Med. 2016;164:374.)