Mirror therapy is an often used evidence-based therapeutical approach to improve motor function after stroke. During mirror therapy, a mirror is placed in the person’s midsagittal plane, thus reflecting movements of the non-paretic side as if it were the affected side.
Picture copyright: http://aerzteblatt.de
In the July 11. 2018 updated Cochrane review the researchers conducted a meta-analysis out of 62 relevant studies with a total of 1982 stroke participants to summarise the effectiveness of mirror therapy compared with no treatment, placebo or sham therapy, or other treatments for improving motor function and motor impairment after stroke.
When compared with all other interventions, the researches found evidence that mirror therapy has a significant positive effect on motor function (SMD 0.47, 95%CI 0.27 to 0.67; 1173 participants; 36 studies)
and motor impairment (SMD 0.49, 95% CI 0.32 to 0.66; 1292 participants; 39 studies). The results are indicating that mirror therapy "moderately" improves movement of the affected upper and lower limb of stroke patients.
When interpreting those effect sizes, researchers very often use Cohen's guidelines of small (0.2), medium (0.5), and large (0.8) effects. However, these guidelines were originally more proposed as a fallback if the effect size distribution (ESD) is unknown. Despite the availability of the "true" effect sizes from existing mirror therapy studies, researchers often rely on Cohen's guidelines to interpret effect sizes and to perform power analyses to calculate required sample sizes for future research. For sure that is the easiest but not always the best option.
Because I'm a bit a nerd in that, I wanted to know whether the ESD's for mirror therapy really fit to Cohen's rules or if we go wrong when interpreting the effects of mirror therapy with those standards.
Fig 1. The graph is showing the effect size distributions for the two main outcomes: 1) Motor function at the end of intervention phase (red) and 2) Motor impairment at the end of intervention phase (blue).
The ESD analysis of the included studies reveals that the 25th, 50th, and 75th effect size percentiles for motor function are corresponding with effect sizes of 0.09, 0.37, and 0.85 and that the percentiles for motor impairment are corresponding with effect sizes of 0.18, 0.5 and 0.92.
In conclusion the analyses suggest that Cohen's guidelines overestimate the magnitude of a small effect size for both outcomes and underestimate the magnitude of a large effect size, especially for the impairment outcome. While the medium effect size for the impairment - with an exact value of 0.5 - perfectly fits to the rule, we also overestimate the medium effect size for the functional outcome if we are following Cohen's rules.
Therefore, to better reflect the observed ESD, effect sizes of 0.1, 0.35, and 0.85 for motor function and 0.2, 0.5 and 0.9 for impairment (rounded to the closest .05) should be interpreted as small, medium, and large effect.
Fig 2. A priori computed required sample size - given alpha .5, beta .8 and ES from 0.1 to 1.0 in steps of 0.1
Based on power calculations using the ESDs from above, we should keep in mind that for a sufficiantly powered study with medium effect size a minimum sample size of 128 participants (ES 0.5, impairment) respectively 260 participants (ES 0.35, motor function) is needed and that existing mirror therapy studies with round about 33 participants are generally underpowered! Together with lacking reports of methodological details these are the major limitations of the updated review. The suggested sample sizes should be achieved in future to get suitably powered studies that are more likely to replicate and hopefully underpin the true population effect size.