In the field of medicine, no drug reaches the market without going through rigorous clinical trials. In EdTech, tools that affect millions of children are often adopted based on testimonials, uncontrolled case studies, and impressive demonstrations. Randomized Controlled Trials (RCTs) are the antidote to that methodological fragility.

What is an RCT and why does it matter?

A Randomized Controlled Trial randomly assigns participants (in our case, institutions or teachers) to an intervention group—which receives the tool—or a control group—which continues with its usual practices. Randomness is key: it ensures that observed differences in results are due to the intervention and not to previous characteristics of the groups.

“Without a control group, we cannot distinguish between ‘the tool worked’ and ‘this time of year students always improve’. The RCT makes that distinction possible.” — Daniela Reyes, Mentu Labs

RCTs in EdTech: the real challenges

Implementing an RCT in education is significantly more difficult than in medicine. Students are not pills; they interact with each other, with teachers, and with their environment in ways that can contaminate the intervention’s effect. The absence of a teacher in the control group can influence results more than the tool itself.

  • Contamination between groups: teachers in the control group may learn from their colleagues in the intervention group.
  • Hawthorne Effect: participants change their behavior simply by knowing they are being observed.
  • Required sample sizes: to detect modest effects (d=0.2), hundreds of students are needed.
  • Implementation time: real educational effects usually take months or years to manifest.
  • Ethics of denying access: random assignment implies that some students will not receive a potentially beneficial intervention.

Our approach at Mentu Labs

Mentu Labs research team

We adopt a stepped approach: we start with simpler quasi-experimental designs and, when context and resources allow, we scale to full RCTs. Not because RCTs are always the right answer, but because they are the standard that allows us to make honest causal claims about our tools’ impact.

What we have learned from our RCTs

An uncomfortable result: In one of our RCTs, the tool we were evaluating showed no significant effects on learning at 6 months. We published the full results, including null effects. That honesty cost us a contract. It also won us the trust of the partners who matter most.

RCTs don’t always give the answers we want. But they always give honest answers. And in a field where press optimism is rarely backed by rigorous data, methodological honesty is a competitive advantage and, more importantly, an ethical obligation.

Our Evidence:

  • 4 RCTs completed or in progress.
  • 100% of results published, including nulls.
  • d=0.31 Average effect size in successful interventions.