We Teach People to Write Reviews, Not to Run Systems

When I say systematic reviews are misunderstood, I don’t mean people are lazy or incapable. I mean we’ve quietly trained them to look in the wrong direction.

In academia, the thing that gets rewarded is the final paper, the PDF, the journal name, and the citation count. That’s what gets graded, promoted, and funded. But the real work of a systematic review is in the decisions that led there.

It lives in the discipline of sticking to your protocol when you’re tempted to bend it. In keeping control of versions so you’re not quietly working from three slightly different documents. In those long conversations between two tired humans trying to agree on what “relevant” actually means. In the small, slightly uncomfortable judgement calls about borderline abstracts. In writing down why something was excluded, even when you suspect no one will ever read that note again.

That work is invisible. So students learn to “produce a review” rather than to design a review system. They celebrate conclusions, and optimise the document. And then they wonder why it feels so chaotic.

I see this over and over again in the people I work with (PhD students, research groups, pharma teams), all intelligent but stuck in the same fog. They think the problem is that they’re not working hard enough, which usually isn’t the case. It’s that they were never taught to build the decision environment properly in the first place.

Technique Isn’t the Same as Architecture

To be clear, most researchers are trained well in methods. They can recite PRISMA in their sleep. But they’re rarely trained in what I would call decision architecture.

What they’re rarely shown is how the weight of it builds over time. How making hundreds of small screening decisions gradually blurs your clarity. How uncertainty quietly compounds when criteria aren’t fully aligned at the start. How a simple spreadsheet can drift out of sync once a few different people have edited it at different moments.

They learn the methods. But not how to shape the environment those methods depend on. That distinction sounds subtle, but it changes a lot because a systematic review is a controlled decision system operating under uncertainty.

When the system hasn’t been designed properly, decisions start to feel heavier than they need to. Disagreements stop feeling like part of the process and start feeling personal. People begin to question themselves, because the structure isn’t holding them up. And when the structure isn’t doing its job, confidence naturally starts to slip.

The Familiarity Illusion

Part of the problem is that systematic reviews look familiar. On the surface, it’s reading papers, summarising them, and writing it up. So people assume it’s just a bigger literature review, but it isn’t.

In reality, a systematic review is about controlling bias, curating evidence carefully, managing how errors accumulate over time, and keeping your reasoning fully traceable under scrutiny, which makes it far closer to systems engineering than to writing.

That mismatch catches people off guard. They begin thinking it’s an academic writing task and realise halfway through that they’re actually running a complex decision process with hundreds of moving parts.

The discomfort that follows is usually mislabelling the task.

AI, Craft, and Identity

There is another layer to this that people do not always say out loud. When AI comes up in the context of systematic reviews, the visible conversation tends to focus on reliability, validation, bias, and whether the tools are mature enough. Those are reasonable concerns, and they should be discussed seriously.

But underneath that, there is often something more personal. For many researchers, a systematic review is a rite of passage. It is proof that you can endure complexity, manage uncertainty, and carry responsibility. It is something you survive, and that survival becomes part of your professional identity.

When parts of that process become automated, it can feel as though that identity is being diluted. If software helps screen abstracts or structure evidence tables, some people quietly wonder whether the intellectual weight of the work is being reduced. I do not believe it is, but I understand why it can feel that way.

When a task is tied to identity, any change to how it is done can feel like a threat. Not necessarily to quality, but to tradition and to ego. Academia, for all its strengths, is not immune to that dynamic.

What I have observed, however, is that the tone changes when AI is framed correctly. When it is positioned as an accelerator rather than a replacement, and when it is clear that humans still design the system, define the criteria, and make the final decisions, the tension eases. The craft does not disappear. It adapts.

The mindset shifts from enduring manual effort to designing a stronger process. In my view, that is a healthier posture.

The Part We Don’t Like to Admit

There is a deeper reason systematic reviews feel uncomfortable.

They force us to confront how fragile evidence really is. Individual studies conflict with each other. Methods vary more than we would like to admit. Bias is rarely absent. Results shift depending on definitions, thresholds, and interpretation. Two reasonable reviewers can read the same abstract and reach different conclusions without either of them being careless.

Systematic reviews exist precisely because knowledge is messy. That is not a comforting realisation. It challenges the idea that science moves forward in neat, linear steps. It reminds us that much of what we call “evidence” is conditional, contextual, and open to revision.

It is far easier to treat a systematic review as an extended literature summary, something that gathers papers together and produces a tidy conclusion at the end. That framing feels stable. It suggests that if we just work hard enough, we will arrive at a clean answer.

In reality, a systematic review is an exercise in managing uncertainty in a disciplined way. You are not removing doubt; you are containing it. You are documenting it. You are making your reasoning visible so that someone else can examine it and, if needed, challenge it.

That process is psychologically demanding. It requires acknowledging that evidence is not a single solid block but a shifting landscape of probabilities and limitations. Your role is not to declare certainty but to map that landscape carefully enough that another person can follow the path you took and understand why you stepped where you did.

There is nothing glamorous about that kind of work. It does not lend itself to dramatic conclusions or heroic narratives. But it is rigorous, and it is honest.

From Chaos to Clarity

When I talk about helping people move from chaos to clarity, I am talking about a shift in posture.

When the structure is solid, the language inside a team changes. Instead of someone saying, “I think this one probably fits,” you hear, “According to our criteria, this belongs here.” Instead of, “Have we screened this already?” it becomes, “It’s logged, check the latest version.” The tone becomes calmer. There is less guessing and less doubt.

The anxiety does not disappear entirely, but it changes shape. Rather than worrying whether the review would survive scrutiny, there is a steadier confidence that the decisions are traceable because the reasoning is visible.

The work itself is still demanding; it requires judgement, time and focus. But it becomes contained within a clear structure, and when something is properly contained, it feels manageable.

To me, that is where the misunderstanding lies. Systematic reviews are not complicated in the way people assume. They feel overwhelming because they are framed as writing tasks or statistical exercises, when in reality they are decision systems operating under uncertainty.

Once you see them through that lens, the discomfort starts to make sense. You realise it was never about a lack of ability; it was about a lack of framing. And when something finally makes sense, it becomes less intimidating. Sometimes that shift alone is enough to change how the entire process feels.