An AI that reads chest X-rays can ace its tests and still miss the patients who need it most. The problem is rarely the AI itself. It is almost always something that happened to the data before the AI ever saw it.

To see how, let's follow one chest X-ray from a hospital's archive to the moment it is used to teach an AI. Along the way it passes six checkpoints. At each one there is a quiet way to get it wrong, and a careful way to get it right.

01 · Leaving the hospital: remove the patient, keep the medicine

A medical scan file is not just a picture. It is closer to a patient folder with a picture inside. The patient's name, hospital number and visit dates are stored in the file's hidden details. Some scans have names or dates printed directly onto the image. A head scan can even be turned into a 3D model of the person's face.

Where it goes wrong: deleting the obvious name fields and assuming the file is now anonymous.

What safe looks like: removing identity from both the hidden details and the image itself, covering faces on head scans, and swapping real patient numbers for consistent code numbers, so one person's scans can still be grouped together without revealing who they are. A person double-checks a sample, and every change is recorded.

02 · Crossing borders: three sets of rules, one file

That one X-ray now falls under the rules of three places: where the patient lives, where the AI will be sold, and where the labelling happens. In the US, any company handling patient data for a hospital must sign a legal agreement called a Business Associate Agreement, and that duty passes down to every subcontractor. In the EU, data where names have been swapped for codes still counts as personal data, and health data gets extra protection. Europe's AI law treats most medical AI as high risk and expects the labelling work itself to be controlled and documented. Kenya's data protection law also treats health data as sensitive.

Where it goes wrong: assuming "anonymous" means the same thing everywhere. Data that counts as anonymous under US rules can still count as personal data in Europe.

What safe looks like: checking which rules apply before any data moves, letting only the people who need a file see it, blocking downloads, logging every view, and having security independently audited.

03 · The label: even experts disagree

Now someone marks what is in the scan. This is where medicine differs from most labelling work: the experts themselves do not always agree. In one study, six radiologists (doctors who specialise in reading scans) labelled the same 100 chest X-rays, then did it again later. On some labels they agreed almost perfectly. On others, agreement was only fair. Simple descriptions ("there is a shadow in the lower left lung") stayed consistent. Diagnoses ("this is pneumonia") did not, especially among less experienced doctors. Some even disagreed with their own earlier answers.

Where it goes wrong: one person labels each scan, or a team simply goes with the most popular answer, turning real medical uncertainty into a label that sounds certain.

What safe looks like: instructions that describe what can be seen, not what it might mean; each task matched to the right level of expertise; several people labelling the riskiest cases separately; a senior doctor settling disagreements; and a record kept of where people disagreed, because those are the hardest cases. Where firmer proof exists, such as a biopsy or lab result, it beats any opinion. We explain why agreement scores alone can mislead in Your agreement score cannot see half your errors.

04 · The shortcut: what else is in the picture

AI learns the easiest pattern, not necessarily the right one. Models trained to find a collapsed lung on X-rays learned to look for a chest drain instead. A chest drain is a tube doctors insert after they have already found and started treating a collapsed lung. So the AI looked brilliant in testing but did worse on patients without a drain, the untreated patients who need it most. A later study found this same tube shortcut explained why such models seemed to perform differently for men and women.

Where it goes wrong: labelling only the disease, so nobody can later check what the AI is really looking at.

What safe looks like: labellers also mark the other things in the picture that could fool an AI: tubes, wires, rulers, how the patient was positioned, the type of machine. With those notes, a team can test the AI separately on scans with and without each one, and catch the shortcut before patients do.

05 · The people: who is missing from the data

When researchers tested leading skin-condition AIs on images covering a wide range of skin tones, with every diagnosis confirmed by a biopsy, their performance score fell by 27 to 36 percent. They did worst on dark skin and rare conditions. The dermatologists who usually label this kind of data also did worse on those images. When the AIs were retrained on more varied images, the skin tone gap closed.

Where it goes wrong: checking the finished AI for bias, when the bias was built in by the data.

What safe looks like: checking who is represented before labelling starts, recording the details that matter (skin tone, hospital, scanner type, age group), and using confirmed results as the answer key wherever they exist.

06 · The exam: keep the test honest

Before an AI is used, it is tested on scans it has never seen. US, Canadian and UK regulators have jointly said that this test data should be truly separate from the training data. If the same patient's X-rays end up in both, the AI is partly being tested on someone it already knows, and its scores look better than they really are.

Where it goes wrong: splitting the data scan by scan instead of patient by patient.

What safe looks like: using those consistent code numbers from checkpoint 01 to keep each patient's scans together, and, when the goal is to prove the AI works anywhere, testing it on a hospital it never trained on.

What comes out the other end

Our X-ray reaches the AI with no trace of who the patient was, a label experts can stand behind, a note about the tube in the corner, details showing who it represents, and a full record of who handled it and why. None of these steps is exotic. What separates safe from unsafe is doing all six, on every file, every time.

Six questions to ask a medical data partner

  1. How do you remove names and dates printed on the image itself?
  2. Which data laws cover our project in your country, and how are your staff bound by them?
  3. How many people label each risky case, and who settles disagreements?
  4. Do your labellers note tubes, wires and other things that could fool the AI?
  5. Will we see where your experts disagreed, or only the final answer?
  6. Can you show the full history of any single label if asked?

These are the questions we build medical projects around at Impact Outsourcing: ISO 27001 certified security, a named delivery lead on every project, and a Nairobi delivery floor that knows its obligations.

Want to see it on your own data? Send us a sample batch. We will send it back with the agreement score and the cases our team disagreed on.

Frequently asked questions

What is medical data annotation?

It is the work of marking up medical data, such as X-rays, CT scans, skin photos or clinical notes, so an AI can learn from it. Annotators outline findings, tag conditions and record details about each image, under clinical guidelines and expert review.

Is removing the patient's name enough to make a scan anonymous?

No. Scan files also store hospital numbers and dates in hidden fields, some images have details printed on the picture itself, and head scans can be rebuilt into a face. Data that counts as de-identified under US rules can also still count as personal data under European law.

Why do medical labels need more than one expert?

Because trained specialists often disagree on the same image, and sometimes with their own earlier reads. Several independent reads on risky cases, a senior reviewer to settle disagreements, and confirmed results such as biopsies where they exist give an AI a far more reliable answer key.

Sources

  • Li et al., Inter- and intra-observer agreement when annotating findings on chest X-rays, Diagnostics, 2022. PubMed
  • Seah, Oakden-Rayner et al., Hidden stratification in pneumothorax detection on chest radiography, BMJ Open, 2021. BMJ Open
  • Olesen et al., Slicing through bias: explaining performance gaps in medical image analysis, 2024. arXiv
  • Daneshjou et al., Disparities in dermatology AI performance on a diverse, curated clinical image set, Science Advances, 2022. PMC
  • FDA, Health Canada and MHRA, Good Machine Learning Practice for Medical Device Development: Guiding Principles, 2021. FDA
  • Office of the Data Protection Commissioner (Kenya), Guidance Note on Cross-Border Data Transfers, 2026. ODPC