2026 Young Science Writer Award winner Adrian Dredge explores the hidden biases shaping AI

The Digital Mapmaker: Navigating the Future with the Biases of the Past

A famous mapmaker is tasked with drawing a map of a new city. They are locked in a windowless room and cannot step outside. However, they are only given maps that are fifty years old. They have never been to this city. They have no concept of what exists beyond their four walls. Now imagine the mapmaker has created their new map of the city and a person follows this map. That person finds themselves stopping for walls that were demolished years ago, trying to follow roads that have long since been removed. The map is completely unreliable and there is nothing new about it. The creator’s only recourse was to remake the old maps. AI is the same; it is like the mapmaker trapped in a windowless room and has no independent perspective.

People train AI systems to be the most optimised possible using data given by humans. This often leads to pattern recognition and self-fulfilling prophecies. If the mapmaker were given a ledger of crimes and were told to sketch where they thought the next incident might occur, and they shaded specific areas on the map to create target zones, and the police were to patrol them, it would follow that a higher volume of crime would be recorded in these areas because the police would be focused on specific areas. Any areas outside of these target zones would have lower recorded crime rates as there would be no one to catch the criminals. 

A crime list is not a list of crimes; it is a list of where police have been told to stand, catching criminals in the act. This makes it a record of intervention, not occurrence. Crucially, this is not hypothetical; it is reality. As Sarah Brayne said in her research article, Big data surveillance: the case of policing (2017), 

“...data are used for predictive, rather than reactive or explanatory, purposes.” 

Overhead view of a street intersection in Seattle’s South Lake Union neighbourhood, showing surrounding buildings, roads and pedestrian crossings. Photo: Zoshua Colah on Unsplash.

As Richardson, Schultz, and Crawford (2019) argue, many predictive policing systems are trained on 'dirty data'. Dirty data is data that is fed to the AI, which has a bias attached to it. This is data that is collected from police departments that have a history of racism or bias. When this biased and ‘malignant’ data enters, the system does not see it as a biased piece of data but rather as a pattern that needs to be replicated.

It is like the mapmaker has a compass that only directs to where crime has happened. The mapmaker thinks they have found a groundbreaking crime zone, when in fact all they have done is retrace the footprints of their predecessors. This outputting of data becoming the input of the new data creates a dangerous feedback loop that must be cut off. As it is impossible to ‘clean’ all of the data we have collected over the last century, we must conclude that as long as AI relies on biased data in the police system, it can never be fair.

As AI can only see the world through the data it is given, it can often make consequential errors that can harm people. For example, a study that was published in Science discovered how the healthcare industry would use financial data as a ‘proxy’ for health (Obermeyer et al., 2019). By using the most convenient proxy for the company (financial details), it unknowingly causes Black patients to be discriminated against. For example, in chronic conditions, Black patients would only receive 17.7% additional help; comparatively, White patients would receive 46.5% additional help (Obermeyer et al., 2019). This arose due to the historical economic disparity, which led to Black patients spending less money on healthcare even if they were very ill. This is as if the mapmaker mistook a wealth landmark for a health landmark, accidentally mislabelling the entire city.

Photo caption: CCTV cameras at Charing Cross station, London. Photo: Steph Gray / Unsplash.

Suppose the mapmaker is tasked with splitting up the city and labelling certain areas as ‘safe’ or ‘hazardous’ without ever stepping outside to walk them. The only way to know is what someone tells them about these areas. If that person holds a grudge against a specific neighbourhood, the mapmaker unknowingly weaves that bias into the map. AI operates in that same windowless state. AI does not possess a biological consciousness; it has no personality or beliefs. Its training parameters dictate its only morals. Thus, there will always have to be a ‘human in the loop’. The AI can never be impartial if a human is deciding what a suspicious person looks like on a security camera feed, or what a risky loan is for a bank. Each person has their own cultural, educational, and political biases that are inevitably woven into the AI training. If the AI is always given arguably biased data, then we can draw the conclusion that the AI is inherently biased and partial. Raw data cannot exist because in order for the AI to know what is ‘right’ and ‘wrong’, it needs to be told. Just as the mapmaker has no idea whether what they are labelling as safe is in fact safe, neither does the AI.

Much like a mapmaker retracing their lines in ink, we are trapped in a loop of our own making, being blind to the past, unable to catch up to the truth.