Bias Detection in an AI Training Dataset for Automated Vehicles

Kurzfassung

A high-quality data set has to be unbiased, which in the context of automated vehicles in real-life traffic outside of simulation or training grounds means that the images not only have to contain all kinds of traffic signs, obstacles, and road types, but also need to contain them under various operational conditions, such as weather conditions, lighting conditions, backgrounds, etc. One challenge is to think of every possible operational condition in which the AI might be used. Also, there have to be sufficient representations of every obstacle, traffic sign, and other road user that the automated vehicle might encounter. For that and the correct annotation, human intervention will always be needed. Confirming a sufficient number of representations of each object for the AI to learn to detect it is straightforward. Confirming that two attributes which should be independent are represented independently, for example, whether stop signs are equally distributed over images taken by day and night, is tedious, but doable, but with more possible combinations, the task grows exponentially. I performed research on algorithms to test data sets for possible bias, evaluated whether they are suited or adaptable for this problem, and finally found one that I implemented.