Detecting atypical motion for early awareness of potential safety-critical situations

Kurzfassung

Atypical behavior, such as ignoring specified pathways, driving wiggly lines, abrupt stops or sudden direction changes, adopting shortcuts, etc., can increase the potential risk of collision and may lead to critical situations in road traffic. The capability of detecting such atypical behavior allows road users to be prepared and thus to increase their situation awareness and to decrease the risk of collision. In this paper an approach to detect atypical traffic situations of moving pedestrians not interacting with other road users within an urban environment is presented. The resulting abnormality model is created on the basis of data of normal motion behavior of pedestrians. Clearly, the underlying assumption is that by modelling and training of road user motion behavior typical behavior can be learned. Further, exceeding some deviation threshold suggests atypical behavior. For instance, an atypical point or part of a trajectory is called to be atypical, if its prediction is too far away from its expected point or part. For that we applied a recurrent neural network (RNN) to predict trajectories of pedestrians, which is known to be capable of learning temporal dynamics in time-series data. Although location, site-specific based a-priori knowledge (e.g. zebra crossing, traffic lights) is valuable for the task in question, we first concentrate on more generic variables like velocity and heading. The deviation between expected and predicted trajectory point or part is measured by Mean Squared Error (MSE). The threshold for an input sequence to be considered as anomaly is set to the 90th percentile of the distribution of MSE on training data. The algorithm was tested on video data at the DLR research intersection at Brunswick. The videos are recorded at 25 fps and by object detection and tracking 249 trajectories of pedestrians could be extracted from these data. It is planned to to use a month of data in order to get valid results. To find anomalies in the test set the sequences of 10 points observation length and 10 points prediction length and examined and each prediction compared with the ground truth. The resulting loss is compared to the anomaly threshold for a sequence at a trajectory time step, which is 0.081. As a next step to classify a trajectory as anomaly we can look at the percentage of points or sequences which are considered as abnormal of a whole trajectory. There are three trajectories with a percentage of over 50% classified as abnormal and clearly stand out of the set and are strong candidates for atypical behavior. In order to find the reason for the atypical behavior we examined the three trajectories visually on the video recordings, which will be presented in the full paper. It has been shown that it is possible to model the motion of pedestrians and to derive a measure to define abnormal behavior. Preliminary examination of the 249 trajectories showed the concept works in principle. Severe detection errors result in high abnormality values. The method to detect anomalies compares at each trajectory point the prediction accuracy with a threshold derived from training data. Analyzing whole trajectories this way results in an Anomaly percentage which tells what percentage of a trajectory is considered to be abnormal. The three trajectories with more than 50% abnormal are shown to be errors of the object detection system which is the basis for the data. Misclassifications of cyclists as pedestrians and positioning errors leading to abrupt velocity changes physically impossible were detected correctly as anomalies. More trajectories will be analyzed and the results presented in the full paper and severe object detection errors will be filtered out before analysis so real atypical behavior can be found. Moreover, some abnormal actions only last for a short time, e.g. an abrupt and strong braking maneuver may only take approximately 0.5-1s; in order to find such abnormal behavior it would be better to use sliding windows for anomaly detection instead of calculating the percentage of anomaly of the whole trajectory.