Evaluation von Embeddings aus Foundation Modellen zur Ausreißererkennung aus multivariaten tabellarischen Daten

Kurzfassung

Tabular data is the most common data format in practice, and real world tables combine numeric attributes with free text columns that are usually discarded during preprocessing. Recent work has shown that embedding these columnsimproves predictive performance for Tabular Foundation Models (TFMs), but that evidence is restricted to supervised classification and regression. This work asks whether the same holds for unsupervised anomaly detection, where no label guides the model towards the relevant part of the representation. Two datasets that differ in their share of free text are used, and an attribution analysis first establishes that the free text columns carry information relevant to the prediction. Across three TFMs, hyperparameter optimised baseline detectors and two families of text representations, no configuration achieves a consistent advantage. A FastText representation improves the detection on the text rich dataset, but the benefit does not transfer to the more numeric one, so in contrast to the supervised case no representation strategy makes the free text usable for unsupervised detection in general. A supervised control shows that the same models and representations perform well once labels are available. The obstacle therefore lies not in the quality of the learned representations but in their transfer into an unsupervised detection procedure.