Improving the Generalization Capabilities of Machine Learning by Set-Based Reasoning

Kurzfassung

By design, most neural networks only consider a single sample in the data space for each forward pass. Generalization over all inputs is achieved by training on sufficiently large sets of such data points. Recent work in this area has shifted the focus from a point-based approach to set-based reasoning in the form of interval arithmetic in interval neural networks (INNs). To generalize data points to intervals, this work makes use of domain-specific, transitive "generalization" functions that aggregate axis-aligned regions in the input space. These intervals are fed into an interval neural network (INN) and an SMT solver (Z3) for performance comparison. The benefits of generalization functions are analyzed against three datasets, most notably FashionMNIST, under various configurations. Overall, results show that generalization functions lead to better generalization ability. Furthermore, neural models consistently outperform SMT solvers for the majority of configurations, especially when factoring in aspects such as scalability.