文件名称：Generalization in Machine Learning via Analytical Learning Theory.pdf
文件大小：308KB
文件格式：PDF
更新时间：2023-04-19 04:30:10
DL
This paper introduces a novel measure-theoreticlearning theory to analyze generalization behaviors of practical interest. The proposed learningtheory has the following abilities: 1) to utilizethe qualities of each learned representation onthe path from raw inputs to outputs in representation learning, 2) to guarantee good generalization errors possibly with arbitrarily rich hypothesis spaces (e.g., arbitrarily large capacity andRademacher complexity) and non-stable/nonrobust learning algorithms, and 3) to clearly distinguish each individual problem instance fromeach other. Our generalization bounds are relative to a representation of the data, and hold true even if the representation is learned. We discuss several consequences of our results on deep learning, one-shot learning and curriculum learning. Unlike statistical learning theory, the proposed learning theory analyzes each probleminstance individually via measure theory, rather than a set of problem instances via statistics. Because of the differences in the assumptions and the objectives, the proposed learning theory is meant to be complementary to previous learning theory and is not designed to compete with it