Recently, increasing interest has been shown in applying the concept of Pareto-optimality to machine learning, particularly inspired by the successful developments in evolutionary multi-objective optimization. It has been demonstrated that the multi-objective approach to machine learning is particularly effective in improving the performance of traditional single-objective machine learning methods, generating highly diverse multiple Pareto-optimal models for constructing ensemble models, and achieving a desired trade-off between accuracy and interpretability of neural networks or fuzzy systems. This monograph presents a selected collection of research work on the multi-objective approach to machine learning, including multi-objective feature selection, multi-objective model selection in training multi-layer perceptrons, radial basis function networks, support vector machines, and decision trees.