Statistical Approaches to Machine Learning in a Big Data Environment

Abstract

This thesis explores statistical approaches to machine learning in a big data environment. Firstly, the connections and differences between machine learning and statistics in a big data environment are introduced, as well as the statistical foundations in machine learning models. Secondly, the application of statistical methods in big data analysis is discussed, including the combination of traditional data analysis and machine learning. Then, the challenges and limitations of statistical methods in the big data environment, such as high dimensionality and huge amount of data, are discussed. Then, common statistical methods in the big data environment, including linear regression, decision trees, and support vector machines, are described in detail. Finally, the research findings are summarised and future directions and research trends are outlined. Through the research in this paper, a deeper understanding of statistical methods for machine learning in big data environment is provided, which provides an important reference for big data analysis and application.