Machine learning (ML) in Internet of Vehicles (IoV) applications enhanced intelligent transportation, autonomous driving capabilities, and various connected services within a large, heterogeneous network. However, the increased connectivity and massive data exchange for ML applications introduce significant privacy challenges. Privacy-preserving machine learning (PPML) offers potential solutions to address these challenges by preserving privacy at various stages of the ML pipeline. Despite the rapid development of ML-based IoV applications and the growing data privacy concerns, there are limited comprehensive studies on the adoption of PPML within this domain. Therefore, this study provides a comprehensive review of the fundamentals, recent advancements, and the challenges of integrating PPML into IoV applications. We first review existing surveys of various PPML techniques and their integration into IoV across different scopes. We then categorize IoV applications into three key domains and analyze the privacy challenges in leveraging ML in these application domains. Building on these fundamentals, we review recent advancements in integrating various PPML techniques within IoV applications, discussing their frameworks, key features, and performance in terms of privacy, utility, and efficiency. Finally, we identify current challenges and propose future research directions to enhance privacy and reliability in IoV applications.