Intrusion Detection Systems (IDS) are widely employed to detect and mitigate external network security events. Vehicle ad-hoc Networks (VANETs) continue to evolve, especially with developments related to Connected Autonomous Vehicles (CAVs). In this study, we explore the detection of cyber threats in vehicle networks through ensemble-based machine learning, to strengthen the performance of the learnt model compared to relying on a single model. We propose a model that uses Random Forest and CatBoost as our main investigators, with Logistic Regression used to then reason on their outputs to make a final decision. To further aid analysis, we use SHAP (SHapley Additive exPlanations) analysis to examine feature importance towards the final decision stage. We use the Vehicular Reference Misbehavior (VeReMi) dataset for our experimentation and observe that our approach improves classification accuracy, and results in fewer misclassifications compared to previous works. Overall, this layered approach to decision-making combining teamwork among models with an explainable view of why they act as they do can help to achieve a more reliable and easy-to-understand cyber security solution for smart transportation networks.