Convolutional neural networks (CNNs) are known for their good performance and generalization in vision-related tasks and have become state-of-the-art in both application and research-based domains. However, just like other neural network models, they suffer from a susceptibility to noise and adversarial attacks. An adversarial defence aims at reducing a neural network's susceptibility to adversarial attacks through learning or architectural modifications. We propose the weight map layer (WM) as a generic architectural addition to CNNs and show that it can increase their robustness to noise and adversarial attacks. We further explain that the enhanced robustness of the two WM variants results from the adaptive activation-variance amplification exhibited by the layer. We show that the WM layer can be integrated into scaled up models to increase their noise and adversarial attack robustness, while achieving comparable accuracy levels across different datasets.