A drive by download is a download that occurs without users action or knowledge. It usually triggers an exploit of vulnerability in a browser to downloads an unknown file. The malicious program in the downloaded file installs itself on the victims machine. Moreover, the downloaded file can be camouflaged as an installer that would further install malicious software. Drive by downloads is a very good example of the exponential increase in malicious activity over the Internet and how it affects the daily use of the web. In this paper, we try to address the problem caused by drive by downloads from different standpoints. We provide in depth understanding of the difficulties in dealing with drive by downloads and suggest appropriate solutions. We propose machine learning and feature selection solutions to remedy the the drive-by download problem. Experimental results reported 98.2% precision, 98.2% F-Measure and 97.2% ROC area.