Crypto-ransomware is characterized by its irreversible effect even after the detection and removal. As such, the early detection is crucial to protect user data and files of being held to ransom. Several solutions have proposed utilizing the data extracted during the initial phases of the attacks before the encryption takes place. However, the lack of enough data at the early phases of the attack along with high dimensional features space renders the model prone to overfitting which decreases its detection accuracy. To this end, this paper proposed a novel redundancy coefficient gradual up-weighting approach that was incorporated to the calculation of redundancy term of mutual information to improve the feature selection process and enhance the accuracy of the detection model. Several machine learning classifiers were used to evaluate the detection performance of the proposed techniques. The experimental results show that the accuracy of proposed techniques achieved higher detection accuracy. Those results demonstrate the efficacy of the proposed techniques for the early detection tasks.