The increasing connectivity of data and cyber-physical systems has resulted in a growing number of cyber-attacks. Real-time detection of such attacks, through the identification of anomalous activity, is required so that mitigation and contingent actions can be effectively and rapidly deployed. We propose a new approach for aggregating unsupervised anomaly detection algorithms and incorporating feedback when it becomes available. We apply this approach to open-source real datasets and show that both aggregating models, which we call experts, and incorporating feedback significantly improve the performance. An important property of the proposed approaches is their theoretical guarantees that they perform close to the best superexpert, which can switch between the best performing experts, in terms of the cumulative average losses.