The design of existing techniques for active learning does not take into account the incremental nature of the task. Ensemble filters on the other hand do, by utilizing Bayes' rule, but they are more concerned with close approximation of the posterior distribution and do not offer good estimation of variance which is needed for state space exploration. Based on Kalman filter, we propose Deep ensemble filter (DEnFi). Key idea of DEnFi is to evolve an ensemble of neural networks by iterating two steps: inflation and localization. On examples from Bayesian optimization and Active classification we show the superiority of DEnFi in regards to finding the true minimum and to provide good classification with correct uncertainty estimation, respectively.