Breiman’s Random Forest (RF)[1] algorithm remains as one of the most popular algorithms in Machine Learning. It’s competitive performance, accuracy, scalability and robustness in real-world classification and regression problems made it very popular. There are lots of variant implementation of RF is now available such as Decision Forest, Decision Jungle [2] etc.. Some of the current Machine Learning use cases demands effective online algorithms, which can bring the power of ensemble learning algorithms such as RF. Many researchers proposed online learning system by combining RF. One of the most recent advances in this area is Mondrian Forests[3][4]. The seminal paper was authored by Lakshminarayan et.all. They combine the power of RF with properties of Mondrian Process [5]. The key attraction to this paper is clean reproducibility and availability of a Python implementation [6] by the authors.

The papers are worth reading along with experimenting with the package and some data. Happy Machine Learning Hacking !!!

References

[1] L. Breiman. Random forests. Mach. Learn., 45(1):5–32, 2001.

[2] Jamie Shotton, Toby Sharp, Pushmeet Kohli, Sebastian Nowozin, John Winn, Antonio Criminisi, Decision Jungles: Compact and Rich Models for Classification, Proc. NIPS,2013.

[3] Balaji Lakshminarayanan, Daniel M. Roy, Yee Whye Teh, Advances in Neural Information Processing Systems (NIPS), 2014.

[4] Mondrian Forests for Large-Scale Regression when Uncertainty Matters, Balaji Lakshminarayanan, Daniel M. Roy, Yee Whye Teh Proceedings of AISTATS, 2016.

[5] D. M. Roy and Y. W. Teh. The Mondrian process. In Adv. Neural Inform. Proc. Syst. (NIPS), Volume 21, pages 27–36, 2009.

[6] Mondrian Forests,https://github.com/balajiln/mondrianforest, Last Accessed 01/19/2017.