Academic Report Notice of Witold Pedrycz：Pursuing Green and Granular Machine Learning: Developments in Federated Learning, Knowledge Transfer, and Knowledge Distillation

  Green Machine Learning (also referred to as Green AI) has recently emerged as an interesting and application-oriented endeavour in the realm of intelligent systems. It stresses a genuine need for a holistic multicriteria assessment of the design practices of Machine Learning architectures and learning schemes by analyzing computing overhead (and associated carbon footprint), interpretability, and robustness, among others. Quite often in real-world environment, data can be available only locally coming with strict constraints imposed on their usage beyond individual data islands. Such restrictions constitute genuine conceptual and algorithmic challenges when it comes to solving problems of global data analysis and the development of global models. A lot of pursuits located in this realm fall under the umbrella of federated learning. We discuss a way of realizing learning in this environment and advocate that the resulting model is built at a higher level of abstraction formalized with the aid of information granule. Knowledge transfer is about a thoughtful and prudently arranged knowledge reuse to support energy-aware Machine Learning computing. Rather than starting from scratch, the existing experience (model) gathered in a source domain is transferred to the target domain. We discuss passive and active modes of knowledge transfer. In both modes, the essential role of information granularity is identified. The passive approach leads to the construction of a granular model in the target domain on a basis of the original model coming from the source domain where information granularity of the model serves as a vehicle to quantify the credibility of the transferred knowledge. In the active approach, a new model is constructed in the target domain whereas the design is guided by the loss function, which involves granular regularization produced by the granular model transferred from the source domain. A generalized scenario of multi-source domains is discussed. Knowledge distillation leading to model compression is studied in the context of transfer learning. We advocate that in order to conveniently address the quest of green machine learning, it becomes beneficial to engage the fundamental framework of Granular Computing. We demonstrate that various ways of conceptualization of information granules in terms of fuzzy sets, sets, rough sets, and others may lead to efficient solutions. To proceed with a detailed discussion, a concise information granules-oriented design of rule-based architectures is outlined. An information granules-oriented design of rule-based architectures in transfer learning and knowledge distillation is used for illustrative purposes. 

Personal Introduction:

  Witold Pedrycz (IEEE Life Fellow) is Professor in the Department of Electrical and Computer Engineering, University of Alberta, Edmonton, Canada. He is also with the Systems Research Institute of the Polish Academy of Sciences, Warsaw, Poland. Dr. Pedrycz is a foreign member of the Polish Academy of Sciences and a Fellow of the Royal Society of Canada. He is a recipient of  several awards including Norbert Wiener award from the IEEE Systems, Man, and Cybernetics Society, IEEE Canada Computer Engineering Medal, a Cajastur Prize for Soft Computing from the European Centre for Soft Computing, a Killam Prize, a Fuzzy Pioneer Award from the IEEE Computational Intelligence Society, and 2019 Meritorious Service Award from the IEEE Systems Man and Cybernetics Society. His main research directions involve Computational Intelligence, Granular Computing, and Machine Learning, among others. Professor Pedrycz serves as an Editor-in-Chief of Information Sciences, Editor-in-Chief of WIREs Data Mining and Knowledge Discovery (Wiley), and Co-editor-in-Chief of Int. J. of Granular Computing (Springer) and J. of Data Information and Management (Springer). 

[Editor: Xiaohan Liu]