Automatic Feature Engineering Using Self-Organizing Maps

Silva Rodrigues E, Martins DML, Lima Neto FB

Abstract

Feature Engineering (FE) consists of generating new, better features to improve the results obtained by Machine Learning models. Very often, FE is performed in a series of trial-and-error steps conducted manually by data scientists. Moreover, FE requires data-specific and domain knowledge, both rarely easy to acquire. To alleviate these problems, we propose an automatic FE approach based on Self-Organizing Maps (SOM) in which new features are generated via pattern recognition. The use of the SOM algorithm in variable generation tasks can identify data elements that help Machine Learning models to obtain better results and points out to a broad direction for future researches.