Demand Prediction with Machine Learning Models; State of the Art and a Systematic Review of Advances

Electricity demand prediction is vital for energy production management and proper exploitation of the present resources. Recently, several novel machine learning (ML) models have been employed for electricity demand prediction to estimate the future prospects of the energy requirements. The main objective of this study is to review the various ML models applied for electricity demand prediction. Through a novel search and taxonomy, the most relevant original research articles in the field are identified and further classified according to the ML modeling technique, perdition type, and the application area. A comprehensive review of the literature identifies the major ML models, their applications and a discussion on the evaluation of their performance. This paper further makes a discussion on the trend and the performance of the ML models. As the result, this research reports an outstanding rise in the accuracy, robustness, precision and the generalization ability of the prediction models using the hybrid and ensemble ML algorithms.


Introduction
Electrical energy is an essential element for the sustainable development of today's nations in economic, environment and social aspects. The global energy consumption has been ever increasing exponentially. Therefore, implementing energy management can be a major step in the progress of economic development and environmental security. As the electrical energy cannot be stored, managing an efficient balance between electricity demand and generation is crucial (Jebaraj and Iniyan 2006). Electricity demand forecasting aims at predicting the precise amount of this kind of energy Debnath and Mourshed 2018). Both under-and over-estimating can have very costly consequences. The high operating cost of the network, excess supply and network balance problems are examples of overestimation whereas failure in delivering enough electrical energy is the most important issue of underestimation (Palensky and Dietrich 2011).
In general, electricity demand is generated as a quantity of electricity and distributed in a specific area over a given period (Engle, Mustafa et al. 1992). Electricity demand forecasting is considered as one of the most important areas in the research in the electric power industry due to its decision maker role in the management of power grid in response to changes in the consumption of subscribers. It is also attractive for companies related to the fields of energy generation, transmission, and marketing. Most importantly, the nation's gross national income, values by the methods and n is the numbers of data.
Classification ML employs advanced statistical techniques to able the computer systems to learn based on data, without specific programming. One of the applications of ML, these days, is predicting electricity demand. There is a number of different methods which are used for this purpose.

Single methods
Future prediction can be developed in the presence of one single ML algorithm. In the literature, some of the single methods like KNN, SVM, ANN, DT, and other ML methods have been used to overcome these problems. KNN is one of the most simple and traditional nonparametric techniques to classify sample (Tsai, Hsu et al. 2009). SVM which is proposed by Vapnik (1998), first maps the input vector into a higher dimensional feature space and then obtain the optimal separating hyperplane in the higher dimensional feature space. The ANN is information processing units which to mimic the neurons of the human brain (Haykin 1994). Table 1 presents the latest research work has been done for electricity demand prediction with single ML methods. The first columns show the reference, while the second columns give us information about the model which is used for prediction. In some cases just a single method used for this purpose, in some others one method has been compared with some other methods to give credit for the one which outperforms the other like (Ali and Azad 2013).  research, is for the time from 1990 to 2007 in Italy. By doing this research, the writers report that they could find a relationship between temperature and electricity demand and as a result, they claim that the anomaly of the electricity prediction in that region is because of heat-waves over Europe and for this purpose SVM is generally better than the linear model. (Zjavka 2015) use a differential polynomial neural network to predict electricity demand for a short period. (Zahedi, Azizi et al. 2013)   short-term which can predict 1 to 10 days ahead. The data that is used in this work is for 22 months, which contains daily data from 1 January 1997 to 31 October 1998 for estimating the parameters of the model and from 1 November 1998 to 30 April 2000 to test the methods. In a conclusion, the writers claim that there is a potential use for weather ensemble prediction to improve the accuracy and uncertainty assessment of electricity demand prediction.

Hybrid methods
Hybrid methods are a combination of completely different techniques to increase the performance (Tsai, 2010). This kind of methods generally consists of two functional components. The first one takes raw data as input and generates intermediate results.
The second one will then take the intermediate results as the input and produce the final results. The following table also has the same structure as the previous ones. It contains information about the references which uses hybrid prediction modeling for electricity demand. They have to combine different techniques to improve the performance of the single methods. This hybrid algorithm is used for improving the annual electricity demand prediction in India. The historical data, which is used in this paper, is from 25 years from 1991 to 2015. COGSA is a method that helps to optimize the forecasting procedure. First, CO algorithm tries to search the global search space to find global optima, then GSA as an algorithm that search the local, tries to fine some better solutions near to the optima which has been found by COA. This process is used to reach the benefits of exploiting Pre-processing of data in an integral part of data mining. It is done to clear the initial data from the noise, therefore the result data is efficient for data modelling. For example Fourier transform can be used in this step. However this method has some drawbacks like phase shift. WT can overcome these disadvantageous by providing better temporal resolution for components which have low frequency. Also it provides better frequency resolution for low frequency components at the same time. MWD is a way of analysing that considers both time and frequency domain. In general, a series of wavelets are derived from a mother wavelet by displacement and scaling on time shaft and shift translations. The role of MWD is to find how and how much inevitably random sections in the dataset can be reduced. (Chen and Tan 2017) propose a hybrid model based on SVR and multi-resolution WD for predicting hourly electrical demand prediction in the building sector. In this research work, WD is used as a pre-processing step. The data, which is used in this paper, is collected from an electric consumption of a mall and a hotel in China. The writers report that introducing WD, the prediction accuracy can always be improved for the hotel, and it is not necessary for the mall.
ANN can also be combined with MLRs in order to make a hybrid method to produce better results. It this combination, a MLR is used to be applied on the whole data to specify that a specific descriptor variable is appropriate or not. In this combination, then the ANN is used to construct a prediction model on the variables which have been already specified by the MLR. The hybrid model can be used in electricity demand prediction as it is use by (Günay 2016). In this research work, the writers show that how they combine ANN and MLR to predict annual electricity demand in Turkey.
The data, which is used for the work is for the years between 1975 and 2013. The writer concludes that the presented approach can be used by other countries to make a precise prediction for the future.
One of the main reason for the weakness in the accuracy of the electricity the paper reports that MFES improves the prediction of the electricity demand significantly. (Wang, Wang et al. 2010) propose an improved version of BP wavelet neural network (IBPWNN) to forecast electrical energy demand. In this study, the annual power demand of Taiwan between the years 1985 and 2000 is used. This data is divided to train and test data, which the data of the years 1985 to 1996 is used as training data and the rest as the text data. It is reported by this paper that IBPWNN has high precision and a good ability for estimating electricity demand.

Discussions
Here come the discussions on the ML methods used and on the outstanding rise in the accuracy, robustness, precision and the generalization ability of the prediction models using the hybrid and ensemble ML algorithms. This section provides the graphical results related to the studies presented in the above sections. These results provide a better comparison capability as well as a better understand about the performance of each method. Figures 1 and 2 presents the results in term of MAPE and RMSE for the highlighted studies, respectively. In this way, the method with a low MAPE provides a better performance compared with other methods.

Conclusion
Energy demand prediction models using ML methods have been reviewed. It is found that today, various nations are investing in the advancement of prediction models of ML for a detailed energy planning for their sustained development. Such models are formed using hybrid and ensemble ML techniques. ANNs, SVRs, Neuro-fuzzy in an integration with SC techniques yielded better prediction performance.