The post is by Ilan Reiter, Principal Data Science Manager at Microsoft.

The past few years have witnessed dramatic changes to the energy sector. Renewable energy sources along with the emergence of IoT (Internet of Things) are creating exciting new opportunities. On the consumption side, utilities and indeed the entire energy sector have seen consumption flatten out, with consumers demanding better ways to monitor and control their energy usage. Furthermore, with many grids becoming outdated and expensive to maintain, utilities and smart grid companies are in ever greater need to innovate.

In this environment, advanced analytics and cloud based solutions are in the front and center of the operations of smart grids.

Figure 1: Smart Grid

As seen in Figure 1, the control center of the Smart Grid oversees its operation while ensuring reliable supply of energy to all consumers. The control center makes extensive use of data that’s being collected from smart meters and sensors. This opens up numerous opportunities whereby advanced analytics can be used to predict the future state of the grid, thus offering better ways to manage the system and deliver early responses to potential issues.

Among the most fundamental data-driven use cases is energy demand forecasting (or load forecasting). By being able to predict future demand, utilities can plan ahead and respond to that predicted demand by purchasing or generating the required amount of energy, thereby avoiding energy wastage and controlling costs. Furthermore, energy demand forecast is driving other use cases such as demand-supply balancing, grid optimization, revenue protections and more.

Accurate and reliable forecasting is the key, and that’s an advanced analytics problem at its core. Technologies such as big data storage and compute (e.g. Hadoop) and machine learning techniques let us use historical consumption data along with other data sources such as weather reports to produce highly accurate energy demand forecasts.

To jump-start developers and data scientists on this problem, we have developed the Demand Forecasting for Energy solution template, which is implemented on the Microsoft Cortana Analytics Suite. The solution template implements an example of an end-to-end energy demand forecast along with its underlying data pipeline architecture and visualization of results. The template is easily replicated in its entirety under your own Azure subscription. You can quickly establish a baseline solution, inspect its elements and customize things for your own needs. The solution template also includes a playbook document and technical instructions to help shorten your learning curve.

During the deployment process, you will be able to see the underling architecture, its components and the progress of the deployment process on top of it.

Figure 2: Underling Solution Architecture

To get started and learn more, check out the solution template in the Cortana Analytics Gallery. We hope you make the most of the template, and please do share your feedback with us.

Ilan