Gathering Up-To-Date and Reliable Fundamental Data is a Complex Task

Gathering Up-To-Date and Reliable Fundamental Data is a Complex Task
Power market fundamental analysis requires accurate and consolidated timely data. Gathering such data may seem very straightforward, particularly with the growing amount of transparency platforms. However, experience shows that obtaining accurate and reliable data is complex, and automated data collection needs to be supported by strong data expertise. 
Power demand is one of the most important drivers of power markets. In this blog, we examine this specific data type to illustrate the traps and pitfalls of power market data collection.

The Obvious Source: Gathering Data from the TSO Website

Most TSOs have data portals where you can find power market data, although not all of them offer capabilities to easily download actual and historical data. 
This blog focuses on the example of Portuguese data. The Portuguese TSO REN does not have an extensive data portal that allows users to easily access up-to-date recent historical hourly demand data. Only hourly demand for the current day is available, published in real-time as a chart. An example is shown below in Figure 1. Note that the demand level at 11 am (local time) reaches 6215 MW.
electric energy consumption

Gathering Data from ENTSOE's Transparency Platform

Given the difficulty of accessing data via REN’s website, the logical next step is to explore ENTSOE’s transparency platform. This offers some download capabilities, and provides historical and actual demand data in almost real-time. 
Portuguese hourly demand from entso-e transparency platform

In terms of shape, the ENTSO-E data is consistent with the data from REN. In particular, the morning peak for 11 May 2017, takes place at 12 am CET/CEST, i.e. 11 am Portugal local time.

However, it is very interesting to notice that the demand levels published by REN and ENTSO-E differ: 6215 MW vs. 6358 MW, respectively, for the same hour. This 2.3 percent difference is significant and needs to be understood before we can determine which data is most accurate for fundamental analysis.

Differences in data value levels between sources can generally be explained by the definition and the scope of the data. In particular, for demand data this relates to the manner in which aspects like pump consumption and losses are treated. In this specific case, the reconciliation is difficult, as REN does not provide clear data definition information. 

Can We Rely On "Real-Time" Published Data?

In addition to its transparency platform, ENTSO-E has a second data platform which provides consolidated (and hence delayed) hourly demand data as well as some monthly statistics. In total, ENTSO-E demand data can be found from three different places (the letters A, B, C will be used in the following tables/charts):
  • A. ENTSO-E Transparency: Hourly Actual Total Load (almost real-time data)
  • B. Monthly Hourly Load (few months lag in publication)
  • C. Monthly Consumption (few months lag in publication)
The power demand data for Germany, available on the different ENTSO-E platforms, clearly illustrates the issues raised above and differs greatly:
2016 monthly German demand levels from different ENTSO-E portals
In the case of Portugal, the data is much more consistent (on average): 
2016 monthly Portuguese demand levels from different ENTSO-E portals
These examples clearly show that an in-depth analysis of a data scope is needed before directly using the data provided by a source. 

Time-Zone Definition and Hourly Shape

Data scope inconsistencies are not the only difficulty that analysts face when they deal with demand data. Time-zone definition is also a concern, as even a one-hour offset can have a very strong impact on analysis and price forecasts. 
A comparison of Portuguese demand data for the second Thursday of May shows that the hourly curves reported on the website (source B) and the ENTSO-E Transparency Platform (source A) are relatively consistent both in terms of level and hourly shapes. Both curves present the same profile with peaks at H12. In addition, both sources clearly indicate that they report data in CET/CEST. Evening peaks are more concerning as a one-hour offset seems to be visible between the two ENTSO-E platforms.
PT hourly demand data

Data inconsistencies become clearer when we have a closer look at the same sources but for a “winter” day (last Thursday of January). 

PT hourly demand data

In Figure 6, we see that the demand data reported on for Portugal for the last Thursday of January in 2016 and 2017 peaks one hour earlier than the same data (same days), but from the ENTSO-E Transparency platform. Furthermore, even historical data from appears to be inconsistent across the years. A comparison with the last Thursdays of January during 2014 and 2015 from the same platform (ie. using the same time-zone definition) also reveal a similar one-hour offset.

This discrepancy does not appear for all countries and does not follow any obvious time-zone logic - markets in CET are also affected. This illustrates the need for an in-depth analysis and full crosscheck for each individual source to be able to gather reliable demand data.


Genscape’s EPSI Research Team strives to understand and consolidate all of the different available data in order to provide reliable fundamental datasets in the EPSI Platform. Right after the publication of the statistics, Genscape’s teams work on the collection, the analysis, and the correction of the data to offer the most reliable market data in the EPSI Platform. These datasets are perfect to support your mid-to-long term analyses or to benchmark your own data as they ensure completeness and accurate coverage. For those who need short-term demand data (actuals and day/week-ahead forecasts), Genscape partnered with specialists such as TESLA and PRT, who provide reliable demand data for European markets directly usable in the EPSI Platform. To learn more about the EPSI Platform, please click here.