The increasing capacities of renewable energy sources and the opportunities emerging from the smart grid technology lead to new challenges for energy forecasters. Energy output fluctuates stronger compared to conventional power production. More time series data is available through the usage of sensor technology. New supply forecasting approaches are developed to better address those characteristics, but meaningful benchmarks of such solutions are rare. Conducting detailed evaluations is time-intensive and unattractive to customers as this is mostly handwork.