The increasing integration of renewable energy into power grids demands smarter tools for managing generation and consumption. In this context, energy forecasting plays a crucial role, especially in renewable energy communities (RECs), where prosumers both generate and consume electricity.

The idrECO project aims to place end users at the center of energy systems, empowering them through technology. The project focuses on applying machine learning (ML) and deep learning (DL) techniques to predict three essential energy-related variables: electricity prices, solar production, and user consumption.

The Three Pillars of Prediction in idrECO

To build a comprehensive prediction system, the project focused on:

  • Electricity price forecasting: Helping users decide when to buy or sell energy.
  • Solar energy production forecasting: Estimating renewable supply in real time.
  • Consumption forecasting: Predicting users' energy demand patterns.

Each of these targets was approached with specialized AI models trained on real datasets, ensuring relevance to real-world applications in smart energy markets.

Datasets: A Strong Foundation

idrECO's models rely on high-quality, granular data from trusted sources:

  • Electricity prices: Hourly prices from the Spanish Red Eléctrica.
  • National production data: Energy mix (solar, wind, nuclear, etc.) from ENTSO-E.
  • Prosumers data: Detailed hourly data from EneFit’s Kaggle competition, including usage patterns, business status, contract types, and weather.

The integration of diverse datasets allows for more nuanced predictions that consider both market dynamics and local behavior.

Data Preprocessing and Engineering

Before training any models, extensive data preprocessing was performed:

  • Merging price and production data.
  • Aligning time formats and handling daylight saving inconsistencies.
  • Creating new features (hour, weekday, month, business hours).
  • Removing highly null variables and scaling data for model input.

For prosumer data, preprocessing included matching weather information by location, cleaning missing data with KNN imputation, and ensuring time series integrity for each client.

AI Models Applied: LSTM, RNN, XGBoost and More

The project implemented and compared various time series prediction models:

  • LSTM (Long Short-Term Memory): Excellent for capturing temporal dependencies.
  • RNN (Recurrent Neural Networks): A simpler architecture, less effective over long sequences.
  • XGBoost: A powerful tree-based method adapted for time series by flattening input sequences.
  • Baseline persistence models: Used for benchmarking (e.g., "tomorrow = today").

Each model was tested with different prediction horizons: 1 hour, 8 hours, and 24 hours, to simulate real usage scenarios in community grids.

Neuro-Symbolic Models for Interpretability

To ensure model transparency, idrECO explored neuro-symbolic AI, combining:

  • Neural networks (LSTM) for pattern recognition.
  • Symbolic rules (e.g., increasing consumption prediction during cold weather).

Two approaches were tested:

  • Neural Symbolic Machines (NSM): Adding logical rules to neural outputs.
  • Rule-Based Neural Networks (RBNN): Embedding rules directly into model layers.

An attention mechanism was also introduced to LSTM, allowing the model to focus on the most relevant time steps — improving explainability, crucial for user trust and decision-making.

Evaluation Metrics: How Good Are the Models?

To assess prediction quality, the team used multiple standard metrics:

  • MAE (Mean Absolute Error)
  • RMSE (Root Mean Squared Error)
  • Percentage Error
  • MAPE (Mean Absolute Percentage Error)

These were applied to test datasets — data never seen during training — to ensure generalizability. LSTM models often outperformed others, particularly when longer historical windows were provided.

Application to Renewable Energy Communities

By providing accurate predictions, the idrECO platform supports:

  • Optimized energy sharing between prosumers and the grid.
  • Informed battery management: deciding when to store or sell energy.
  • Market-aware decisions based on price forecasts.
  • More resilient local energy ecosystems, reducing dependency on fossil-based grids.

These tools empower citizens to actively participate in distributed energy markets, a step toward decentralized and democratic energy systems.

Lessons Learned and Future Work

We have demonstrated that AI can enhance both the efficiency and transparency of community-based energy systems. Among the key takeaways:

  • Quality data and preprocessing are as important as the model choice.
  • LSTM models shine in time series tasks but require careful tuning.
  • Neuro-symbolic systems offer a path to explainable and trustworthy AI in energy.

Future directions include real-time deployment of these models and integration with Reinforcement Learning-based optimization systems developed in parallel tasks of the project.