Welcome to Grid2Viz’s documentation!¶

Grid2Viz is a web application that offers several interactive views into the results of Reinforcement Learning agents that ran on the Grid2Op platform.

Installation¶

Requirements¶

Python >= 3.6

Step 1 (Optional, recommended): Create a virtual environment¶

pip3 install -U virtualenv
python3 -m virtualenv venv_grid2viz

Step 2: Install from sources¶

source venv_grid2viz/bin/activate
git clone https://github.com/rte-france/grid2viz.git
cd grid2Viz/
pip install -U .

Step 2 (bis): Install from PyPI¶

source venv_grid2viz/bin/activate
pip install grid2viz

Running the application and Configuration¶

In order to use this tool, you need to have serialized the RL process of grid2op. The expected file system is :

root_dir
- agent_1
  
  scenario_1
  
  scenario_2
- agent_2
  
  scenario_1
  
  scenario_2
  
  scenario_3

Each of the scenario_* files have to contain all files given by serialisation of your RL through grid2op. In order to add a new agent to the app, you will have to add the agent’s folder to this root_dir For the update process of this folder chain, see the section Caching (in particular, when you want to overwrite the current agents in root_dir with new versions with the same names)

The grid2viz commandline tool allows to start the server and provide all the necessary information. The help function of the tool can be found below.

usage: grid2viz [-h] [--agents_path AGENTS_PATH] [--env_path ENV_PATH] [--port PORT] [--debug] [--n_cores N_CORES] [--cache CACHE] [--warm-start WARM_START]
                [--config-path CONFIG_PATH] [--activate-beta ACTIVATE_BETA]

Grid2Viz

optional arguments:
  -h, --help            show this help message and exit
  --agents_path AGENTS_PATH
                        The path where the log of the Agents experiences are stored. (default to None to study the example agents provided with the package)
  --env_path ENV_PATH   The path where the environment config is stored. (default to None to use the provided default environment)
  --port PORT           The port to serve grid2viz on. (default to 8050)
  --debug               Enable debug mode for developers. (default to False)
  --n_cores N_CORES     The number of cores to use for the first loading of the best agents of each scenario
  --cache               Enable the building of all the cache data for all agents at once before relaunching grid2viz. (default to False)
  --warm-start          If True, the application is warm started based on the parameters defined in the WARMSTART section of the config.ini file. (default to False)
  --config-path CONFIG_PATH
                        Path to the configuration file config.ini.
  --activate-beta       Enable beta features. (default to False)

For example, to run the server on port 8000 with the default agents provided in the :doc: Starting-Kit<starting-kit>:

source venv_grid2viz/bin/activate
grid2viz --port 8000

Configuration¶

The –config-path argument allows to provide a configuration ini file to pass some options as describe bellow.

[DEFAULT]
agents_dir= # Same as the --agents_path command line argument
env_dir=# Same as the --env_path command line argument

[WARMSTART] # Section used to warm start the application form a specfici view
scenario= # Name of the scenario to be loaded
agent_ref= # Name of the reference agent to be loaded
agent_study= # Name of the study agent to be loaded
page= # End-Point of the view to be loaded (episodes, overview, macro, micro, simulation)
time_step= # Time step to be loaded

The command line arguments have precedence over the config file.

WARNING: Due to the caching operation the first run can take a while. All the agents present in the configuration files will be computed and then registered in cache. Depending on your agents it could take between 5 to 15min. You can follow the progress in the console.*

Starting Kit¶

Grid2Viz provides 2 agents with a scenario for one day and for one month available in /grid2viz/data/agents folder:

do-nothing-baseline
greedy-baseline

By default the config.ini is targeting these agents as well as the environment configuration folders.

To launch this demo, just Run gri2viz with no arguments:

grid2viz

Caching¶

The cache system allows you to only compute long calculations of the app once per agent/scenario. The app will create a folder _cache in the base_dir of the config.ini which will contain these long calculations serialized.

If you add a new folder in your base_dir (either an agent, or a scenario) you will have to restart the server so the app reads the folder tree again.

_WARNING_ : If you overwrite the agents while they were already cached, you will have to manually reset the cache so the app knows to compute everything again with the updated data. To do so, you just need to delete the _cache folder.

Scenario Selection¶

This view offers a synthetic summary of the available scenarios and the agents that ran on these scenarios. Each card represent a scenario that

displays some synthetic indicators about the scenarios and the agents that ran on them
can be further analyzed by clicking the open button which will open the Scenario Overview view

Scenario Overview¶

The Scenario Overview view gathers some useful information about the selected scenario, through the “eyes” of the agent with the highest score on the scenario (called the best agent in the sequel). This agent is the one that survied the longest and thus that has seen the biggest part of the environment (among all the agents that ran on the scenario). It also provides some initial information about a reference agent that can be selected in this view. The view is composed of three main sections described below.

Indicators¶

This sections displays :

A Line chart representing the distribution of the daily consumption profiles for the best agen.
A Pie Chart for the production shares (in energy) for the best agent.
A set of indicators for the best agent :
- The number of time steps played over the duration of the scenario
- The number of seen Hazards
- The number of seen Maintenances
- The duration, in minutes, of seen Maintenances

Summary¶

This section displays :

A Line chart presenting the actual environnement time series, namely loads, productions, hazards and maintenances, seen by the best agent. You are able to search by name the individual elements you want to display. You can look at aggregated variable such as total load, wind, solar, hydro, nuclear and thermal productions.
A Dropdown to select a reference agent that can serve as a baseline to compare with a studied agent. Any agent that ran on the scenario can be selected here.
A Line chart representing some quantiles of the usage rate accross lines for the reference agent.
A Line chart enumerating the overflows seen by the reference agent.

Inspector¶

This section displays a detailed interactive data table comprising the values of the environnement time series:

You can filter the date range your interested in through a calendar.
You can filter each column according to some values following the syntax here: https://dash.plotly.com/datatable/filtering
You can add columns of the specific productions and loads you want to inspect through the search field.

Agent Overview¶

The Agent Overview offers some preliminary insights on the studied agent (compared to the reference agent selected in the Scenario Overview view). The view is composed of three main sections described below.

Indicators¶

This section allows to select the agent to be studied through a dropdown and :

Some synthetic information :
- The score
- The number of overflows
- The Number of actions played
A Pie Chart representing the types of actions that the agent played.
An Histogram representing the duration of Maintenances (counted on the whole scenario).

Overview¶

This section allows to interactively explore a series of 4 synchronised graphs :

Instant and Cumulated Reward : Reward and cumulative reward time series for the reference agent and the studied agent. The time steps when the studied agent took an action are highlighted with a yellow mark.
Overflow and Maintenances : Time series for the overflows generated by the studied agent and the maintenances seen by the agent.
Actions : Time series for the action events (1 if an action is taken, 0 otherwise) and the distance of the current network topology from the orignal topology. Hovering over the action events time series will display the string representation of the action.
Usage Rate : Same graph as in Scenario Overview but for the studied agent.

The time frame for these graphes is the one of the studied agent.

Time step Selection for Study¶

Additionnaly, clicking on any of the lines of the Instant and Cumulated Reward graph will select the corresponding time step and add it in the table on the left. These selected time steps will then be available in the Agent Study to allow specific focuses around those time steps.

Inspector for studied agent¶

This section offers a detailed table referencing the impacts of the actions taken by the studied agent. Clicking on any cell will display, below the table, the string representation of the action, if any, taken at the corresponding time step.

Summary of action used¶

The section is completed by two graphs representing histograms of substations and lines targeted by the action of the studied agent over the whole scenario.

Agent Study¶

The Agent Study view allows to explore further around a time step of interest the actions taken by an agent and their impacts. The view is composed of three main sections described below.

The navbar displays for this view a special component, namely a time frame selector. It is composed of a dropdown menu allowing to select one of the time step extracted from the Agent Overview view. The -5 and +5 buttons allows to extend the default time frame (10 timesteps on each side of the selected time step).

Indicators¶

This section displays the same Instant and Cumulated Reward and Actions graphs as the Agent Overview view, but centered around the selected time step.

Flow¶

This sections provides a Grid2Op plot representation of the grid. A slider, on the same time frame as the Indicators graphs allows to navigate between the corresponding grid states. The string representation of the action for a given time step is again provided below the slider.

The section is completed with a collection of time series representing flows accross lines (either in MW, Amps or usage rate) and voltages.

Context¶

The Context section is a useful reminder of the environment time series, as seen by the studied agent, the usage rate graph and the overflows and maintenances graph. It is intended to help explore in greater details the context in which an action was taken to have a comprehensive understanding of the situation.

Simulation¶

Note

The simulation view is still a BETA feature (works only with a DoNothing agent) and is accessible through the –activate-beta command line argument

The Simulation view allows to replay a specific time step with custom actions, either built as a collection of atomic actions, or with an assistant. The resulting simulated observation can then be compared to the original observation.

This view allows to explore what would have happen if the agent had taken another set of actions.

As for the Agent Study view, the studied time step in the Simulation view can be selected in the Agent Overview view and then changed in the navigation bar.

Chose or Assist¶

This section allows to configure the action to be simulated. Two options are available :

A manual configuration through a set of radio buttons and dropdowns that allow to select a specific object and a specific action to apply on. Such actions add up with the Add button.
An assisted action configuration through an assistant pluggin:

Right now, only the expert system ExpertOp4Grid is provided with the application. An interface is exposed to implement other pluggins.

Compare¶

This sections provides the Grid2Op plot representation of the grid after the action has been simulated and a set of KPIs.

Use Case for the Assistant¶

With the data provided in the starting kit, we can go through a small use case of the ExpertOp4Grid assistant. Configuration :

agent do-nothing-baseline
scenario 000
time_step 1895
Line to Act on : 1_4_4
Flow ratio (in %) to get below : 80 (The current flow is 83.95%)

Evaluate the expert system and choose the first (i.e. best) action. It will reconfigure the substation 4 to make to flow on line 1_4_4 go below 80%.

_images/simulation_use_case_expert_results.png

You can now see the results by simulating the action in the Compare section. The flow on 1_4_4 is now way below 80% but at the cost of a higher flow on adjacent lines and higher losses.

_images/simulation_use_case_expert_results_graph.png