Core DataSignals API#
The core DataSignals API module is designed to provide a more direct interaction with the 24SEA API and is the base for the DataSignals module.
Installation#
The package supports Python 3.8 and above. To install it, run the following command in your terminal:
pip install api_24sea
Usage#
The following example shows the classical usage of the core DataSignals API module, which can be integrated within other standalone classes or functions.
The first step is to import the package and the necessary libraries.
Then, the environment variables are loaded from a
.envfile.After that, the package is initialized and the user is authenticated with the API.
Finally, the user can get data from the API.
The first two steps are the same as in the DataSignals module and will not be repeated here.
Note
The core DataSignals API module contains two main classes with the same
functionality, but different logic: API and AsyncAPI.
The first one is a synchronous class and the second one is an asynchronous
class. The user can choose which one to use depending on the needs.
The API class is the default class and is used in the following examples.
The AsyncAPI class is used in the
Asynchronous API Usage section.
Authenticating with the API (optional)#
The authentication step is performed automatically if the environment variables
24SEA_API_USERNAME and 24SEA_API_PASSWORD are loaded. The user can also
authenticate manually by calling the authenticate method from the API.
This step is optional, since the API is able to authenticate lazily if the environment variables are loaded, or the user can authenticate manually.
# %%
# Package Imports
from api_24sea.datasignals.core import API
# %%
# **Authentication**
api = API()
api.authenticate("some_other_username", "some_other_password")
Checking the available metrics#
# %%
# **Metrics Overview**
# The metrics overview is a summary of the metrics available in the API and
# can be accessed from a hidden method in the DataSignals class.
api.metrics_overview
# It will show all the available metrics with the corresponding units
# and the time window for which the user is allowed to get Data
Getting sample data from the API#
After loading the environment variables and authenticating with the API, the user can get data from 24SEA API endpoints.
The retrieved data can be retrieved in multiple formats, depending on the combination of the following three options:
as_dict: If True, any dataframe output is returned as a dictionary according to the following formula:dataframe.reset_index().to_dict('records').as_star_schema: If True, the data is returned in a star schema format, i.e. a dictionary with the following keys:DimCalendar,DimWindFarm,DimDataGroup,DimMetric, andFactData.outer_join_on_timestamp: if True, the response from each metric is joined on the timestamp which is then set as the index of DataFrame. This option will necessarily drop thelocationandsitecolumns from the DataFrame, but they can still be retrieved from the metrics overview. If False, the response from each metric is stored in a separate DataFrame and thesiteandlocationcolumns are kept. This means that the DataFrame will be “diagonal” with repeated timestamps (as many times as the number of queried (locations, sites) pairs).
Therefore, the following combinations are possible:
Single
pandas.DataFrame:outer_join_on_timestamp=True,as_star_schema=False, andas_dict=False. This is the default behavior.dict[str, Union[float, dict[str, Any]]]:outer_join_on_timestamp=False,as_star_schema=False, andas_dict=True.Star schema as
dict[str, pandas.Dataframe]:outer_join_on_timestampis irrelevant,as_star_schema=True, andas_dict=False.Star schema as
dict[str, dict[str, Any]]:outer_join_on_timestampis irrelevant,as_star_schema=True, andas_dict=True.
The optional method parameter controls the HTTP verb used for the backend
request. The default is "GET". Supported values are "GET", "POST",
"PUT", "PATCH", and "DELETE".
# %%
# **Data Retrieval**
sites = ["wf"]
locations = ["a01", "a02"]
metrics = ["mean WinDSpEed", "mean pitch", "mean-Yaw", "mean_power"]
start_timestamp = "2020-03-01T00:00:00Z"
end_timestamp = "2020-06-01T00:00:00Z"
data_with_get = api.get_data(sites, locations, metrics,
start_timestamp, end_timestamp,
method="GET")
data_with_post = api.get_data(sites, locations, metrics,
start_timestamp, end_timestamp,
method="POST")
Data as Star Schema#
Overview#
The data as star schema feature is designed to provide a more user-friendly experience when getting data for BI purposes. It is implemented only for the core DataSignals API module.
from api_24sea.datasignals.core import API
# %%
# **Star schema**
# %%
# **Data Retrieval**
sites = ["wf"]
locations = ["a01", "a02"]
metrics = ["mean WinDSpEed", "mean pitch", "mean-Yaw", "mean_power"]
start_timestamp = "2020-03-01T00:00:00Z"
end_timestamp = "2020-06-01T00:00:00Z"
star_schema = api.get_data(sites, locations, metrics,
start_timestamp, end_timestamp, as_star_schema=True)
This command is equivalent to the following:
from api_24sea.datasignals.core import to_star_schema, API
# %%
# **Data Retrieval**
api = API()
sites = ["wf"]
locations = ["a01", "a02"]
metrics = ["mean WinDSpEed", "mean pitch", "mean-Yaw", "mean_power"]
start_timestamp = "2020-03-01T00:00:00Z"
end_timestamp = "2020-06-01T00:00:00Z"
data = api.get_data(sites, locations, metrics,
start_timestamp, end_timestamp)
star_schema = to_star_schema(data, api.selected_metrics(data).reset_index())
The star_schema variable will contain a dictionary with the following keys:
DimCalendar: A DataFrame with the timestamps and their corresponding calendar information.DimWindFarm: A DataFrame with the wind farm information (site and locations).DimDataGroup: A DataFrame with the metric group information (e.g. TP, SCADA).DimMetric: A DataFrame with the metric information (e.g. mean pitch, mean power).FactData: A DataFrame with the data itself.FactPivotData: A DataFrame with the data pivoted by metric.
Data as Category-Value#
The data as category-value feature is designed to reshape the data so that all the information for a metric is stored in a single row.
# %%
# **Data Retrieval**
from api_24sea.datasignals.core import to_category_value
sites = ["wf"]
locations = ["a01", "a02"]
metrics = ["mean WinDSpEed", "mean pitch", "mean-Yaw", "mean_power"]
start_timestamp = "2020-03-01T00:00:00Z"
end_timestamp = "2020-06-01T00:00:00Z"
data = api.get_data(sites, locations, metrics,
start_timestamp, end_timestamp)
category_value = to_category_value(data, api.selected_metrics(data).reset_index())
Example table#
The data is transformed from the following (example) shape:
timestamp |
mean_WF_A01_TP_SG_LAT005_DEG000 |
mean_WF_A01_TP_SG_LAT005_DEG045 |
mean_WF_A01_TP_SG_LAT005_DEG090 |
mean_WF_A02_TP_SG_LAT015_DEG000 |
mean_WF_A02_TP_SG_LAT015_DEG045 |
mean_WF_A02_TP_SG_LAT015_DEG090 |
|---|---|---|---|---|---|---|
2020-03-01 00:00:00 |
1.0 |
2.0 |
3.0 |
4.0 |
5.0 |
6.0 |
2020-03-01 00:10:00 |
1.1 |
2.1 |
3.1 |
4.1 |
5.1 |
6.1 |
2020-03-01 00:20:00 |
1.2 |
2.2 |
3.2 |
4.2 |
5.2 |
6.2 |
2020-03-01 00:30:00 |
1.3 |
2.3 |
3.3 |
4.3 |
5.3 |
6.3 |
to the following (example) shape:
timestamp |
full_metric_name |
value |
unit |
statistic |
short_hand |
site_id |
location_id |
lat |
heading |
site |
location |
metric_group |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
2020-03-01 00:00:00 |
mean_WF_A01_TP_SG_LAT005_DEG000 |
1.0 |
unit |
mean |
TP_SG_LAT005_DEG000 |
WF |
A01 |
5.0 |
0.0 |
WindFarm |
WFA01 |
TP |
2020-03-01 00:10:00 |
mean_WF_A01_TP_SG_LAT005_DEG000 |
1.1 |
unit |
mean |
TP_SG_LAT005_DEG000 |
WF |
A01 |
5.0 |
0.0 |
WindFarm |
WFA01 |
TP |
2020-03-01 00:20:00 |
mean_WF_A01_TP_SG_LAT005_DEG000 |
1.2 |
unit |
mean |
TP_SG_LAT005_DEG000 |
WF |
A01 |
5.0 |
0.0 |
WindFarm |
WFA01 |
TP |
2020-03-01 00:30:00 |
mean_WF_A01_TP_SG_LAT005_DEG000 |
1.3 |
unit |
mean |
TP_SG_LAT005_DEG000 |
WF |
A01 |
5.0 |
0.0 |
WindFarm |
WFA01 |
TP |
2020-03-01 00:00:00 |
mean_WF_A01_TP_SG_LAT005_DEG045 |
2.0 |
unit |
mean |
TP_SG_LAT005_DEG045 |
WF |
A01 |
5.0 |
45.0 |
WindFarm |
WFA01 |
TP |
2020-03-01 00:10:00 |
mean_WF_A01_TP_SG_LAT005_DEG045 |
2.1 |
unit |
mean |
TP_SG_LAT005_DEG045 |
WF |
A01 |
5.0 |
45.0 |
WindFarm |
WFA01 |
TP |
2020-03-01 00:20:00 |
mean_WF_A01_TP_SG_LAT005_DEG045 |
2.2 |
unit |
mean |
TP_SG_LAT005_DEG045 |
WF |
A01 |
5.0 |
45.0 |
WindFarm |
WFA01 |
TP |
2020-03-01 00:30:00 |
mean_WF_A01_TP_SG_LAT005_DEG045 |
2.3 |
unit |
mean |
TP_SG_LAT005_DEG045 |
WF |
A01 |
5.0 |
45.0 |
WindFarm |
WFA01 |
TP |
2020-03-01 00:00:00 |
mean_WF_A01_TP_SG_LAT005_DEG090 |
3.0 |
unit |
mean |
TP_SG_LAT005_DEG090 |
WF |
A01 |
5.0 |
90.0 |
WindFarm |
WFA01 |
TP |
2020-03-01 00:10:00 |
mean_WF_A01_TP_SG_LAT005_DEG090 |
3.1 |
unit |
mean |
TP_SG_LAT005_DEG090 |
WF |
A01 |
5.0 |
90.0 |
WindFarm |
WFA01 |
TP |
2020-03-01 00:20:00 |
mean_WF_A01_TP_SG_LAT005_DEG090 |
3.2 |
unit |
mean |
TP_SG_LAT005_DEG090 |
WF |
A01 |
5.0 |
90.0 |
WindFarm |
WFA01 |
TP |
2020-03-01 00:30:00 |
mean_WF_A01_TP_SG_LAT005_DEG090 |
3.3 |
unit |
mean |
TP_SG_LAT005_DEG090 |
WF |
A01 |
5.0 |
90.0 |
WindFarm |
WFA01 |
TP |
2020-03-01 00:00:00 |
mean_WF_A02_TP_SG_LAT015_DEG000 |
4.0 |
unit |
mean |
TP_SG_LAT015_DEG000 |
WF |
A02 |
15.0 |
0.0 |
WindFarm |
WFA02 |
TP |
2020-03-01 00:10:00 |
mean_WF_A02_TP_SG_LAT015_DEG000 |
4.1 |
unit |
mean |
TP_SG_LAT015_DEG000 |
WF |
A02 |
15.0 |
0.0 |
WindFarm |
WFA02 |
TP |
2020-03-01 00:20:00 |
mean_WF_A02_TP_SG_LAT015_DEG000 |
4.2 |
unit |
mean |
TP_SG_LAT015_DEG000 |
WF |
A02 |
15.0 |
0.0 |
WindFarm |
WFA02 |
TP |
2020-03-01 00:30:00 |
mean_WF_A02_TP_SG_LAT015_DEG000 |
4.3 |
unit |
mean |
TP_SG_LAT015_DEG000 |
WF |
A02 |
15.0 |
0.0 |
WindFarm |
WFA02 |
TP |
2020-03-01 00:00:00 |
mean_WF_A02_TP_SG_LAT015_DEG045 |
5.0 |
unit |
mean |
TP_SG_LAT015_DEG045 |
WF |
A02 |
15.0 |
45.0 |
WindFarm |
WFA02 |
TP |
2020-03-01 00:10:00 |
mean_WF_A02_TP_SG_LAT015_DEG045 |
5.1 |
unit |
mean |
TP_SG_LAT015_DEG045 |
WF |
A02 |
15.0 |
45.0 |
WindFarm |
WFA02 |
TP |
2020-03-01 00:20:00 |
mean_WF_A02_TP_SG_LAT015_DEG045 |
5.2 |
unit |
mean |
TP_SG_LAT015_DEG045 |
WF |
A02 |
15.0 |
45.0 |
WindFarm |
WFA02 |
TP |
2020-03-01 00:30:00 |
mean_WF_A02_TP_SG_LAT015_DEG045 |
5.3 |
unit |
mean |
TP_SG_LAT015_DEG045 |
WF |
A02 |
15.0 |
45.0 |
WindFarm |
WFA02 |
TP |
2020-03-01 00:00:00 |
mean_WF_A02_TP_SG_LAT015_DEG090 |
6.0 |
unit |
mean |
TP_SG_LAT015_DEG090 |
WF |
A02 |
15.0 |
90.0 |
WindFarm |
WFA02 |
TP |
2020-03-01 00:10:00 |
mean_WF_A02_TP_SG_LAT015_DEG090 |
6.1 |
unit |
mean |
TP_SG_LAT015_DEG090 |
WF |
A02 |
15.0 |
90.0 |
WindFarm |
WFA02 |
TP |
2020-03-01 00:20:00 |
mean_WF_A02_TP_SG_LAT015_DEG090 |
6.2 |
unit |
mean |
TP_SG_LAT015_DEG090 |
WF |
A02 |
15.0 |
90.0 |
WindFarm |
WFA02 |
TP |
2020-03-01 00:30:00 |
mean_WF_A02_TP_SG_LAT015_DEG090 |
6.3 |
unit |
mean |
TP_SG_LAT015_DEG090 |
WF |
A02 |
15.0 |
90.0 |
WindFarm |
WFA02 |
TP |
Tip
Try it live Run these examples directly in the browser with the Interactive Example.