Get data availability#

This notebook shows how to use the get_availability method to get data from the AsyncAPI. It allows to get information about the availability of the data without loading it, i.e. which timestamps are available for a given location and metric.

API-24SEA endpoint: https://api.24sea.eu/routes/v1/datasignals/availability

[ ]:

# **Package Imports**
# - From the Python Standard Library
import logging
import os
import sys

# - API-24SEA
from api_24sea.version import __version__, parse_version
from api_24sea.datasignals.core import AsyncAPI

[18]:

# **Package Version**
print(f"Package {parse_version(__version__)}")

# **Notebook Configuration**
logger = logging.getLogger()
logger.setLevel(logging.WARNING)

Package Version(major=2, minor=1, patch=6, release=None, num=None)

Login Credentials

Do not store API credentials in plain text in your notebook. Rather use the python-dotenv package to load environment variables from a .env file.

[19]:

# **Set Sample API Credentials**
os.environ["API_24SEA_USERNAME"] = "Sample.User"
os.environ["API_24SEA_PASSWORD"] = "CheckOutSomeData!"

Availability endpoint#

The availability endpoint allows you to query the availability of data for specific sites, locations, metrics, over a specified time range and granularity. The API returns the percentage of available data points for each metric, location, and time bucket. This can be useful to check if the data you want to query is available before making a data query, or to monitor the availability of your data over time.

Endpoint parameters are:

sites: Optional[Union[List, str]] The sites to filter the data.
locations: Optional[Union[List, str]] The locations to filter the data.
metrics: Union[List, str] The metrics to retrieve.
start_timestamp: Union[str, datetime.datetime] The start timestamp for the data retrieval.
end_timestamp: Union[str, datetime.datetime] The end timestamp for the data retrieval.
granularity: Union[str, int] The granularity of the data, can be a string, or an integer number of seconds. String values are restricted to “day”, “week”, “calendarmonth”, “30days”, or “365days”. If “calendarmonth” is used, the availability will refer to the specific calendar month (e.g. January 2023), and not to a rolling period of 30 days.
sampling_interval_seconds: Optional[int] The sampling interval in seconds. If None, the default value is used, which is 600 seconds (10 minutes).
as_dict: bool Whether to return the data as a dictionary of dictionaries of dataframes (one per location), or as a single dataframe with site and location as columns.
headers: Optional[Union[Dict[str, str]]] Headers to include in the request.
timeout: int The timeout for the request.
threads: Optional[int] The number of threads to use for the request.
location: Optional[Union[List, str]] The location name or List of location names. This is a legacy parameter, and it is deprecated. Please use the locations parameter instead.

[20]:

# **API Initialization**
# API initialization can happen either explicitly via the API class, or
# implicitly when using the pandas `datasignals` accessor.
# In this example we will use the Pandas accessor.
api = AsyncAPI()
# The Metrics Overview is a table containing all the locations and metrics
# available in the API, together with their metadata. It is useful to query it
# before getting data, to check which locations and metrics are available for
# the site you want to query.
m_o = api.metrics_overview

# site is the wind farm name you want to query. The parameter can also be
# a list of site names, e.g. ["windfarm", "windfarm2"]. Matching is
# case-insensitive and passing the "site-id" is also accepted, e.g. "windfarm"
# will match "WindFarm", but also "WF", "wf", etc.
site = "windfarm"
# Matching locations from Metrics Overview for the specified site
# Also partial names of locations are accepted, e.g. "a01" will match
# all locations containing "a01" in their name, such as "A01", "a01". Matching
# is case-insensitive.
locations = m_o[m_o["site"].str.lower() == site]["location"].unique().tolist()
# Metrics: partial matches and regexes are accepted.
# Spaces are interpreted as .* in the name.
# If you want to query all metrics, pass "all" or ["all"].
metrics = ["all"]
# Start and end timestamp in ISO 8601 format. The API accepts also other
# formats, such as the one provided by
# https://www.elastic.co/docs/reference/elasticsearch/rest-apis/common-options#date-math
start_timestamp = "2020-03-01T00:00:00Z"
end_timestamp = "2020-06-01T00:00:00Z"

data = await api.get_availability(site, locations, metrics, start_timestamp,
                                  end_timestamp, granularity="calendarmonth")
data

Getting chunk: [1-1]: 100%|██████████| 1/1 [00:00<00:00,  1.39it/s]

[20]:

	location	dem5_wf_a01_tp_sg_lat010_mtl	dem5_wf_a01_tp_sg_lat010_mtn	dem_wf_a01_tp_sg_lat010_mtl	dem_wf_a01_tp_sg_lat010_mtn	max_wf_a01_nac_acc_fa	max_wf_a01_nac_acc_ss	max_wf_a01_nac_acc_z	max_wf_a01_pitch	max_wf_a01_power	...	std_wf_a02_tp_sg_lat010_deg220_0_nr1	std_wf_a02_tp_sg_lat010_deg280_0_nr1	std_wf_a02_tp_sg_lat010_deg280_t_nr2	std_wf_a02_tp_sg_lat010_deg340_0_nr1	std_wf_a02_tp_sg_lat010_mtl	std_wf_a02_tp_sg_lat010_mtn	std_wf_a02_tp_sg_lat010_n	std_wf_a02_winddirection	std_wf_a02_windspeed	std_wf_a02_yaw
timestamp
2020-03-01 00:00:00+00:00	wfa01	0.997312	0.997312	0.997312	0.997312	0.0	0.0	0.0	0.0	0.0	...	0.998208	0.998208	0.998208	0.998208	0.963038	0.963038	0.963038	0.0	0.964830	0.0
2020-04-01 00:00:00+00:00	wfa01	0.937269	0.937269	0.937269	0.937269	0.0	0.0	0.0	0.0	0.0	...	0.993981	0.993981	0.994213	0.994213	0.993287	0.993287	0.993287	0.0	0.996528	0.0
2020-05-01 00:00:00+00:00	wfa01	0.974238	0.974238	0.974238	0.974238	0.0	0.0	0.0	0.0	0.0	...	1.000000	1.000000	1.000000	1.000000	0.931900	0.931900	0.931900	0.0	0.931900	0.0

3 rows × 301 columns

[21]:

# Extract data for a specific location only
av_as_dict = await api.get_availability(site, locations, metrics, start_timestamp,
                                        end_timestamp, granularity="calendarmonth",
                                        as_dict=True)
av_as_dict
display(av_as_dict[site][locations[0]])
display(av_as_dict[site][locations[1]])

Getting chunk: [1-1]: 100%|██████████| 1/1 [00:00<00:00,  1.91it/s]

metric	DEM5_WF_A01_TP_SG_LAT010_Mtl	DEM5_WF_A01_TP_SG_LAT010_Mtn	DEM_WF_A01_TP_SG_LAT010_Mtl	DEM_WF_A01_TP_SG_LAT010_Mtn	max_WF_A01_NAC_ACC_FA	max_WF_A01_NAC_ACC_SS	max_WF_A01_NAC_ACC_Z	max_WF_A01_TP_ACC_LAT015_DEG240_X_nr1	max_WF_A01_TP_ACC_LAT015_DEG240_Y_nr2	max_WF_A01_TP_ACC_LAT015_FA	...	std_WF_A01_TP_SG_LAT010_DEG340_0_nr1	std_WF_A01_TP_SG_LAT010_Mtl	std_WF_A01_TP_SG_LAT010_Mtn	std_WF_A01_TP_SG_LAT010_N	std_WF_A01_pitch	std_WF_A01_power	std_WF_A01_rpm	std_WF_A01_winddirection	std_WF_A01_windspeed	std_WF_A01_yaw
timestamp
2020-03-01 00:00:00+00:00	0.997312	0.997312	0.997312	0.997312	0.0	0.0	0.0	0.999552	0.999552	0.996864	...	1.0	0.997312	0.997312	0.997312	0.0	0.0	0.0	0.0	0.997312	0.0
2020-04-01 00:00:00+00:00	0.937269	0.937269	0.937269	0.937269	0.0	0.0	0.0	0.937269	0.937269	0.937269	...	0.937269	0.937269	0.937269	0.937269	0.0	0.0	0.0	0.0	0.943287	0.0
2020-05-01 00:00:00+00:00	0.974238	0.974238	0.974238	0.974238	0.0	0.0	0.0	0.97379	0.97379	0.97379	...	0.974238	0.974238	0.974238	0.974238	0.0	0.0	0.0	0.0	0.974238	0.0

3 rows × 150 columns

metric	DEM5_WF_A02_TP_SG_LAT010_Mtl	DEM5_WF_A02_TP_SG_LAT010_Mtn	DEM_WF_A02_TP_SG_LAT010_Mtl	DEM_WF_A02_TP_SG_LAT010_Mtn	max_WF_A02_NAC_ACC_FA	max_WF_A02_NAC_ACC_SS	max_WF_A02_NAC_ACC_Z	max_WF_A02_TP_ACC_LAT015_DEG240_X_nr1	max_WF_A02_TP_ACC_LAT015_DEG240_Y_nr2	max_WF_A02_TP_ACC_LAT015_FA	...	std_WF_A02_TP_SG_LAT010_DEG340_0_nr1	std_WF_A02_TP_SG_LAT010_Mtl	std_WF_A02_TP_SG_LAT010_Mtn	std_WF_A02_TP_SG_LAT010_N	std_WF_A02_pitch	std_WF_A02_power	std_WF_A02_rpm	std_WF_A02_winddirection	std_WF_A02_windspeed	std_WF_A02_yaw
timestamp
2020-03-01 00:00:00+00:00	0.963038	0.963038	0.963038	0.963038	0.0	0.0	0.0	0.998208	0.998208	0.963038	...	0.998208	0.963038	0.963038	0.963038	0.0	0.0	0.0	0.0	0.96483	0.0
2020-04-01 00:00:00+00:00	0.993287	0.993287	0.993287	0.993287	0.0	0.0	0.0	0.994676	0.994676	0.994676	...	0.994213	0.993287	0.993287	0.993287	0.0	0.0	0.0	0.0	0.996528	0.0
2020-05-01 00:00:00+00:00	0.9319	0.9319	0.9319	0.9319	0.0	0.0	0.0	0.93078	0.93078	0.93078	...	1.0	0.9319	0.9319	0.9319	0.0	0.0	0.0	0.0	0.9319	0.0

3 rows × 150 columns