DataSignals client migration#

This document defines the migration from the DataSignals clients published in api-24sea 2.3.0 to two independent synchronous and asynchronous facades over a shared implementation.

The intended audience is SDK maintainers and reviewers working on the DataSignals client architecture.

Baseline#

Version 2.3.0 exposes two clients from api_24sea.datasignals.core:

  • API, whose methods return results synchronously.

  • AsyncAPI, whose data-retrieval methods return coroutines.

In that baseline, AsyncAPI inherits from API. This inheritance shares authentication state and helper behavior, but it also makes AsyncAPI a subtype of a synchronous client while replacing several inherited methods with coroutines. The two clients therefore do not satisfy the same calling contract.

The pandas DataSignals accessor constructs API internally and must remain synchronous.

Goals#

The migration must:

  • preserve the API and AsyncAPI public names;

  • preserve imports from api_24sea.datasignals.core;

  • preserve all public parameters and defaults;

  • preserve synchronous and asynchronous calling conventions;

  • preserve return types, DataFrame mutation, ordering, and serialization;

  • preserve authentication behavior and credential overrides;

  • preserve exception types and the point at which they are raised;

  • share query planning, size estimation, splitting, and response assembly;

  • keep synchronous and asynchronous request execution separate;

  • make both facades independently testable.

Non-goals#

The migration will not:

  • change DataSignals backend endpoints;

  • redesign the pandas accessor;

  • introduce a unified method that sometimes returns a value and sometimes a coroutine;

  • make a synchronous facade run an asynchronous client through a hidden event loop;

  • change retry, timeout, or caching policies without a separate change;

  • remove deprecated public parameters as part of the refactor.

Compatibility contract#

The 2.3.0 release is the behavioral baseline.

Imports and construction#

The following imports and constructors must remain valid:

from api_24sea.datasignals.core import API, AsyncAPI

api = API()
async_api = AsyncAPI()

Both clients must continue to accept a custom version root through base_url. The normalized value must end in /datasignals/.

Authentication#

authenticate() remains synchronous on both facades. Environment-based lazy authentication and manual credential overrides must update the same observable attributes as in 2.3.0.

Client methods#

API methods continue to return their final value directly. Corresponding AsyncAPI methods continue to require await.

The migration must preserve:

  • get_metrics;

  • get_data;

  • put_data;

  • patch_data;

  • get_stats;

  • get_null_timestamps;

  • get_availability;

  • get_oldest_timestamp;

  • get_stats_predefined_intervals;

  • metrics_overview;

  • selected_metrics.

Results and errors#

The migration must preserve:

  • DataFrame columns, indices, dtypes, and ordering;

  • dictionary and star-schema output shapes;

  • location and timestamp split ordering;

  • partial and empty response behavior;

  • validation and authentication exceptions;

  • GET and POST retrieval method forwarding;

  • cache-bypass forwarding.

Target architecture#

API and AsyncAPI are separate facades. They coordinate shared domain services but own different execution mechanisms.

API facade                         AsyncAPI facade
    |                                     |
    +---------------+---------------------+
                    |
                    v
        Query planning and response assembly
             |                      |
             v                      v
       Sync execution         Async execution
       httpx + threads        AsyncClient + gather

The shared layer produces immutable request descriptions and assembles completed responses. It does not perform HTTP requests and does not own an event loop.

Module ownership#

The target module responsibilities are:

  • core.py: compatibility exports and DataFrame conversion functions;

  • client.py: synchronous API facade;

  • async_client.py: asynchronous AsyncAPI facade;

  • planning.py: request descriptions and query-splitting plans;

  • responses.py: deterministic response merging and output assembly;

  • execution.py: synchronous HTTP execution;

  • async_execution.py: asynchronous HTTP execution;

  • schemas.py: public query validation.

  • writes.py: shared write validation and immutable request descriptions.

Modules may be introduced incrementally. A module is extracted only when it owns a concrete responsibility shared by more than one client.

For 2.4.0, the facade class definitions remain in core.py so the behavioral change stays easy to review. Authentication, planning, response assembly, and both execution mechanisms have concrete module owners. Moving the two facade definitions to client.py and async_client.py is a later mechanical extraction; core.py will continue to export both names.

Migration stages#

Stage 1: capture the contract#

Add architectural tests that record:

  • public imports;

  • method signatures and defaults;

  • sync versus async calling behavior;

  • base URL normalization;

  • authentication-state behavior;

  • representative output and exception behavior.

Existing endpoint tests remain the primary behavioral safety net.

Stage 2: extract request planning#

Move the duplicated get_data request-list construction into a shared, side-effect-free planner. The planner receives validated query information and returns ordered request descriptions.

Both clients must produce identical request descriptions for identical query inputs.

Stage 3: extract response assembly#

Move deterministic DataFrame merging into a shared response module. Preserve the current ordering rules:

  • timestamp order for timestamp-joined results;

  • site, location, and timestamp order for location-preserving results;

  • first non-null value when timestamp rows are collapsed.

Stage 4: isolate execution#

Keep two execution paths:

  • synchronous execution using blocking HTTP and a thread pool;

  • asynchronous execution using httpx.AsyncClient and bounded coroutine gathering.

Executors consume the same ordered request plan.

Stage 5: separate the facades#

Remove AsyncAPI(API). Both facades may share an authentication-focused base only if that base contains no synchronous endpoint methods. Prefer delegation to focused shared services for planning and response handling.

The inheritance relationship itself is not part of the supported public contract.

Stage 6: preserve compatibility exports#

Keep API, AsyncAPI, to_category_value, and to_star_schema importable from api_24sea.datasignals.core.

Moving implementation classes to focused modules must not require downstream import changes.

Stage 7: validate and release#

Run the complete unit and integration test matrix and compare benchmark results before publishing the stable version.

Acceptance criteria#

The refactor is complete when:

  • the full suite passes on Python 3.10, 3.11, 3.12, and 3.13;

  • pandas and pytest deprecations remain warning-free;

  • sync and async contract tests pass against the same fixtures;

  • oversized queries split identically for both facades;

  • merged results retain deterministic ordering;

  • local API integration passes, with staging used only when local is unavailable;

  • multi-location benchmarks do not regress from 2.3.0;

  • wheel and source distributions install and import successfully;

  • AsyncAPI no longer inherits from API;

  • existing documented imports continue to work.

Versioning#

The target version is 2.4.0. This project publishes stable semantic versions directly and does not use alpha or release-candidate versions.

Publish 2.4.0 when the public 2.3.0 contract is preserved.

Escalate the target to 3.0.0 only if implementation proves that a documented public contract must change.

Risks and rollback#

The main risks are:

  • authentication state diverging between facades;

  • exceptions being raised at a different time;

  • accidental event-loop ownership in synchronous code;

  • response rows merging in a different order;

  • DataFrame mutation changing from in-place to replacement;

  • endpoint parameters or defaults drifting between clients.

Each extraction must be independently reversible. The 2.3.0 implementation and its tests remain the rollback baseline until the stable 2.4.0 release.