Components

Core building blocks of the DataLinks Python SDK.

1. DLConfig

DLConfig reads configurations (e.g., API keys) via environment variables or .env files. This enables dynamic adaptation across deployment environments.

2. DataLinksAPI

DataLinksAPI handles interactions with the API. You can:

• Ingest data directly or via multipart upload for large files.
• Track and wait for async ingestion completion.
• Query or retrieve data with complex parameters.
• Manage namespaces.

3. Inference Workflow

Use a chain of inference and validation steps defined through classes like ProcessUnstructured, Normalize, and Validate to automate data preparation workflows.

from datalinks.pipeline import Pipeline, ProcessUnstructured, Normalize, Validate, ValidateModes

# Define an inference pipeline
inference_steps = Pipeline(
   ProcessUnstructured(derive_from="source_field", helper_prompt="This extracts tables."),
   Normalize(target_cols={"email": "email_address"}, mode="all-in-one"),
   Validate(mode=ValidateModes.FIELDS, columns=["email", "phone"]),
)

4. Entity Resolution

Supports multiple resolution strategies, configurable via MatchTypeConfig:

from datalinks.links import MatchTypeConfig, ExactMatch

entity_resolution = MatchTypeConfig(
   # parameters are optional
    exact_match=ExactMatch(minVariation=0.2, minDistinct=0.3)
)

5. Loaders

Abstract base loaders (e.g., JSONLoader) allow seamless data ingestion from custom file formats like .json.

6. Parametrize LLMs

You can choose the model and provider to be used in inference steps (eg.: ProcessUnstructured, Normalize, Validate).

from datalinks.pipeline import Pipeline, ProcessUnstructured

steps = Pipeline(
        ProcessUnstructured(
            derive_from="text",
            helper_prompt="If you find a numeric field use only the value and omit the rest.",
            model="gpt-4.1-nano-2025-04-14",
            provider="openai"
        )
    )