vendure-data-hub-plugin

DSL Reference

The Data Hub DSL (Domain Specific Language) provides a fluent TypeScript API for defining pipelines.

Contents

1. Pipeline Builder - Core pipeline construction
2. Operators - All transform operators
3. Examples - Real-world pipeline examples

Quick Start

import { createPipeline } from '@oronts/vendure-data-hub-plugin';

const pipeline = createPipeline()
    .name('My Pipeline')
    .trigger('start', { type: 'MANUAL' })
    .extract('fetch', { adapterCode: 'httpApi', url: 'https://api.example.com/products' })
    .transform('process', { operators: [...] })
    .load('save', { adapterCode: 'productUpsert', strategy: 'UPSERT', matchField: 'slug' })
    .edge('start', 'fetch')
    .edge('fetch', 'process')
    .edge('process', 'save')
    .build();

Key Concepts

Builder Pattern

The DSL uses a fluent builder pattern. Each method returns the builder, allowing chaining:

createPipeline()
    .name('Name')           // Returns builder
    .description('Desc')    // Returns builder
    .trigger(...)           // Returns builder
    .build();               // Returns PipelineDefinition

Step Keys

Each step has a unique key within the pipeline:

.extract('my-unique-key', { ... })

Keys are used in edges to connect steps.

Edges

Edges define the execution order:

.edge('step-a', 'step-b')  // step-a runs before step-b

Configuration Objects

Each step type has specific configuration:

// Extract step
.extract('fetch', {
    adapterCode: 'httpApi',
    url: 'https://api.example.com/products',
    method: 'GET',
})

// Transform step
.transform('process', {
    operators: [
        { op: 'rename', args: { from: 'old', to: 'new' } },
    ],
})

// Load step
.load('save', {
    adapterCode: 'productUpsert',
    strategy: 'UPSERT',
    matchField: 'slug',
})

Types

The DSL is fully typed. Import types for IDE support:

import {
    createPipeline,
    PipelineBuilder,
    PipelineDefinition,
    ExtractStepConfig,
    TransformStepConfig,
    LoadStepConfig,
} from '@oronts/vendure-data-hub-plugin';

Using with Plugin Options

Pass built pipelines to the plugin:

import { DataHubPlugin, createPipeline } from '@oronts/vendure-data-hub-plugin';

const myPipeline = createPipeline()
    .name('My Pipeline')
    // ... steps ...
    .build();

export const config: VendureConfig = {
    plugins: [
        DataHubPlugin.init({
            pipelines: [{
                code: 'my-pipeline',
                name: 'My Pipeline',
                enabled: true,
                definition: myPipeline,
            }],
        }),
    ],
};

Next Steps

• Pipeline Builder - Complete builder API
• Operators - All available operators
• Examples - Common patterns

This site is open source. Improve this page.