vendure-data-hub-plugin

Changelog

All notable changes to the Data Hub Plugin are documented here.

[0.1.6] - 2026-03-29

Added

Fixed

Changed

[0.1.5] - 2026-03-05

Added

Fixed

Changed

[0.1.4] - 2026-03-03

Added

Fixed

[0.1.3] - 2026-02-26

Added

Fixed

[0.1.2] - 2026-02-24

Fixed

[0.1.1] - 2026-02-24

Fixed

[0.1.0] - 2026-02-24

Initial production release of the Data Hub Plugin for Vendure.

Core Features

ETL Pipeline Engine

Data Extractors (9 Built-in)

  1. HTTP REST API - Paginated REST endpoint extraction with authentication
  2. GraphQL API - GraphQL query execution with cursor/offset pagination
  3. Vendure Query - Native Vendure entity queries
  4. File - CSV, JSON, XML, XLSX file parsing with streaming
  5. Database - SQL database extraction (PostgreSQL, MySQL, SQL Server) with incremental sync
  6. S3 - AWS S3 and compatible object storage
  7. FTP/SFTP - File transfer protocol servers
  8. Webhook - Incoming webhook data capture
  9. CDC - Change Data Capture for real-time database sync

Transform Operators (61 Built-in)

String Operations (12): split, join, trim, lowercase, uppercase, slugify, concat, replace, extractRegex, replaceRegex, stripHtml, truncate

Numeric Operations (9): math, currency, unit, toNumber, toString, parseNumber, formatNumber, toCents, round

Date Operations (5): dateFormat, dateParse, dateAdd, dateDiff, now

Logic Operations (4): when, ifThenElse, switch, deltaFilter

JSON Operations (4): parseJson, stringifyJson, pick, omit

Data Operations (8): map, set, remove, rename, copy, template, hash, uuid

Enrichment Operations (5): lookup, coalesce, enrich, default, httpLookup

Aggregation Operations (8): aggregate, count, unique, flatten, first, last, expand, multiJoin

Validation Operations (2): validateRequired, validateFormat

Script Operation (1): script (custom JavaScript transform)

File Operations (3): imageResize, imageConvert, pdfGenerate

Entity Loaders (22 Built-in)

Vendure Entity Loaders (16):

Operation Loaders (6):

Field Transforms (49 Built-in)

String (16): TRIM, LOWERCASE, UPPERCASE, SLUGIFY, TRUNCATE, PAD, REPLACE, REGEX_REPLACE, REGEX_EXTRACT, SPLIT, JOIN, CONCAT, TEMPLATE, STRIP_HTML, ESCAPE_HTML, TITLE_CASE, SENTENCE_CASE

Number (10): PARSE_NUMBER, PARSE_FLOAT, PARSE_INT, ROUND, FLOOR, CEIL, ABS, TO_CENTS, FROM_CENTS, MATH

Date (3): PARSE_DATE, FORMAT_DATE, NOW

Boolean (2): PARSE_BOOLEAN, NEGATE

Type Conversion (6): TO_STRING, TO_NUMBER, TO_BOOLEAN, TO_ARRAY, TO_JSON, PARSE_JSON

MAP (1): MAP

Conditional (3): IF_ELSE, COALESCE, DEFAULT

Array (6): FIRST, LAST, NTH, FILTER, MAP_ARRAY, FLATTEN

Expression (1): EXPRESSION (custom JS expressions)

Async (1): LOOKUP (external data enrichment)

Triggers (6 Types)

  1. MANUAL - On-demand execution via dashboard or API
  2. SCHEDULE - Cron-based scheduling with timezone support
  3. WEBHOOK - HTTP webhook endpoints with authentication (API key, HMAC, Basic, JWT)
  4. EVENT - Vendure event subscriptions (ProductEvent, OrderEvent, etc.)
  5. FILE - File watch triggers for FTP/SFTP/S3 with glob patterns
  6. MESSAGE - RabbitMQ/Redis message queue consumers

Pipeline Hooks (24 Stages)

Step-Level Hooks (18): BEFORE/AFTER for each step type

Global Hooks (6):

Hook Actions: INTERCEPTOR (modify records), SCRIPT (custom JS), WEBHOOK (HTTP call), EMIT (event), TRIGGER_PIPELINE (chain), LOG (audit)

Product Feeds (4 Built-in)

Search Sinks (7 Built-in)

Advanced Features

Security

Horizontal Scaling

Secret Management

Visual Pipeline Builder

Code-First DSL

Complete TypeScript DSL for programmatic pipeline definitions:

import { createPipeline, DataHubPlugin } from '@oronts/vendure-data-hub-plugin';

const productSync = createPipeline()
  .name('Product Import')
  .description('Daily product sync from ERP')
  .trigger('start', { type: 'SCHEDULE', cron: '0 2 * * *' })
  .extract('fetch-products', {
    adapterCode: 'httpApi',
    'connection.endpoint': 'https://erp.example.com/api/products',
    'connection.apiKeySecretCode': 'erp-api-key',
  })
  .transform('prepare', {
    operators: [
      { op: 'map', args: { mapping: { sku: 'itemNumber', name: 'productName' } } },
      { op: 'slugify', args: { source: 'name', target: 'slug' } },
      { op: 'toNumber', args: { source: 'price' } },
    ],
  })
  .load('upsert-products', {
    adapterCode: 'productUpsert',
    strategy: 'UPSERT',
    matchField: 'sku',
  })
  .edge('SCHEDULE', 'fetch-products')
  .edge('fetch-products', 'prepare')
  .edge('prepare', 'upsert-products')
  .build();

DataHubPlugin.init({
  pipelines: [productSync],
});

Import & Export Wizards

Guided Wizards: Step-by-step pipeline creation without code

Built-in Import Templates (6):

Built-in Export Templates (8):

Custom Templates: Register your own templates via DataHubPluginOptions.importTemplates and exportTemplates

Dashboard Features

API

GraphQL API (22 resolvers):

REST API (2 controllers):

Permissions System (27 permissions):

SDK & Extensibility

Registration Functions (10):

// Register custom adapters
registerExtractor(definition);
registerLoader(definition);
registerOperator(definition);
registerExporter(definition);
registerFeed(definition);
registerSink(definition);
registerValidator(definition);
registerEnricher(definition);
registerTransform(definition);
registerScript(name, function);

Public API Exports:

Event System

Event Publishing (22 event types):

Event Subscriptions:

Gate Steps (Human-in-the-Loop)

Pimcore Connector

Pre-built integration for Pimcore PIM/DAM → Vendure sync:

4 Import Templates:

1 Export Template:

Features:

Developer Experience

Configuration

Plugin Options:

Environment Variables:

Migration Notes

From Pre-release to 0.1.0:

  1. Database Migration: Run Vendure migrations to create new tables 
    npx vendure migrate
  2. Secret Encryption (Optional but Recommended):
    • Generate master key: openssl rand -hex 32
    • Set DATAHUB_MASTER_KEY environment variable
    • Re-save existing INLINE secrets through the UI (auto-encrypted on save)
  3. Breaking Changes: None - this is the initial release

Support

Security Notes

Known Limitations

Version History

This site is open source. Improve this page.