# sz-napi Node.js/TypeScript SDK for Senzing v4 entity resolution, built with NAPI-RS. [![CI](https://github.com/brianmacy/sz-napi/actions/workflows/ci.yml/badge.svg)](https://github.com/brianmacy/sz-napi/actions/workflows/ci.yml) [![License: Apache-2.0](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE) ## Overview This monorepo provides five npm packages: - **`@senzing/types`** -- Shared TypeScript interfaces (`SzEngine`, `SzConfigManager`, `SzDiagnostic`, `SzProduct`, `SzEnvironment`) that define the canonical Senzing API contract. All transports implement these interfaces, enabling transport-agnostic consumer code. - **`@senzing/sdk`** -- Runtime bindings for the Senzing entity resolution engine. Add records, resolve entities, search by attributes, analyze relationships, and manage configurations. Includes `SzEngineNative` and other adapter classes that implement `@senzing/types` interfaces. - **`@senzing/configtool`** -- Pure JSON manipulation of Senzing configuration documents. No Senzing runtime needed. Works anywhere Node.js runs. - **`@senzing/trpc`** -- tRPC routers that expose the SDK as typed remote procedures over HTTP. Enables browser and remote clients to call Senzing without native libraries installed locally. Includes `SzTrpcEnvironment` for lifecycle management, Zod input schemas, and superjson for bigint flag serialization. - **`@senzing/electron`** -- Electron IPC bridge that runs the SDK in a worker thread and exposes it to the renderer via `contextBridge`. Uses positional args matching the native SDK signatures for seamless integration with Angular or other frontend frameworks. `@senzing/sdk` and `@senzing/configtool` are built with [NAPI-RS](https://napi.rs) (Rust native bindings), providing full TypeScript type definitions generated from Rust source code and prebuilt native binaries for all supported platforms. `@senzing/types`, `@senzing/trpc`, and `@senzing/electron` are pure TypeScript packages. All transports implement the shared `@senzing/types` interfaces, so consumer code can be written once and work with any transport. ## Prerequisites ### Node.js Node.js 18 or later. ### Senzing Runtime (for @senzing/sdk only) The `@senzing/sdk` package requires the Senzing runtime to be installed separately. The npm package contains only the NAPI bridge code (a few MB); the runtime libraries and support data are installed via platform package managers. **macOS (arm64):** ```bash brew install senzingsdk-runtime-unofficial ``` **Linux (x64, arm64):** ```bash # Debian/Ubuntu apt install senzingsdk-runtime # RHEL/CentOS yum install senzingsdk-runtime ``` **Windows (x64):** ```bash scoop install senzingsdk-runtime-unofficial # Ensure Sz.dll is on your PATH ``` **`@senzing/configtool` has no Senzing runtime dependency.** It works anywhere Node.js runs without installing Senzing. ## Installation ```bash npm install @senzing/sdk npm install @senzing/configtool ``` Each package uses platform-specific optional dependencies so npm installs only the binary for your OS and architecture. ## Quick Start ### SDK: Add Records and Search ```typescript import { SzEnvironment, SzFlags } from "@senzing/sdk"; const settings = JSON.stringify({ PIPELINE: { CONFIGPATH: "/opt/senzing/er/resources/templates", RESOURCEPATH: "/opt/senzing/er/resources", SUPPORTPATH: "/opt/senzing/data", }, SQL: { CONNECTION: "sqlite3://na:na@/tmp/senzing.db", }, }); const env = new SzEnvironment("my-app", settings); const engine = env.getEngine(); // Add a record engine.addRecord( "CUSTOMERS", "1001", JSON.stringify({ NAME_FULL: "Robert Smith", DATE_OF_BIRTH: "1985-02-15" }), ); // Search by attributes const result = engine.searchByAttributes( JSON.stringify({ NAME_FULL: "Bob Smith" }), undefined, SzFlags.SEARCH_BY_ATTRIBUTES_DEFAULT_FLAGS, ); console.log(JSON.parse(result)); // Cleanup env.destroy(); ``` ### ConfigTool: Edit Configuration Offline ```typescript import { readFileSync, writeFileSync } from "node:fs"; import { addDataSource, listDataSources } from "@senzing/configtool"; // Load a config JSON (from createConfig() or a saved file) let config = readFileSync("config.json", "utf-8"); // Add data sources config = addDataSource(config, { code: "CUSTOMERS" }); config = addDataSource(config, { code: "WATCHLIST" }); // List all data sources const sources = JSON.parse(listDataSources(config)); console.log(sources); // Save the modified config writeFileSync("config-modified.json", config); ``` ## API Reference ### @senzing/sdk | Export | Description | | -------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `SzEnvironment` | Lifecycle management. Constructor takes module name, settings JSON, and optional verbose logging flag. Provides `getEngine()`, `getConfigManager()`, `getDiagnostic()`, `getProduct()`, `destroy()`, and `reinitialize()`. | | `SzEngine` | Entity resolution operations: `addRecord`, `deleteRecord`, `getEntityById`, `getEntityByRecord`, `searchByAttributes`, `whyEntities`, `whyRecords`, `howEntity`, `findPath`, `findNetwork`, export iteration, and more. | | `SzConfigManager` | Configuration management: `createConfig`, `registerConfig`, `setDefaultConfig`, `replaceDefaultConfigId`, `getConfigRegistry`, and more. | | `SzDiagnostic` | System diagnostics: `checkRepositoryPerformance`, `getRepositoryInfo`, `getFeature`, `purgeRepository`. | | `SzProduct` | Product info: `getVersion`, `getLicense`. | | `SzFlags` | Frozen object containing all flag constants as `bigint` values. | | `SzError` (and subclasses) | Error hierarchy for structured error handling. | Full type definitions are in `packages/sdk/sdk.d.ts`. ### @senzing/configtool All functions are stateless: they accept a config JSON string and return a modified config JSON string (or query results as a JSON string). | Category | Functions | | ------------------ | ---------------------------------------------------------------------------------------------------------------------------- | | Data Sources | `addDataSource`, `setDataSource`, `getDataSource`, `deleteDataSource`, `listDataSources` | | Attributes | `addAttribute`, `setAttribute`, `getAttribute`, `deleteAttribute`, `listAttributes` | | Features | `addFeature`, `setFeature`, `getFeature`, `deleteFeature`, `listFeatures` | | Elements | `addElement`, `setElement`, `getElement`, `deleteElement`, `listElements` | | Rules | `addRule`, `setRule`, `getRule`, `deleteRule`, `listRules` | | Fragments | `addFragment`, `setFragment`, `getFragment`, `deleteFragment`, `listFragments` | | Functions | Standardize, expression, comparison, distinct, matching, scoring, candidate, validation -- each with add/set/get/delete/list | | Calls | Standardize, expression, comparison, distinct calls -- each with add/set/get/delete/list | | Thresholds | `addComparisonThreshold`, `setComparisonThreshold`, `deleteComparisonThreshold`, `listComparisonThresholds` | | Behavior Overrides | `addBehaviorOverride`, `deleteBehaviorOverride`, `getBehaviorOverride`, `listBehaviorOverrides` | | Generic Plans | `cloneGenericPlan`, `deleteGenericPlan`, `setGenericPlan`, `listGenericPlans` | | System Params | `listSystemParameters`, `setSystemParameter` | | Config Sections | `addConfigSection`, `removeConfigSection`, `getConfigSection`, `listConfigSections` | | Versioning | `getVersion`, `getCompatibilityVersion`, `updateCompatibilityVersion`, `verifyCompatibilityVersion` | | Script Processing | `processScript`, `processFile` | Full type definitions are in `packages/configtool/configtool.d.ts`. ## SzFlags Flag constants are `bigint` values because the `WITH_INFO` flag occupies bit 62 (`1n << 62n`), which exceeds `Number.MAX_SAFE_INTEGER`. Using `bigint` uniformly avoids mixing numeric types and is future-proof for additional high-bit flags. ```typescript import { SzFlags } from "@senzing/sdk"; // Use a predefined composite flag engine.addRecord("DS", "1", record, SzFlags.ADD_RECORD_ALL_FLAGS); // Combine flags with bitwise OR engine.getEntityByRecord( "DS", "1", SzFlags.ENTITY_DEFAULT_FLAGS | SzFlags.ENTITY_INCLUDE_ALL_FEATURES, ); // No flags engine.deleteRecord("DS", "1", SzFlags.NO_FLAGS); ``` Common composite flags include `ENTITY_DEFAULT_FLAGS`, `SEARCH_BY_ATTRIBUTES_DEFAULT_FLAGS`, `EXPORT_DEFAULT_FLAGS`, `WHY_ENTITIES_DEFAULT_FLAGS`, and more. See the type definitions for the complete list. ## Error Handling ### @senzing/sdk Errors are mapped to a class hierarchy for `instanceof` filtering: ```typescript import { SzError, SzBadInputError, SzNotFoundError, SzRetryableError, SzUnknownDataSourceError, } from "@senzing/sdk"; try { engine.addRecord("UNKNOWN_DS", "1", record); } catch (e) { if (e instanceof SzNotFoundError) { // Record or entity not found } else if (e instanceof SzUnknownDataSourceError) { // Data source does not exist in the config } else if (e instanceof SzBadInputError) { // Catches all bad input errors (parent of NotFound, UnknownDataSource) } else if (e instanceof SzRetryableError) { // Transient errors that may succeed on retry } else if (e instanceof SzError) { // Base class -- check category helpers console.log(e.isRetryable(), e.isBadInput(), e.isDatabase()); } } ``` **Error hierarchy:** ``` SzError +-- SzBadInputError | +-- SzNotFoundError | +-- SzUnknownDataSourceError +-- SzConfigurationError +-- SzRetryableError | +-- SzDatabaseConnectionLostError | +-- SzDatabaseTransientError | +-- SzRetryTimeoutExceededError +-- SzUnrecoverableError | +-- SzDatabaseError | +-- SzLicenseError | +-- SzNotInitializedError | +-- SzUnhandledError +-- SzReplaceConflictError +-- SzEnvironmentDestroyedError ``` ### @senzing/configtool ```typescript import { SzConfigError } from "@senzing/configtool"; try { addDataSource(config, { code: "DUPLICATE" }); } catch (e) { if (e instanceof SzConfigError) { console.log(e.errorType); // 'AlreadyExists', 'NotFound', 'InvalidInput', etc. } } ``` ## Platform Support | Platform | Architecture | @senzing/sdk | @senzing/configtool | | -------- | ------------ | :----------: | :-----------------: | | macOS | arm64 | Yes | Yes | | Linux | x64 | Yes | Yes | | Linux | arm64 | Yes | Yes | | Windows | x64 | Yes | Yes | macOS x86_64 (Intel) is not supported by the Senzing v4 runtime. ## Thread Safety The Senzing engine is thread-safe. Multiple engine calls can be in-flight simultaneously across libuv worker threads without external synchronization. NAPI-RS automatically schedules synchronous Rust functions on the libuv thread pool so they do not block the Node.js event loop. For heavy workloads in Electron or Node.js, use `worker_threads` with a separate `SzEnvironment` per worker: ```typescript import { Worker } from "worker_threads"; const worker = new Worker("./worker.js", { workerData: { settings }, }); ``` See [examples/worker-threads/](examples/worker-threads/) for the complete pattern. ## Building from Source ### Prerequisites - Rust toolchain (edition 2024, MSRV 1.88) - Node.js 18+ - The Senzing runtime (for @senzing/sdk builds) ### Build ```bash # Build the Rust workspace cargo build --workspace # Build the SDK native module cd packages/sdk && npx napi build --platform cd ../.. # Build the configtool native module cd packages/configtool && npx napi build --platform cd ../.. ``` ### Test ```bash # Run all tests (SDK tests require Senzing runtime) npm test # Run only configtool tests (no runtime needed) cd packages/configtool && npx vitest run ``` ## Documentation Full API reference and guides are published to GitHub Pages via TypeDoc. - [Getting Started](docs/guides/getting-started.md) — Zero-to-first-entity walkthrough - [Config Management](docs/guides/config-management.md) — Configuration lifecycle tutorial - [Error Handling](docs/guides/error-handling.md) — Error hierarchy and patterns - [Deployment](docs/guides/deployment.md) — Docker, PostgreSQL, monitoring - [Code Snippets](docs/guides/code-snippets.md) — 27 runnable snippets covering every SDK operation ## Examples Each example is a self-contained project with its own `package.json`: ```bash cd examples/basic-sdk-usage npm install npm start ``` | Example | Description | Requires Runtime | | ------------------------------------------------ | ------------------------------------------------------------- | :--------------: | | [basic-sdk-usage](examples/basic-sdk-usage/) | Add records, search, entity resolution, export | Yes | | [config-management](examples/config-management/) | Create, modify, register, and activate configs | Yes | | [configtool-usage](examples/configtool-usage/) | Offline config editing with pure JSON | No | | [worker-threads](examples/worker-threads/) | Worker thread pattern for heavy workloads | Yes | | [entity-graph](examples/entity-graph/) | Interactive D3.js entity graph visualization | Yes | | [electron-app](examples/electron-app/) | Electron desktop app with worker threads | Yes | | [trpc-server](examples/trpc-server/) | Express server exposing SDK via tRPC, loads truthset from GitHub | Yes | | [trpc-client](examples/trpc-client/) | Standalone tRPC client — no native SDK needed | No | ## Code Snippets Focused, runnable TypeScript examples for every SDK operation category. See the [full snippet reference](docs/guides/code-snippets.md) for details. ```bash cd code-snippets npm install npx tsx information/get-version/index.ts ``` | Category | Snippets | Requires Runtime | | ----------------------------------------------- | -------------------------------------------------------------------------- | :--------------: | | [Information](code-snippets/information/) | get-version, get-license, check-datastore-performance | Yes | | [Initialization](code-snippets/initialization/) | environment-and-hubs, engine-priming, purge-repository, lifecycle-patterns | Yes | | [Configuration](code-snippets/configuration/) | init-default-config, register-data-sources | Yes | | [Loading](code-snippets/loading/) | load-records, load-with-info, load-worker-pool | Yes | | [Searching](code-snippets/searching/) | search-records, search-worker-pool, why-search | Yes | | [Deleting](code-snippets/deleting/) | delete-records, delete-loop | Yes | | [Redo](code-snippets/redo/) | redo-continuous, redo-worker-pool, load-with-redo, redo-with-info | Yes | | [Error Handling](code-snippets/error-handling/) | error-inspection, retry-with-backoff | Yes | | [Stewardship](code-snippets/stewardship/) | force-resolve, force-unresolve | Yes | | [ConfigTool](code-snippets/configtool/) | basic-usage, process-script | No | ## Repository Structure ``` sz-napi/ Cargo.toml # Workspace root package.json # npm workspace root README.md packages/ types/ # @senzing/types src/ # Shared TypeScript interfaces engine.ts # SzEngine interface config-manager.ts # SzConfigManager interface diagnostic.ts # SzDiagnostic interface product.ts # SzProduct interface environment.ts # SzEnvironment interface sdk/ # @senzing/sdk Cargo.toml # napi-rs crate, depends on sz-rust-sdk package.json sdk.js # Entry point with error wrapping sdk.d.ts # TypeScript definitions src/ # Rust source js/ # JS error hierarchy and wrapper npm/ # Platform-specific packages __tests__/ configtool/ # @senzing/configtool Cargo.toml # napi-rs crate, depends on sz_configtool_lib package.json configtool.js # Entry point with error wrapping index.d.ts # TypeScript definitions src/ # Rust source js/ # JS error mapping and wrapper npm/ # Platform-specific packages __tests__/ trpc/ # @senzing/trpc package.json src/ # TypeScript source router.ts # Combined szRouter with sub-routers context.ts # SzTrpcEnvironment lifecycle wrapper schemas.ts # Zod input schemas for all procedures errors.ts # SzError → TRPCError mapping client.ts # createSzClient() for browser/Node clients routers/ # Per-service routers (engine, configManager, etc.) electron/ # @senzing/electron package.json src/ # TypeScript source main/index.ts # SzElectronEnvironment — IPC handler + worker management main/worker.ts # Worker thread owning SzEnvironment preload/index.ts # contextBridge exposing window.senzing renderer/types.ts # TypeScript declarations for window.senzing shared/errors.ts # Error serialization across IPC boundary examples/ basic-sdk-usage/ # Load records, search, get entities config-management/ # Register data sources, manage configs configtool-usage/ # Edit config JSON offline worker-threads/ # Worker thread pattern for heavy workloads entity-graph/ # Interactive D3.js entity graph visualization electron-app/ # Electron desktop app with worker threads trpc-server/ # Express + tRPC server with truthset loading trpc-client/ # Standalone typed tRPC client code-snippets/ _utils/ # Shared init/cleanup helper information/ # Version, license, performance initialization/ # Environment lifecycle patterns configuration/ # Config creation and data sources loading/ # Record ingestion patterns searching/ # Search and why analysis deleting/ # Record deletion patterns redo/ # Redo processing patterns error-handling/ # Error hierarchy and retry stewardship/ # Force resolve/unresolve configtool/ # Offline config editing (no runtime) docs/ index.md # TypeDoc landing page guides/ getting-started.md # Zero-to-first-entity walkthrough config-management.md # Configuration lifecycle tutorial error-handling.md # Error hierarchy and patterns deployment.md # Docker, PostgreSQL, monitoring code-snippets.md # Snippet reference and usage guide ``` ## License [Apache-2.0](LICENSE)