--- title: "How a connector works" description: "The mechanics of a connector — the outbound half that polls, maps and publishes, the inbound bridge that answers identity questions, and the design rules the code holds you to." canonical_url: "https://openepcis.io/docs/connectors/how-a-connector-works" last_updated: "2026-09-02T11:48:26.079Z" --- ## Two halves, and why they are separate A connector joins one of your existing systems — an ERP, a PIM — to the OpenEPCIS platform, and it does so in two directions that have almost nothing in common. The **outbound half** reads records out of the source system, maps them onto GS1 terms and publishes them into the catalog behind the resolver. The **inbound half** is a small HTTP API, the bridge under `/v1`, which the source system's own user interface calls when it needs an answer the platform holds: is this GTIN sound, whose prefix is it on, what does the linkset currently say, give me the next number. Keeping them apart matters because they have different failure modes and different owners. The outbound half runs unattended on a schedule and must never lose or duplicate a record. The inbound half runs while somebody is looking at a screen, so it must answer in milliseconds, say honestly when it is guessing, and never make a person wait on a registry that has stopped answering. Everything below describes the shared machinery. What each adapter contributes is thin by design: it says which system it speaks for, lists what changed since a cursor, fetches one record by its id, and answers a cheap authenticated ping for the diagnosis ladder. Two further hooks are optional, and an adapter that wants neither is complete without them: writing a value back into the source, and receiving the identity verdict for a record it published. Where the source system offers webhooks, the adapter's module also brings the endpoint that verifies their signatures. The loop itself, the mapping reader, the catalog client, the hashing and the scheduling belong to the shared core and are identical for every source system. ## The outbound half: poll, map, publish ### The loop Per connection and per entity kind, the sequence is always the same: read the cursor, ask the adapter what changed since it, run the record through the mapping, derive the sync key, ask the catalog what it holds for that key, skip the record if the catalog already holds what we would send, otherwise `PUT` it, and advance the cursor. What is *not* in that sequence is a connector-side record of what was published. The loop used to keep one and no longer does: every question it needs answered — has this been published, has it changed — is asked of the catalog, because the catalog is the master data index and a second copy of that truth is a second thing to disagree with it. One row survives, for instance documents alone: the management API takes a `PUT` on `/products/{gtin}/10/{lot}` and offers no `GET`, so for a lot or a serial there is nothing to read back, and without that row every instance would look newly published on every tick. Timing comes from two clocks. A global tick — every ten minutes by default — wakes the scheduler, which then honours each connection's own `poll-interval`, fifteen minutes unless configured otherwise. On top of that runs a nightly full sweep, `0 30 3 * * ?` by default and switchable per connection with `full-sync-enabled`. The sweep is not ceremony: a delta poll only ever sees what the source system remembered to mark as changed, and the sweep is what catches everything that quietly did not get marked. Four properties of the loop are worth knowing before you operate one. Each record's cursor advance commits in its own transaction, immediately after that record's write, so trouble late in a batch keeps the successes before it. A publish failure stops that entity kind rather than the whole connection, and it leaves the cursor where it was, so the next tick simply retries from the failed record. An invalid key skips one record rather than sinking the batch around it: the cursor moves past it, and the record comes back when it is next modified, or with the nightly sweep. And in dry run — which is where every connection starts — nothing is written anywhere: no `PUT`, no stamp, no commissioning event, only a log line naming the path that would have been written. Arming the connection afterwards therefore still publishes everything the catalog does not already hold; the rehearsal consumed none of it. ### Delta, cursor, and what a cursor actually is The delta is taken over whatever the source system offers as an "updated at" notion, and that differs enough between systems that the cursor is deliberately the adapter's business rather than the core's. UnoPim is filtered on `updated_at`; because it pages by id rather than by timestamp, the cursor is kept as a running maximum. ERPNext keeps the last `modified` timestamp it saw per DocType, because one catalog kind can be fed by several of them — an organization comes from both `Supplier` and `Customer`, and each pages independently. metasfresh treats the `since` value only as an optimisation hint and stamps the poll's own start time, in epoch milliseconds, as the cursor. ### The hash skip Most records that come back from a delta poll have not changed in any way that matters to the catalog: a PIM touches `updated_at` when somebody opens and saves a product without editing a field. Publishing those anyway would fill version histories with noise and spend write capacity on nothing. So before publishing, the connector fetches the catalog's own document for that key and compares it with the payload it is about to send — canonical JSON with map keys sorted, SHA-256 over that. Identical means nothing is sent. Two details make that comparison safe to trust. It is **projected**: only the keys this connection's mapping produces are compared, because a catalog record also carries another connector's contribution, an editor's correction and whatever a registry enriched it with, and comparing the whole document would call every record changed on every tick. And numbers are **normalised** before hashing, in two senses — `30` and `30.0` are the same number, and 250 GRM and 0.25 KGM are the same weight. Both of those were real faults here, each one making a record differ from its own stored copy forever. Alongside the comparison the connection leaves a **stamp**: the hash of exactly what it last published, written into the record's `connectorStamps` field and keyed by connection name, so several connections can stamp the same record without erasing one another. The stamp answers a different question from the comparison — not *did anything change* but *which side changed it* — and the next section is where it earns its keep. It lives in the catalog rather than in a table on the connector's side on purpose, for the same reason the loop keeps no state of its own. ### Publish is a PUT, and that is a correctness property Publishing is a `PUT`, which the catalog treats as create-or-update and merges rather than replaces. That single choice is what makes the whole retry story safe. Because a repeated write of the same document cannot duplicate anything, the loop is free to leave the cursor standing after a failure and simply try again on the next tick — no bookkeeping of "did that one get through", no compensating deletes, no window in which a crash produces two of something. ### What publishing does not do Two things the loop deliberately leaves alone, and both surprise people who expect a sync to be symmetrical. It never deletes. A delta poll asks the source for records changed since the cursor and therefore never sees one that has been removed; a webhook for a record the source API no longer knows is logged and the catalog document is kept, on the grounds that removing master data is a deliberate act and not a side effect of a delivery that arrived late. The same holds one level up: when a source stops offering a link — a withdrawn passport page in the PIM, say — the connector registers the shortened list rather than retracting what it registered before, so the old entry stays in the linkset and keeps pointing at a page that is no longer serving data. Undoing any of that is manual work on the catalog and linkset APIs. A product that is merely deactivated in the source is not affected at all: its status travels into the payload, nothing reads it, and the product keeps being published. And it does not produce a digital product passport. The passport is computed by a separate service, the **DPP API** — its own application with its own address, neither part of the resolver nor ever called by the connector. Its interface is EN 18222 and the passport it returns is EN 18223; the two numbers are frequently conflated, including in our own code comments. Configured with `dpp.masterdata.backend=resolver`, the DPP API derives the passport on read out of exactly the GS1 master data the connector wrote, which is what makes a create call unnecessary; on its built-in default backend it knows only what was written into it through its own create call, which the connector never makes, and every read ends in a 404. The resolver, for its part, only advertises a `gs1:dpp` link when it has been told where that service lives, through `openepcis.digital-link-resolver.dpp-base-url` (environment variable `OPENEPCIS_DLR_DPP_BASE_URL`), which is empty by default and belongs to the deployment rather than to any address fixed in the code. One derivation decides whether a passport exists at all. `economicOperatorId` is mandatory, and it is derived from `gs1:manufacturer` on the product record — either the organization's own party Digital Link or one built from its GLN. None of the shipped default mappings set a manufacturer, and organizations are synchronised as records of their own without the product pointing at them, so out of the box the derivation has nothing to work with and the read fails rather than returning a passport with a hole in it. What a first sync does give you is the whole GS1 master data document in the catalog, a resolving Digital Link and its linkset. Getting a passport on top of that means two deliberate steps: a mapping rule of your own onto `manufacturer`, which the connector vocabulary knows and types as an organization, and the DPP API pointed at the resolver backend. ### The webhook is a doorbell, never a data source Where a source system can call out on change, the connector accepts that call — but only as a signal. The payload is read for one thing, the identity of the record that moved. Then the record is fetched fresh from the source API and enters exactly the same loop the poll enters, at the same function. This is not a fastidious detail; it is the difference between a connector you can reason about and one you cannot. A webhook body is a snapshot of somebody else's transaction at a moment you did not choose, and it goes wrong in ways that are invisible from your side. Measured against UnoPim 3.0.0: creating a product fires two identical deliveries, `product.created` and `product.updated`; a failed delivery is never retried, because the delivery service swallows its own error; and the create payload omitted the `dpp_gtin` that had been set in the very same request. Treat any of that as data and the catalog inherits it. Treat it as a doorbell and none of it can hurt you — the worst a bad payload can do is cause one unnecessary re-read. The second half of the rule matters as much as the first: poll and webhook share one code path. If they did not, every fix, every mapping change and every edge case would need to be got right twice, and the rarely-exercised path would be the one carrying the bug. Signatures are checked before anything else — HMAC-SHA256, hex or Frappe's base64 variant, `X-Frappe-Webhook-Signature` for ERPNext and `X-Unopim-Signature: sha256=` for UnoPim — and an unsigned or wrongly signed call gets a 401 and no re-read. ## The mapping is data, not code Everything else in a connector — authentication, paging, webhooks — is code that rarely changes. The part that differs per customer is which field over there becomes which GS1 term over here, and that part is a YAML file, one per connection, so that adapting an installation is a review of one readable document rather than a release. It is also the only file involved: everything else a connection is configured with — its credentials, its intervals, its locale, its `dry-run` — are Quarkus properties under `openepcis.connectors.connections..*`, handed to the container as environment variables. Which mapping a connection reads is one of those properties, `mapping-file`, and it takes a path; the Kubernetes deployment mounts no volume for it today, so a connection running there uses the mapping the build ships and a customer-specific file has no way in yet. A file holds one document per entity family — `product`, `lot`, `serial`, `lot-serial`, `organization` — and each document is a list of rules: ```yaml system: unopim entity: product description: Default mapping for UnoPim products. rules: - source: name target: productName kind: localized direction: both description: The merchant's product name, maintained in either system. ``` Both sides of a rule are dotted paths. `brand.brandName` addresses a nested object; missing intermediate objects are created on write, and a missing one on read is simply an absent value rather than an error, because source systems are sparsely filled. A target segment may end in `[]`, which fans the value out into a list element, so that `targetMarket[].targetMarketCountries.countryCode` produces the one-element list a repeatable GS1 term expects from a source that holds a single value. `kind: localized` files a value under a language key, so a bare `"Leinenhemd"` becomes `{"de": "Leinenhemd"}` under the language the connection reads its values in — `source.options.locale`, with the region dropped rather than invented, so `de_DE` files under `de`. Without a locale, every translation the source itself already delivers as a language map survives: such a value passes through untouched and the rule works with no locale configured at all, which is what lets one rule serve a single-language ERP and a multilingual PIM. Only a value the source delivers as a single text without a language needs the locale. If it is missing, the connector says so loudly for that record, naming the rule and the setting, rather than filing text under a blank language key the catalog would reject — that record is not published and the rest of the run carries on. Whether a given ERP ever delivers language maps is a property of that system and not something the connector can promise; for a source that only ever sends plain text, a locale is in practice required as soon as a `localized` rule applies. A `unit:` stamps a UN/CEFACT code onto a bare number for sources that keep the unit somewhere other than the data; leave it off and a source that carries its own unit has that unit converted into whatever the catalog holds, against the platform's own Recommendation 20 table of 721 units across 51 dimensions. There is deliberately no second table — it is the same table the EPCIS repository converts sensor values against — and a conversion between different measurands fails loudly rather than being approximated, as does a unit the table does not know. A `unit:` on a `kind: localized` rule is refused outright: text has no unit. Unknown keys are rejected the moment the file is read, at both levels. A misspelled `direciton:` that silently fell back to the default would be worse than an error, because the failure would be a field quietly not travelling. ### `direction`, and why it is declared `direction` decides who owns a field. `to-catalog` is the default: the source system owns it, it travels one way, and the source is never written into — writing into a customer's system is the act that should need asking for, not the act that happens unless forbidden. `from-catalog` means the platform owns it: it is skipped entirely when the outgoing document is built, and written into the source instead — by those adapters that accept a write-back at all, since an ERP is usually the authority for what it sends and simply never opts in. That is the right direction for everything the platform derives rather than receives. `both` means the field is genuinely maintained in either place. For `both`, which edit is more recent is not decided by timestamp. Two systems have two clocks and two notions of "modified", and a PIM that touches `updated_at` on every save would win every race whether or not anything changed. The stamp decides instead. If the catalog matches what we would send, nothing moved. If the catalog differs but the stamp still matches what we would send, the source has not moved since the last publish, so the change happened in the platform: the value travels back into the source and nothing is published. If the catalog differs and the stamp does not match, the source moved, and it publishes. The write-back is best effort — the catalog is correct either way, and a PIM that refuses a courtesy write must not stop a publication. This is why the key is declared and not inferred, and the reason is worth stating plainly because it was learned the hard way. Direction used to be derived from the shape of a rule — reversible if it neither fanned out nor carried a unit — which quietly made every ordinary rule bidirectional. Both systems then claimed the same field, and on each tick the publish overwrote the catalog edit that the pull-back had just carried into the source. Nothing failed, nothing was logged: the correction simply disappeared and both sides settled on the stale value. Two rule shapes cannot run backwards at all, and the file is refused rather than having the direction quietly ignored: a target that fans out has no single element to write back to, and a rule with a fixed `unit` has thrown away which unit the source meant. Both are rejected at read time, naming the rule. Identifiers deliberately do not appear in this file. The GTIN, the GLN and the batch identifier are read into the sync key by the adapter, because they say *which* record this is rather than what it holds, and the catalog derives the Digital Link from the key itself. The JSON-LD `type` is likewise stamped on the way out. And every target name is checked against a vocabulary index built from the GS1 web vocabulary and the platform's own regulation extensions — a target no vocabulary knows would be dropped silently by the resolver, so the build fails instead. ## The inbound half: the bridge under `/v1` The bridge is what an adapter, or a plugin inside the host system, calls when it needs an answer rather than wanting to deliver data. Seven endpoints:
Endpoint What it answers
POST /v1/products/resolve A normalised payload in, an identity verdict out
GET /v1/products/{gtin}/linkset The current linkset ( ?lot= and ?serial= narrow it)
POST /v1/products/{gtin}/links Register or update link-type targets
POST /v1/identifiers/allocate Draw a key from the tenant's own prefix
GET /v1/gpc Search the GPC classification by words
GET /v1/tenants/{id}/quota Allocation counts by state and prefix
GET /v1/health What this bridge is configured to serve
A tenant here is a configured connection, and its name is the tenant id; every call names one, in the body or as `?tenantId=`, and there is no second tenancy concept. An unknown tenant is a 404 rather than an empty result, so a misconfigured adapter finds out immediately instead of quietly getting nothing. The bridge is not published outside the cluster: callers reach it over the cluster-internal service address on the connector's own HTTP port, which is why a plugin inside a host system is configured with that internal address rather than with a public URL. Allocation is worth one extra word, because it is the one call with a consequence outside the platform. A drawn key is held as a candidate in the resolver's ledger and is not registered with GS1 until the record using it is saved — registration upstream cannot be undone, and an abandoned form would otherwise burn a number permanently. The payload is **host-agnostic**. Nothing in it knows what a UnoPim value section or an ERPNext DocType is — adapters flatten their own shape into a `NormalizedProduct` first. That is precisely what lets one bridge serve a PIM and an ERP without a branch per host system, and it is why a new adapter gets the whole inbound half for free. **No GS1 logic is reimplemented here.** The identifier grammar comes from the shared `aitable.json` of the Digital Link translator toolkit — 477 application identifiers with their qualifier chains, formats and check-digit positions — and the prefix boundary from that same toolkit's GCP Length Table. Verified by GS1, linkset management and key allocation are calls to the resolver, which already holds the credentials and the ledgers; registry credentials are never held in the connector at all. What the bridge adds is only four things: the normalised payload, the verdict shape, per-fact provenance and the cache policy. ### The shape of a verdict A verdict is deliberately more than a boolean: ```json { "ai": "01", "value": "9521123456785", "canonicalKey": "09521123456785", "digitalLink": "https://id.example.test/01/09521123456785", "syntacticallyValid": true, "checkDigitValid": true, "offlineValid": true, "gcpLength": 7, "gcpOwnership": "foreign_prefix", "registryStatus": "verified", "licenseeName": "Muster Handel GmbH", "reason": null, "dataProblem": false, "degraded": false, "provenance": { "syntax": {"source": "offline", "observedAt": "..."}, "registry_status": {"source": "cache", "observedAt": "..."} } } ``` `provenance` is per fact, because a verdict is not one fact from one place. A cached entry carries the timestamp of the **registry call**, not of the cache hit, so what a person sees is the age of the knowledge rather than the age of the answer. A silent cache hit that somebody believes is a live lookup is exactly the trust failure this design argues against. Cache lifetimes are per fact and per tenant: licensee identity changes rarely and is kept 75 days, descriptive attributes read from the registry 30, licence status — the volatile one, where a stale "active" is the most misleading thing this cache could serve — 10, and a negative result only 2, because a GTIN GS1 does not know today is very often one that gets licensed next week. Partitioning by tenant is a confidentiality rule rather than a performance detail: a fact read under one tenant's licence is never served to another, and there is no global fallback lookup anywhere in the cache. The store is per process and bounded, so several replicas cost a repeated lookup after a restart — never a wrong or cross-tenant answer. ## The design rules the code enforces Three rules recur throughout, and each of them is held by the code rather than by discipline. **Readiness informs, never blocks.** Downstream registries have their own expectations — GS1 Germany wanting a GPC code, for instance — and the platform reports them per channel and per entity kind as `requiredTerms`, for whichever channel a connection publishes into. That information is advisory: it feeds "still needed" hints in a screen or a report. An unmet requirement never holds data hostage, because a connector that refuses to publish until a record is perfect is a connector that publishes nothing on the day it is most needed. **A data problem and our problem are different types.** Nothing caused by an outage, a credential or a quota may present itself as bad customer data. `dataProblem` is true only when a person editing the record can fix what is wrong: bad syntax, a failed check digit, an AI that admits no such value, a qualifier its primary key does not take, or GS1 answering that it does not know the key. It is false for every outage, credential and quota reason. Branch on `dataProblem`, never on `registryStatus` — and the rule is drawn once in the code rather than in each caller: the reason code itself carries the flag, and the exception raised when a registry lookup does not complete refuses to be constructed with a reason that claims to be a data problem. **Degraded mode.** When the registry or the resolver does not answer, when a credential is missing or rejected, or when the tenant's allowance is spent, the verdict comes back with `registryStatus: "unavailable"`, `degraded: true` and a `reason` — and every offline fact still standing. Syntax, check digit, GCP boundary and the canonical Digital Link are computed before anything can fail, so an outage costs you the registry's opinion and can never cost a merchant their good GTIN. One operational default belongs beside these. A connection starts with `dry-run` set to `true`, and arming it is a deliberate single line. Behind the catalog sit real registrars, GS1 production among them, and a rehearsal that quietly wrote would be worse than no rehearsal at all. ## Where the boundaries are Not every source system can be joined the same way, and the reason is licensing rather than engineering. metasfresh is GPL-2.0-only, so the connector couples to it across the network and nowhere else: REST v2 is the only interface, there is no metasfresh dependency in any Maven scope, and nothing is copied in — not code, not schemas, not field lists, not sample payloads. The admissible sources of knowledge are the published documentation, the Swagger description of a running instance, and our own observation of HTTP traffic. This is written down as an invariant for every future change rather than as a description of the current state. Odoo goes the other way. It brings its own runtime, its own packaging and its own licence world, so the Odoo connector is a separate repository of Odoo addons under LGPL-3 rather than a module here. Its GS1 helper code is written clean-room against the GS1 General Specifications and shares nothing with the server-side components, which is what lets the addons carry that licence without qualification. The shared core and the bridge stay proprietary. Adapters are distribution rather than intellectual property, and may be published.