The contract fit databases. The market wanted SaaS APIs.

The situation

The catalog was public and MIT-licensed, and openly soliciting outside contributions. The README said the core team reviewed and merged community submissions daily. Connector breadth was the go-to-market, so every hour a connector took to write, and every connector that shipped untested, was a direct cost to it.

The published destination interface was three methods: connect, disconnect, test the connection. The action vocabulary the early destinations declared showed what it had been shaped for: insertData, updateData, deleteData, executeRawQuery for the warehouse; insert / update / delete for the SQL databases; insertOne, insertMany for the document store. That is CRUD over a table.

A real accounting or commerce API is not CRUD over a table. Xero’s accounting API alone exposes 80 distinct operations; Shopify’s REST surface needed 45, across nested resources, per-resource verbs, positional SDK arguments, per-tenant scoping and aggressive rate limits. Written the database-shaped way, that is 125 hand-written methods across two connectors, 125 documentation pages, and 125 places to get an argument order wrong.

What was actually wrong

The contract was well designed for the assumptions it was written under. The catalog then had to meet a different set.

• The action vocabulary assumed tables, not resources

  Real API surfaces are nested and verb-per-resource: orders.refunds.calculate.create, fulfillment_orders.release_hold.create. There is no row to insert.

• Positional SDK arguments had nowhere to live

  Events carry a flat named payload. Vendor SDK methods take ordered arguments. Something has to know that mapping for all 125 operations, and the contract had no place to put it.

• Multi-tenancy was pushed onto the caller

  A single Xero token can be authorised for several organisations. Nothing in the contract said which one an incoming event meant, so the ambiguity would have surfaced as data written to the wrong tenant.

• The warehouse trusted the payload

  Arbitrary event JSON met typed warehouse columns with no reconciliation step, which is the shape of bug that produces a silently wrong row rather than an error.

• Partial failure had no vocabulary

  The published source return type named the registered webhook data and its events, and said nothing about what happens when you ask for twelve subscriptions and three fail. Unspecified partial failure is how a pipeline loses events quietly.

• Almost nothing in the catalog was tested

  Of the twelve destinations in the catalog, five have a test suite at all. The rest shipped with none: 208 lines of Postgres, 211 of MSSQL, 137 of Firestore, 124 of MongoDB, and more.

How the design was derived

Three structural moves, plus one contract question that was answered in public over five days.

• Reflection instead of enumeration. The platform already had a Proxy-driver pattern, introduced with its database connectors. What it lacked was a way to fill positional arguments from a named payload. The driver reads the SDK function’s own parameter names out of its source and uses them as the mapping:

  methodParams = (Function.prototype.toString.call(targetMethod)
    .match(/\((.*?)\)/)?.[1] || "")
    .split(",").map((param) => param.trim());

  An incominginvoiceID lands in the right slot without anyone hand-writing a signature. 80 actions, one implementation, and no map to keep in sync.
• Multi-tenancy resolved at the driver, not the payload. Connecting enumerates every Xero organisation the token is authorised for; performing an action fans out across all of them and returns a per-tenant response map, catching per tenant so one bad tenant does not fail the batch. The connection test rejects a token scoped to zero or more than one organisation, and builds its error by re-querying the vendor, so a refusal names which organisations are attached and what to do about it.
• Schema-driven serialisation instead of trusting the payload. The BigQuery destination reads the target table’s own schema and coerces each incoming JSON value into a type-correct SQL literal: normalising the temporal types, wrapping geography, casting bytes, emitting JSON literals, and recursing into nested records to build struct expressions. Numeric and boolean fields throw a named schema-mismatch error, so a mismatch surfaces as a failure and never as a wrong row.

The contract question, answered in public

The webhook-lifecycle problem is the clearest artifact in the repository, because it is three dated commits five days apart, open to inspection.

1. 18 Aug

   Naive concurrency

   A Promise.all over every subscription request, with no error handling. Added a catch that rethrows with a descriptive message. One failed subscription now failed the whole init(), and the caller learned nothing about the other eleven.

2. 21 Aug

   Reversed to partial success

   Collect the successes, log the failures, and return the topics that actually registered. That changed what the returned event list means: it now reports reality, so a caller can see the gap and act on it.

3. 23 Aug

   Rate-limit-aware serialisation

   Dropped Promise.all for a serial loop that measures each call and sleeps the remainder of a 500ms floor before the next, because concurrent webhook creation was tripping the vendor. Per-event try/catch keeps failure isolated.

Naive concurrency, then fail-fast, then partial success, then rate-limit-aware serialisation. The end state is slower than where it started and correct, which is the trade the pipeline needed. The intermediate step that was wrong is still in the history.

What was rejected, and why

Several of these were built first and reverted. The reasoning matters more than the fix.

• Writing 125 action methods by hand

  The route every existing destination had taken. Chose reflective dispatch instead: one 60-line performAction bought 80 Xero actions and 45 Shopify ones, and a new vendor operation costs nothing to support.

• A hand-maintained argument map per action

  The obvious fix for positional SDK arguments is a lookup table of action to parameter order. That is 125 entries that go stale the moment the vendor SDK changes. Reading the parameter names off the live function removes the map, and a map that does not exist cannot drift.

• Promise.all for webhook creation

  Rejected twice, for two different reasons: first its error semantics, where one rejection discards eleven successes, then its concurrency, which the vendor rate-limited.

• Fail-fast on partial subscription failure

  Implemented, then explicitly reverted three days later. A source that registered nine of twelve webhooks and reports nine is more useful than one that registered nine and throws.

• Coercing type mismatches into the warehouse

  Numeric and boolean mismatches throw a named error. A wrong row in an analytics warehouse is worse than a rejected one, because nobody goes looking for it.

• Hand-rolled backoff per connector

  Rate limits are not a per-connector problem. Pulled in a retry library and configured exponential backoff on HTTP 429, so Shopify did not need its own copy of the loop.

• Credentials living in test files

  Moved test secrets to environment configuration early, with a checked-in sample carrying empty values. It is why the repository history still scans clean today.

• Linting only my own files

  The narrow, polite option. Configured the shared ruleset and retro-fitted all 25 pre-existing files that violated it. A catalog written by ten people is only consistent if the fix is catalog-wide.

What could not break

This is a public catalog with outside contributors and connectors already in production. The blast radius of a careless change is other people’s integrations.

• The published contract. Every destination declares the platform’s destination interface and every source its integration interface. The contract’s reach grew without its shape changing: new capability arrived through the action vocabulary and the driver layer, so existing connectors and the platform’s driver loader were untouched.
• Other people’s connectors. The lint retro-fit touched 25 files belonging to ten other contributors. That is the kind of change that breaks things quietly, so it shipped as two commits scoped strictly to style plus the ruleset, and nothing else.
• Three incompatible vendor auth models. WooCommerce verifies with a base64 SHA-256 HMAC over the raw body, Adyen with a base64 basic-digest compared against the Authorization header, Xero with an OAuth2 refresh-token exchange plus a JWT decode. All three satisfy the same single-signature webhook verification method.
• A live OAuth2 migration under a shipped connector. Xero changed its token lifecycle while the connector was in production. The rework and its test suite moved in one commit, so there was never a build where the tests described the old flow.
• Ordering and dedupe downstream. The Kafka destination exposes partition, key, timestamp and headers on push. A connector cannot guarantee per-key ordering by itself, but it can refuse to throw away the information the platform needs to.
• Documentation as part of the deliverable. 80 Xero and 47 Shopify per-action documentation pages were written as first-class commits against the catalog’s template. An action nobody can find is an action that does not exist.

Result

Four of the catalog’s twelve destinations and four source integrations, built end to end. Roughly 125 vendor actions covered through a single reflective dispatcher, at about 1.7 lines of test per line of implementation.

• Four of the five tested destinations in the catalog. Test suites were not part of the house style before; they were after.
• 130 of the 245 catalog documentation files, including 80 per-action pages for Xero and 47 for Shopify.
• 25 files brought under one lint contract, across work belonging to ten other contributors, alongside the shared ruleset itself.
• Third by commit count in a repository of 25 contributors, and the heaviest external-facing implementer of the contract.