Count the connections before you buy middleware

Count connections, shared mappings and owning teams before buying an integration platform. A connection matrix and one table choose between direct calls, a hub and an event bus.

Inventory Connection matrix Chooser Direct calls Hub Event bus Shared concerns Ownership map who calls whom, how often five counts auth, retry mapping, monitoring a team per concern
Inventory and connection matrix feed a chooser that splits into direct calls, a hub or an event bus, rejoining at shared concerns.
On this page
  1. Seven rows for Ashgrove
  2. Five numbers off the matrix
  3. One table, read from the top
  4. Ashgrove gets a bus and keeps two direct calls
  5. Four concerns that need a name beside them
  6. The same table at eleven systems

A middleware platform is one more place for every integration to break. With four connections, that’s most of what the license buys.

I still like middleware. It earns its keep once the same address mapping or the same retry policy has been written in several places, by different teams, and nobody can say which copy is right. Before that point it’s a server sitting between two systems that could have talked to each other.

So I count before I look at any product: the connections, the mappings that exist in more than one place, and the teams that own them. To show the counting I’ve invented a distributor, Ashgrove Medical Supply, numbers and all.

Seven rows for Ashgrove

Ashgrove’s inventory is five systems: a Salesforce CRM, an ERP, a web storefront, a support desk and a warehouse management system. The connection matrix gets one row for each thing one system sends another.

id,source,target,carries,per_day,needs_reply,mapping,retry,owner
1,crm,erp,account change,400,no,address normalization,3 tries then drop,sales-systems
2,crm,storefront,account change,400,no,address normalization,none,web
3,crm,support-desk,account change,400,no,none,none,sales-systems
4,erp,crm,order change,2600,no,currency to USD,retry for 1 hour,erp-team
5,erp,support-desk,order change,2600,no,currency to USD,none,erp-team
6,storefront,erp,place order,900,yes,none,caller retries,web
7,erp,wms,release to pick,850,yes,none,caller retries,erp-team

Fill it in from code and logs. The architecture diagram calls rows 1 and 2 “account sync” and stops there. The code tells you each one normalizes addresses with its own function, and that rows 4 and 5 each convert currency from their own rate table.

Five numbers off the matrix

systems           5
connections       7
shared mappings   2   address normalization (rows 1, 2), currency (rows 4, 5)
fan-out           3   account change; order change is 2
owning teams      3

A shared mapping is one transformation that appears on more than one row, counted once. Fan-out is how many targets a single change has to reach.

One table, read from the top

Scope Systems Connections Shared mappings Fan-out per change Owning teams Topology
Whole estate 8 or more 15 or more 4 or more Any 3 or more Integration hub
One change Any Any Any 3 or more, or 2 when both targets repeat a mapping Any Event bus for that change
One connection Any Any This row repeats one 1 or 2 Any Direct call, mapping moved to a shared package
One connection Any Any None on this row 1 or 2 Any Direct call, left alone

This table is the chooser, and the five numbers are its inputs. A row matches when every cell holds, and the first match wins. The thresholds are my own judgment. Nobody measured them.

The hub row asks for all four counts, but shared mappings and owning teams carry it. Lots of connections with nothing duplicated between them justify nothing. Teams count because one team can fix a duplicated mapping with a shared library in an afternoon. Four teams on four release schedules can’t, and a hub gives that mapping a single home with a named owner.

Ashgrove gets a bus and keeps two direct calls

The hub row fails three times over: systems, connections and shared mappings are all under the line.

An account change reaches three systems, so it goes on a bus. An order change reaches two, and both targets convert currency, so it goes on the bus as well. Rows 6 and 7 each wait for an answer, an order number and a pick confirmation. With one target and no shared mapping, they stay as they are.

The Salesforce half of the bus is configuration. Change Data Capture publishes account changes once Account is selected, and Event Relay forwards them to Amazon EventBridge. The ERP publishes its order changes to EventBridge as well. A subscriber that doesn’t live in AWS can read the account events from Salesforce through the Pub/Sub API.

The duplicated mappings move to the publisher. The CRM normalizes an address once, when the account is saved, and stores the result in its own fields, so every subscriber receives the same one. The ERP publishes each order amount in the order currency and in USD. Shared mappings drop to zero, which leaves a hub with nothing to do.

A bus has its own bill. Every subscriber has to survive a duplicate, and one that reads from Salesforce directly has to store its replay position too. The usual ways to get that wrong are in four CRM event anti-patterns. Row 4 still ends in a write to the CRM, so it keeps its line in the API call budget.

Four concerns that need a name beside them

Whatever the table returns, auth, retry, mapping and monitoring still exist. They’ve only moved. Ashgrove’s ownership map after the change:

  • Auth: each subscribing team holds its own credential to the bus, and the two direct calls keep theirs.
  • Retry: the subscriber owns it, with a dead-letter queue per subscriber.
  • Mapping: the publishing team, which makes it sales-systems for addresses and erp-team for currency.
  • Monitoring: one alert on dead-letter depth, routed to the subscriber’s owner.

The same table at eleven systems

Now Ashgrove buys a competitor. A second ERP arrives, along with billing, a second storefront and three more systems. The matrix grows to 19 rows across 11 systems, owned by five teams. Six mappings are shared: the first two, plus a product code crosswalk, units of measure, tax region and a customer ID crosswalk.

The hub row matches first.

The publisher trick stops working here. A product code crosswalk differs for each pair of systems, so there’s no single normalized value to publish. And 19 connections can mean 19 stored credentials and 19 retry policies, spread over five teams. Keeping those in one place is what an integration platform is for.

Only then does the product question come up. If the flows are simple and admins will own them, a no-code tool such as MuleSoft Composer may be enough. If they aren’t, it’s a full iPaaS and a team to run it. The bus stays, and the hub becomes one of its subscribers.

That team is the real price. Every change to every integration now waits in its queue, which is why I wouldn’t pay it at seven rows.