Open an onboarding record and the status field gives you a step name. How long it has sat on that step, and whether the job meant to move it still exists, the record can’t say.
That second gap is where long GTM processes go wrong. Onboarding a customer or approving a big deal is one process, but it usually gets built as a relay. A trigger fires on closed-won and calls billing. A webhook from billing flips a field. A scheduled job looks for records in that state and sends a reminder. Every handoff is reasonable on its own. The process as a whole exists nowhere, so nobody can ask it a question.
Greyfriars Payroll and the 38 that sat
Greyfriars Payroll is a payroll vendor I’ve made up. It onboards each new customer in nine steps over about 17 days. Two of the steps are mostly waiting: a credit check that can take up to 48 hours, and a countersignature that can take up to 10 days, with reminders on day 3 and day 7.
In the trigger-chain version, 38 of last quarter’s 610 onboardings stalled for more than five days before anyone noticed. That’s about 6%, and each one was a customer who had signed and then heard nothing. The causes were dull. A webhook arrived while the record was locked. A reminder job filtered on a status value someone had renamed. And one deploy replaced the scheduler that held the countersignature reminders, which wiped 11 pending timers. Those customers never got a day 7 nudge, and nothing errored, because a timer that no longer exists can’t fail.
The position lived in a CRM field, the timers in a scheduler and the retries in each integration’s own code. No single system knew enough to notice.
What the engine holds that a status field can’t
A durable workflow engine records every step of a run as an append-only history: activity scheduled, activity completed, timer started, signal received. Your workflow code is replayed against that history after a crash or a deploy, so a restart resumes at the same step with the same timers still pending. The day 7 reminder is a row in the engine’s database. It survives your release.
Everything that touches the outside world goes through an activity, and the engine owns the retry policy and the timeout for each one. People enter the process as signals: the countersignature and the go-live approval are messages sent to a running workflow, which waits for them without holding a thread or a server.
The CRM still shows a status, but it’s a copy. One small activity writes it after each step so account teams can see progress on the record. If that write fails, it retries like any other activity, and the engine’s history is still the record of where the run is.
The definition, in pseudo-code
This is pseudo-code, deliberately not tied to one engine’s SDK. The structure carries over to any of them.
workflow CustomerOnboarding(opportunityId):
id = "onboarding-" + opportunityId # also the workflow ID: a duplicate event can't start a second run
done = [] # completed steps, kept for compensation
run(step):
activity(step, idempotencyKey = id + ":" + step.name,
retry = step.retry, timeout = step.timeout)
done.push(step)
activity(writeCrmStatus, id, step.name)
try:
run(creditCheck) # result can take up to 48 hours
run(sendContract)
sentAt = now()
signed = false
for day in [3, 7, 10]:
signed = awaitSignal("countersigned", until = sentAt + days(day))
if signed: break
if day < 10: run(reminder(day))
if not signed: fail("countersignature timed out")
parallel: # the branch in the diagram
run(createBillingAccount)
run(provisionTenant)
run(importPayrollData)
awaitSignal("goLiveApproved") # a person: page them, don't time out
run(activateSubscription)
run(handOverToSupport)
catch failure:
for step in reverse(done):
if step.compensation:
activity(step.compensation, idempotencyKey = id + ":undo:" + step.name)
activity(writeCrmStatus, id, "failed at " + failure.step)
rethrow
The idempotency key is the workflow ID plus the step name. A retried billing call presents the same key, and the billing system returns the account it already created. Without that, retries are how you get two tenants and two invoices.
The code is the short half. The step table is where the design arguments happen, and I’d write it first.
| Step | System called | Retry policy | Timeout | Compensation if a later step fails | Who is paged |
|---|---|---|---|---|---|
| 1. Credit check | Credit bureau API | 5 tries, backoff from 1 min | 48 h for the result | None, read-only | Finance ops |
| 2. Send contract | E-signature service | 5 tries, backoff from 30 s | 10 min | Void the envelope | GTM systems on-call |
| 3. Countersignature wait | None, waits for a signal | No retries, reminders on days 3 and 7 | 10 days | Void the envelope | Account executive |
| 4. Create billing account | Billing system | 8 tries, backoff from 1 min | 15 min | Close the account | GTM systems on-call |
| 5. Provision tenant | Provisioning service | 8 tries, backoff from 1 min | 1 h | Suspend the tenant, keep data 30 days | Platform on-call |
| 6. Import payroll data | Import service | 3 tries, then stop | 24 h | Delete the imported rows | Onboarding specialist |
| 7. Go-live approval | None, waits for a signal | No retries | None, page after 2 business days | Nothing to undo | Onboarding manager |
| 8. Activate subscription | Billing system | 8 tries, backoff from 1 min | 15 min | Cancel it and credit any invoice | Finance ops |
| 9. Hand over to support | CRM and ticketing | 5 tries, backoff from 30 s | 10 min | Nothing, it’s the last step | GTM systems on-call |
The paging column gets argued about more than any other, because every name in it is someone agreeing to be woken up.
Stalled onboardings become a query
If each run tags itself with its current step and the time it entered that step, “which onboardings have been stuck for five days” is a filter on the engine’s list of open runs. In Temporal, with a custom search attribute for the step’s start time, the list filter looks roughly like this:
WorkflowType = 'CustomerOnboarding'
AND ExecutionStatus = 'Running'
AND StepEnteredAt < '2024-10-01T00:00:00Z'
Greyfriars runs that every morning and posts the result to the onboarding channel. The 38 would have surfaced on day six, with the step name attached.
Picking an engine, and what you sign up for
I’d choose by where the steps live. Salesforce Flow Orchestration is a fair fit when most steps are people working in the CRM and the outside calls are few. At Greyfriars seven of the nine steps call something outside the CRM, so I’d run the workflow outside it. Temporal suits teams who want the definition as ordinary code with tests. AWS Step Functions suits teams already on AWS who are happy with a state machine definition, and its Standard workflows can run for up to a year, which covers any onboarding I can imagine.
Whichever you pick, start it from a platform event on closed-won. A small starter subscribes to the event and does one thing, which is start the run. Treat the event as at-least-once. The workflow ID derived from the opportunity is what makes a redelivery harmless. I wrote about the event side in four CRM event anti-patterns.
There are real costs. You now operate an engine or pay someone to. And a 17-day process means every deploy happens while old runs are in flight, so changes to the workflow have to be versioned: runs that began on the old definition finish on it. Teams used to shipping a trigger change on a Friday find this slow at first.
In the Greyfriars trigger chain the paging column was blank on all nine rows. No stuck onboarding belonged to anyone, and that’s how 38 customers waited more than five days to hear anything.