Putting an LLM inside the CRM, where the work already happens

An LLM feature in its own tab goes unused. Put it on the record the rep already has open, then deal with the permission and review questions that come with that.

Record Context Model Draft Approve the rep is here
The model comes to the record the rep is already on, and the rep has the last word.
On this page
  1. One click on the record
  2. Small, deliberate context
  3. Fetch it as the rep
  4. A gateway in the middle
  5. The model drafts, the rep sends
  6. Prompts are code, and seconds count
  7. Measure it against a comparison group

The easiest way to build an LLM feature nobody uses is to give it its own tool. Sales reps live in the CRM. They have the account open and a call coming up. If the help is one tab away, most of them will never open the tab.

So I start from the other end. The model comes to the record the rep is already looking at, and most of the design follows from that one choice.

One click on the record

When this works, the rep clicks once on the page they’re viewing, gets a useful draft in a few seconds, fixes two lines and moves on. Nobody thinks about the model. The tedious part of the task just got shorter.

Underneath, the flow is simple. The rep is on an account, an opportunity or a case and asks for something specific, like a summary of the account or a prep sheet for a call. The system pulls what’s relevant to that record: recent activity, open deals, notes, past emails. A prompt built from a tested template and that context goes to the model through a gateway. The answer lands back on the page as a draft, clearly marked as generated, and the rep decides what to do with it. Nothing is saved or sent until they do.

Every one of those steps hides a decision that a prototype tends to get wrong.

Small, deliberate context

The model’s general knowledge is close to worthless here. What it’s good at is reading twenty activities and three email threads faster than any person can. That makes the context you assemble the actual feature, so build it on purpose and keep it small. More context isn’t better context. Past a point it just buries the one email that matters.

This is the prompt shape I keep coming back to:

You are helping a sales rep prepare for a call.
Use only the facts below. If something is missing, say so.

Account: {name}, {industry}, {employee_count} employees
Open deals: {open_opportunities}
Last 5 activities: {recent_activities}
Last email from customer: {last_inbound_email}

Write: 3 bullet points to open the call, and 1 risk to watch.

Nothing in it asks the model to know anything. Every field comes from the rep’s own data, and the second line gives it permission to say a fact is missing instead of inventing one.

Fetch it as the rep

Context has to be gathered as the rep, under the same record sharing and field-level security as the page they’re on. If a rep can’t see a record or a field, the model must not see it on their behalf. On Salesforce that means user-mode queries, since with sharing alone doesn’t check field access.

Prototypes rarely get this right, and it’s what stalls a launch. Somebody in security review will ask whether the assistant can show a rep anything the record page wouldn’t. A prototype that reads data through an integration user with broad access has no good answer.

I’d rather pay for this on day one. Retrofitting user-context queries onto a pipeline that was built with admin access is miserable work.

A gateway in the middle

Between the CRM and the model I want one service that owns authentication, rate limits, redaction of sensitive fields, logging and model choice. The CRM side calls the gateway and has no idea which model answered.

That separation pays for itself the first time you want to switch models or add a safety check. You change one service. The CRM code doesn’t move.

The model drafts, the rep sends

Drafts are cheap. An email that reaches a customer under a rep’s name isn’t. For anything that leaves the building, the model proposes and the person decides.

The interface matters more than people expect here. If accepting is one click and editing means copying text into another box, reps will accept drafts they should have fixed. Make editing as easy as accepting.

Prompts are code, and seconds count

Templates should be versioned, reviewed and tested against a fixed set of real records before release. A small wording change can move output quality more than a model upgrade, which is reason enough to stop editing prompts live in production.

Latency gets less attention than it deserves. A rep will wait a couple of seconds. They won’t wait twenty. Stream the response, load the pieces of context in parallel, and cache whatever doesn’t change within a session.

Measure it against a comparison group

Usage on its own says very little. Someone clicking the button once tells you they were curious. The measures I trust sit closer to the work:

  • How often a draft is accepted with light edits, heavily rewritten, or thrown away.
  • Time spent on the task with and without the feature, compared across similar groups of reps.
  • Whether the saved time shows up somewhere real, such as more customer conversations per week.

Run it as an experiment before anyone claims a productivity number. At LinkedIn I led the GPT-based model inside Salesforce. Whatever you build, agree on the comparison before launch, so the number is settled before anyone has a stake in it.

A measured gain is what funds the next feature.