Node.js Semantic Search: Embeddings, Rerank, and Classify Docs by Topic

Short answer: for business-defined tags, use embeddings to retrieve a small set of taxonomy or policy passages, rerank those passages against the sales-call summary, and ask an LLM classifier for structured JSON using only the best guidance. This is a better portability boundary than putting the entire taxonomy handbook into one provider-specific prompt.

The concrete job here is turning a sales-call summary into CRM actions: a topic label, a follow-up owner, and a next-step tag. The hard part isn't producing JSON. It is making sure that a label such as security_review means the same thing after the taxonomy changes, the model changes, or the team routes inference to another provider.

Keep the definitions outside the prompt template.

Put taxonomy changes on a migration clock

The simple approach is one large prompt containing every label definition and every exception. It can ship quickly, but it couples three things that change at different speeds: the application instruction, the business taxonomy, and the model provider. A revised exception for an enterprise security review forces the full handbook back into every classification request. It also spends tokens on labels that have nothing to do with the current call.

The experiment replaces that bundle with a narrow sequence. First, store policy and taxonomy documents as embeddings. Next, retrieve passages that are semantically close to the call summary. Then rerank that candidate set against the actual classification question. Finally, send the strongest guidance snippets to chat completions and require structured JSON labels for the CRM. Retrieval narrows the field; reranking decides which definitions deserve scarce prompt space.

That sequence matters when two labels share vocabulary. Consider a call that includes “security questionnaire,” “legal review,” and “pilot access.” Embedding similarity may retrieve definitions for security_review, procurement, legal_review, and technical_validation. The reranker gets a more precise job: order those candidate definitions by how directly they govern the call. The classifier then reasons from a few relevant rules rather than guessing from label names or scanning an unrelated handbook.

Provider portability is the evaluation constraint, not an afterthought. The application should own a small contract for embed, rerank, and classify; provider-specific model IDs, credentials, and response adapters stay behind that contract. Switching providers should change an adapter and its evaluation results, not the CRM schema or the taxonomy documents. I wouldn't call the system portable merely because two vendors accept similar chat messages — embeddings and reranking behavior can still move the decision boundary.

The options differ mainly in how much integration ownership a solo team accepts:

No option removes evaluation work. The table is an ownership decision, not a winner board.

How should Node.js semantic search use embeddings and rerank docs before an LLM classifier?

Start with a stable application object. A retrieved passage should carry an internal ID, the taxonomy text, and a retrieval score. After reranking, preserve the passage ID and add its rerank score; don't let provider response objects leak into the CRM writer. The final classifier result should use a schema the application controls.

This focused TypeScript example shows the provider boundary around an already reranked candidate set. Retrieval and rerank adapters should populate that set; their request fields and model IDs belong in adapter tests rather than being guessed in application code. The final step makes a real OpenAI-compatible Infrai chat-completions request, uses model: "auto", and asks for a structured JSON object. Set INFRAI_BASE_URL and INFRAI_API_KEY in the environment before running it.

type RerankedGuidance = {
  id: string;
  text: string;
  rerankScore: number;
};

type CrmClassification = {
  topic: "security_review" | "technical_validation" | "other";
  followUpOwner: "account_executive" | "solutions_engineer";
  nextStep: string;
  guidanceIds: string[];
};

type ChatCompletion = {
  choices?: Array<{ message?: { content?: string } }>;
};

const callSummary = [
  "The buyer requested a security questionnaire.",
  "A solutions engineer must confirm SSO requirements before pilot access.",
].join(" ");

const rerankedGuidance: RerankedGuidance[] = [
  {
    id: "taxonomy-security-review",
    text: "Use security_review when access depends on a security questionnaire.",
    rerankScore: 0.93,
  },
  {
    id: "ownership-sso",
    text: "Assign SSO requirement validation to the solutions engineer.",
    rerankScore: 0.88,
  },
  {
    id: "taxonomy-technical-validation",
    text: "Use technical_validation for a pilot with unresolved technical fit.",
    rerankScore: 0.61,
  },
];

function buildClassifierInput(
  summary: string,
  guidance: RerankedGuidance[],
): { instruction: string; summary: string; guidance: RerankedGuidance[] } {
  return {
    instruction:
      "Return one CrmClassification JSON object. Use only the supplied guidance.",
    summary,
    guidance: guidance.slice(0, 2),
  };
}

const input = buildClassifierInput(callSummary, rerankedGuidance);

function requiredEnv(name: "INFRAI_BASE_URL" | "INFRAI_API_KEY"): string {
  const value = process.env[name];
  if (!value) throw new Error(`${name} is required`);
  return value.replace(/\/$/, "");
}

async function postJson<T>(path: string, body: unknown): Promise<T> {
  const baseUrl = requiredEnv("INFRAI_BASE_URL");
  const apiKey = requiredEnv("INFRAI_API_KEY");

  for (let attempt = 0; attempt < 4; attempt += 1) {
    const response = await fetch(`${baseUrl}${path}`, {
      method: "POST",
      headers: {
        Authorization: `Bearer ${apiKey}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify(body),
    });

    if (response.status === 429 && attempt < 3) {
      const retryAfter = Number(response.headers.get("Retry-After"));
      const delayMs = Number.isFinite(retryAfter)
        ? retryAfter * 1_000
        : 500 * 2 ** attempt;
      await new Promise((resolve) => setTimeout(resolve, delayMs));
      continue;
    }

    const responseBody: unknown = await response.json();
    if (!response.ok) {
      throw new Error(`Request failed with ${response.status}: ${JSON.stringify(responseBody)}`);
    }
    return responseBody as T;
  }

  throw new Error("Rate-limit retry budget exhausted");
}

const completion = await postJson<ChatCompletion>("/v1/chat/completions", {
  model: "auto",
  messages: [
    { role: "system", content: input.instruction },
    { role: "user", content: JSON.stringify(input) },
  ],
  response_format: { type: "json_object" },
});

const content = completion.choices?.[0]?.message?.content;
if (!content) throw new Error("The classifier returned no JSON content");

const classification = JSON.parse(content) as CrmClassification;
console.log(JSON.stringify(classification, null, 2));

The scores above are example fixture data, not benchmark results. Their purpose is to make ordering visible in a test. In production, choose the candidate count and final context count from an evaluation set, not from these numbers. Your mileage may vary when taxonomy passages are long, repetitive, or written by several teams.

The adapter layer also needs boring operational discipline. Every request should set its HTTP method explicitly and send credentials as Authorization: Bearer <key>, with the actual key read from an environment variable. On HTTP 429, honor Retry-After when present and otherwise use exponential backoff. Surface 4xx response bodies to logs that are safe for your data policy. Don't retry in a tight loop.

For this pipeline, retries are easiest to reason about while embedding, retrieval, reranking, and classification remain side-effect free. The CRM write is different. Give that write a client-supplied idempotency key derived from the call ID and taxonomy version, so a retry cannot create two follow-up tasks. This separation also makes a replay useful: the team can reclassify the same call against a new taxonomy version without pretending it is the original decision.

I would ship the retrieve-rerank-classify design when tags depend on definitions that business teams revise. It keeps prompts smaller than copying the full taxonomy handbook into every request, gives each decision traceable guidance IDs, and puts provider replacement behind explicit application interfaces. It does add two stages before classification, so the team must measure the complete path rather than admiring one model response.

The catch is operational complexity. This design is not suitable when labels are a tiny, stable enum that a deterministic rule can classify correctly; use the rule and skip the model. Stick with a single direct model provider when its proprietary behavior is the product requirement and portability has no practical value. Prefer pgvector when the team already has strong Postgres operations and wants vector storage under its control. A separate managed vector system can be the better choice when retrieval scale and search operations need a dedicated owner.

There are also capability boundaries around the broader call workflow. Automatic speech recognition is not available through the described runtime, and real-time voice sessions are pending and limited to the western region. This article therefore starts with an existing call summary; it does not claim to transcribe the audio. There is no dedicated moderation endpoint either, so text or image review needs a chat model with a JSON Schema fallback. Those boundaries matter if someone tries to expand the classifier into an end-to-end call-processing stack.

Make retries and replay one reliability contract

Before copying this choice, build a labeled evaluation set from real taxonomy decisions and record four things: top-k retrieval recall, reranker ordering quality, final label accuracy, and end-to-end latency. Track prompt tokens as the taxonomy grows. Also record provider, model ID, taxonomy version, selected guidance IDs, and request ID for each evaluation run. A portable interface without comparable results is only a tidy abstraction.

I'm not sure which provider arrangement will win for a given call corpus because no runtime-authenticated latency or quality benchmark is available here. The deciding evidence is a replay of the same labeled calls through each adapter, with the same taxonomy version and output schema. Run that test before routing production CRM actions.

Ship the boundary first. Earn the routing decision with measurements.

References

• RFC 9110, HTTP Semantics: https://www.rfc-editor.org/rfc/rfc9110
• pgvector, Postgres vector similarity extension: https://github.com/pgvector/pgvector

Further reading

• HTTP retry and idempotency semantics: https://www.rfc-editor.org/rfc/rfc9110
• Vector similarity search with Postgres: https://github.com/pgvector/pgvector