Search that actually finds things: indexing nested Payload CMS content into Typesense

Search feels simple — until your content stops looking like a blog.

If every page in your CMS maps to one document, you're in luck. A blog post becomes one Typesense record. A product page becomes another. Whatever generated the page is whatever you index, and you barely have to think about the pipeline twice.

That assumption fell apart the moment we started shipping a public portal on Payload CMS 3 and Next.js. Almost none of the important content lived in neat collections. It lived inside globals: big, flexible documents with dynamic tabs, Lexical rich text, nested groups, people, downloadable files, and relationships pointing every which way. To an editor, that looks like one helpful page. To someone searching for a specific person or document, it's a haystack.

Here's why that hurts, and what we did about it.

What goes wrong if you index the page as-is

Imagine an "About Us" global with three tabs:

The tempting move is to shove the whole document into Typesense as one record. Less code. Technically not wrong — Typesense will take it.

Then you try to search it:

• Relevance drops. A search for "Jane Doe" has to fight a long founding story buried in the same record.
• Results point at the wrong place. Someone looks for a working group and lands at the top of "About Us," not the section they needed.
• Content disappears entirely. Text stuck inside Lexical, or a person nested a few levels down, never becomes something the engine can match.

That's not Typesense being difficult. We were asking it to treat a shelf of books like one book.

Stop mirroring the database. Start mirroring the query.

We didn't change the content model. Editors like that flexible page, and they should. What we changed was the index: stop treating it as a copy of the database, and treat it as a projection — reshape the same content into the units people actually search for.

So every save on a Payload global goes through a transform. One document in. Many Typesense records out:

The Typesense collection is intentionally small — enough to rank, deep-link, and facet:

await typesenseAdminClient.collections().create({
  name: 'site_content',
  fields: [
    { name: 'id', type: 'string', store: true },
    { name: 'title', type: 'string', infix: true, store: true },
    { name: 'slug', type: 'string', optional: true, store: true },
    { name: 'url', type: 'string', store: true },
    { name: 'anchor', type: 'string', optional: true, store: true },
    { name: 'content', type: 'string', infix: true, store: true },
    { name: 'collection', type: 'string', facet: true, store: true },
  ],
  default_sorting_field: '',
})

A single team-member record ends up looking like this:

{
  "id": "about-us__team-member__jane-doe",
  "title": "Jane Doe",
  "slug": "jane-doe",
  "url": "/about-us/team",
  "anchor": "#jane-doe",
  "content": "Jane Doe Team Member Northern Region",
  "collection": "team"
}

Small. Self-contained. Independently rankable. That's what you want to hand a search engine.

How the flattening actually works

The transform walks each global and pulls out the units people search for. A few rules keep that from getting messy.

Tabs become location. Each tab turns into its own record, with a stable key in the URL and anchor:

documents.push({
  id: `${baseId}__tab__${tabKey}`,
  title: tabTitle,
  slug: '',
  url: tabRoutes[tabKey] ?? baseUrl,
  anchor: `#${tabKey}`,
  content: resolveTabContent(doc, collection, tabKey, tab),
  collection,
})

Arrays explode — they don't concatenate. Eleven people in a list should become eleven records, not one blob of eleven names:

function extractTeamMembers(baseId: string, doc: PayloadDoc): SiteContentDocument[] {
  return members.map((member) => {
    const slug = slugifyName(member.name)
    const content = [member.name, member.designation, member.region].filter(Boolean).join(' ')

    return {
      id: `${baseId}__team-member__${slug}`,
      title: member.name,
      slug,
      url: '/about-us/team',
      anchor: `#${slug}`,
      content,
      collection: 'team',
    } satisfies SiteContentDocument
  })
}

Keep them as one blob and a search for a single name gets diluted by the other ten every time.

Nested groups recurse, with a limit. Groups inside groups are fine, until they aren't. Cap the depth so a creative editor can't accidentally mint thousands of records.

Getting text out of Lexical

Payload's Lexical editor doesn't store rich text as a string. It stores a tree:

So before anything is searchable, walk the tree and pull plain text — including nested objects, while skipping keys that are only metadata:

function extractText(node: unknown, maxLength = 2000): string {
  const parts: string[] = []

  function collect(value: unknown): void {
    if (isLexicalState(value)) {
      collect((value as { root: unknown }).root)
      return
    }
    if (typeof value === 'string') {
      const trimmed = value.trim()
      if (trimmed) parts.push(trimmed)
      return
    }
    if (Array.isArray(value)) {
      value.forEach(collect)
      return
    }
    if (typeof value === 'object' && value) {
      for (const [key, child] of Object.entries(value)) {
        if (!IGNORED_KEYS.has(key)) collect(child)
      }
    }
  }

  collect(node)
  return parts.join(' ').replace(/\s+/g, ' ').trim().slice(0, maxLength)
}

On the query side, title still gets more weight than body — a hit in the title is a stronger "you're in the right place" signal:

const result = await typesenseSearchClient.collections('site_content').documents().search({
  q,
  query_by: 'title,content',
  query_by_weights: '3,1',
  infix: 'always',
})

Keeping the index in sync without a cron job

We skipped the "re-scan everything every night" approach. Instead, Payload globals call the same transform from an afterChange hook:

afterChange: [
  async (args) => {
    try {
      const { syncToTypesense } = await import('lib/typesense/sync-to-typesense')
      await syncToTypesense('about-us')({
        doc: args.doc as unknown as PayloadDoc,
        operation: 'update',
      })
    } catch (err) {
      console.error('[Typesense] ERROR:', err)
    }

    return args.doc
  },
],

syncToTypesense builds the documents (base page, tabs, people, and so on), then upserts them one by one:

for (const document of documents) {
  try {
    await typesenseAdminClient
      .collections<SiteContentDocument>('site_content')
      .documents()
      .upsert(document)
  } catch (err) {
    syncLog.error({ documentId: document.id, err }, 'Failed to upsert document')
  }
}

Stable ids matter here. Because a team member's id is derived from a slug, an upsert updates the same record on the next save instead of inventing duplicates. You're not diffing nested editorial trees — you're projecting the current document into known ids and writing those.

What happens when you need to rebuild everything

Hooks are great for one save at a time. They're useless when you change the Typesense schema, fix the transform, or need to repair data across the whole site. That's when you want a full reindex.

Our reindex script drops and recreates the collection, then walks Payload collections and globals through the same syncToTypesense path the hooks use:

async function dropAndRecreateIndex(): Promise<void> {
  try {
    await typesenseAdminClient.collections('site_content').delete()
  } catch {
    // Collection did not exist — fine on a fresh environment
  }

  await typesenseAdminClient.collections().create({
    name: 'site_content',
    fields: [/* same schema as above */],
  })
}

// later, for each collection/global:
for (const doc of docs) {
  await syncToTypesense(slug)({ doc, operation: 'update' })
}

That's simpler than an alias swap. The tradeoff is honest: during a rebuild, search can be empty for a bit. For an editorial portal where reindex is rare and usually runs off-peak, that has been acceptable. If you need zero downtime later, the next step is the classic pattern — build a new collection, validate, then flip an alias — without changing the transform at all.

And please: reuse the same transform for hooks and reindex. Two implementations will drift. Not on day one — eventually, when someone fixes a bug in one path and forgets the other exists.

Making sure results land somewhere useful

None of this helps if a result drops someone at the top of "About Us" and makes them hunt.

Every generated record carries a url and optional anchor. The search UI just stitches them together:

<Link key={result.id} href={result.anchor ? `${result.url}${result.anchor}` : result.url}>
  {result.title}
</Link>

So a team member lands on /about-us/team#jane-doe, and a tab lands on its own route with #history (or whichever key you chose). Prefer stable internal keys for those anchors over human-editable labels — editors can rename "Team" to "Our People" without quietly breaking every old bookmark.

Putting it all together

What we used: Payload CMS 3 for the content model and lifecycle hooks, Next.js for the frontend and deep links, Typesense with infix search on title/content, and one shared TypeScript transform (syncToTypesense) behind both the hooks and the reindex script.

What I'd tell someone starting this from scratch

• Design the search record from the query backward. Ask what someone will type before you ask what the database looks like.
• One transform, called from two places. Hooks and reindex should share the same projection code.
• Stable ids + upsert beat nested diffs. Derive ids from slugs and tabs; let Typesense overwrite the current truth.
• Start simple on full rebuilds. Drop-and-recreate is fine until you need zero downtime — then add alias swap around the same transform.

The content model didn't change through any of this. Editors still get one flexible page with tabs and rich text. What changed is everything downstream — so searching for a name, a group, or a notice can land in one click, instead of dumping someone on a page and hoping they scroll.

Further reading

• Payload CMS — Globals
• Payload CMS — Hooks
• Typesense — Ranking and relevance
• Typesense — Collection aliases (useful when you outgrow in-place rebuilds)
• Lexical — Node overview