Search index projection

Security impact is direct when projections copy parent metadata, ACLs, tenant IDs, or classification labels into child documents. If that metadata is omitted, a RAG app can retrieve a chunk that no longer carries the security context of its source file. The projection should preserve fields needed for filters and citations while avoiding unnecessary sensitive content in retrievable fields. Admin keys, data-source credentials, managed identity, and private connectivity still protect the pipeline boundary. The design question is simple: can every projected chunk still prove who may see it? Reviewers should test at least one restricted source document end to end.

A projection has no independent meter, but it changes storage, indexing, enrichment, and AI costs. Splitting documents into many child records increases index document count, storage, and rebuild time. Enrichment skills, vectorization, and semantic ranking can add cost when applied to every chunk. The upside is lower prompt cost and better answer quality because retrieval returns focused passages instead of entire files. FinOps review should track average chunks per source, vector dimensions, storage growth, rebuild frequency, and whether projected fields are truly needed by the application. A small projection mistake can multiply storage and enrichment work across every source file.

Reliability depends on whether projections produce stable, repeatable documents during reindexing. Bad parent keys, changing chunk IDs, or incomplete projection mappings can create duplicates, orphan child records, or missing passages after an indexer run. Operators should validate the number of projected chunks, parent identifiers, and projection selectors after every pipeline change. Keep sample source files for regression testing, and plan rebuilds carefully because projection changes usually affect document identity and retrieval behavior. Users experience projection failures as missing citations or inconsistent AI answers. Stable projection tests should run before every major skillset or chunking change. Operators should record baseline behavior and expected recovery evidence.

Performance impact appears in both ingestion and retrieval. During indexing, projections add mapping work and can multiply document volume. During search, projected chunks usually improve retrieval precision because smaller records match more specific user questions. Too many tiny chunks can increase result noise, while oversized chunks can slow prompts and weaken grounding. The right projection balances chunk size, parent context, vector fields, filters, and citation needs. Measure indexing duration, chunk count, query latency, first-page relevance, and answer quality before declaring the projection successful. Teams should test projected retrieval with real prompts, not only indexer completion status. Performance review should include p95 behavior and worst-case user journeys.

Operators inspect index projections through skillset JSON, indexer status, target index fields, document counts, and sample projected documents. They troubleshoot by reviewing enrichment errors, failed field mappings, null parent IDs, unexpected chunk counts, and stale documents after source changes. The practical runbook compares a known source file with its projected child records: count, key, title, path, ACL metadata, and text length. CLI helps verify the search service and run REST calls, while detailed projection editing is usually REST, SDK, or infrastructure-as-code work. Operators should archive sample source files and expected projected documents for regression checks. The support path should include owner, evidence, and next action.