Answer quality is decided while you prepare the material
Parsing, metadata, chunking by document type, parent-child and late chunking, indexing choices you cannot undo cheaply, and Korean-specific traps.
On This Page
Answer quality is mostly decided while the material is read, split, and stored — not when you pick a model. Garbage in, garbage retrieved.
Collect and parse
PDFs, scans, tables, and slides need a parser that keeps structure. Do not flatten a table into a paragraph. Keep column headers with rows. For web pages and Markdown, keep the heading path. That path is the cheapest way to restore context later.
If an old policy and a new policy sit side by side, the model blends them. Delete duplicates, drafts, and obsolete versions, or label the version.
Every chunk needs a minimum label: source document, section and page, created and updated dates, owner, who may see it, and document type.
| Source | Keep separately |
|---|---|
| PDF, Word | Paragraphs, headings, tables, pages |
| PowerPoint | The link between the slide and the speaker notes |
| Excel, CSV | Column names and data types |
| Sender, recipients, thread, date | |
| Chat | Conversation flow, replies, time |
| Meeting transcript | Speaker, time, and the split between discussion, decision, and action |
| Relational database | Relationships between tables, the meaning of columns, permissions |
| Documents with figures | A separate reading of tables, charts, and images |
Store the original file, the extracted text, the table structure, and the location in the original as linked records. Break that link and you cannot cite later.
There is no universal chunk size, only a starting default
Two lessons show up across studies. A method that looks sophisticated does not always win. Chunks that are too small raise the chance of finding a relevant piece while starving the answer of context. You retrieve more and still answer wrong.
A well-tested start is recursive splitting along structure: paragraph, then sentence, then word, about 512 tokens, with 10 to 20 percent overlap.
| Material | Prefer | Avoid |
|---|---|---|
| Prose and reports | Paragraph boundaries, about 512–800 tokens | Cutting through a paragraph |
| Markdown and technical docs | Split on headings and store the heading path | Separating a heading from its body |
| Code | Function or class | Cutting through a function body |
| Tables | Repeat column headers on every row group | Headerless rows |
| Lists | Repeat the lead-in sentence on each group of items | Bare items |
| Legal, medical, standards | Put the parent chapter and section on the small chunk | Separating a clause number from its conditions |
| Chat and transcripts | Split where the speaker or topic changes | Splitting only by token count |
Fixed-size, semantic, and structure-aware splits all exist. The practical combination is structure-aware splitting plus two extras.
Parent-child retrieval searches small child chunks and, when writing the answer, hands the model the parent section. Search unit and reading unit are different jobs. A starting comparison is 300–600 tokens for search and 1,500–3,000 for the parent. That is not a universal answer. Confirm it on your evaluation set.
Late chunking reverses the order. The model reads the whole document and builds a meaning for each token, then the boundaries are applied. Each chunk carries context from the rest of the document, so a pronoun that points elsewhere still retrieves. Unlike contextual retrieval, you do not pay a model to write a blurb. The limit is the embedding model's window, for example 8,192 tokens or about 10 pages.
Indexing is expensive to undo
Indexing means arranging material so it can be found later.
- Changing the embedding model changes the number space. You rebuild everything. Record the model name and version on each chunk, and plan a period where old and new indexes run together.
- Skip unchanged chunks. A fingerprint of the text plus the model version lets you avoid recomputing them, and lets a stopped job resume.
- Build both a meaning index and a keyword index. You cannot invent the missing one at question time.
- Put dates, departments, and access levels in at write time. Filtering after retrieval fails for both security and accuracy.
Where Korean specifically breaks
Korean attaches particles and endings to words. A computer can treat "계약서를" and "계약서" as different words. Keyword search needs morphological analysis. Tools such as Nori, Kiwi, and MeCab do that work, and the analyzer you choose changes accuracy directly.
An embedding model's Korean skill is not the same as its English benchmark. If the corpus mixes languages, test all of these:
- Korean question, Korean document
- English question, English document
- Korean question, English document
- Exact strings: abbreviations, personal names, jargon, product numbers, contract numbers