8 RAG and Agentic AI interview questions and answers

RAG retrieves relevant passages from your own documents at question time and puts them in the prompt, so the model answers from that text.

It gives the model current and private knowledge without retraining, lets you show citations, and reduces made-up answers.

Embeddings are vectors that represent the meaning of text. Texts with similar meaning end up close together, usually measured with cosine similarity.

They power semantic search: you embed the question and find the nearest document chunks.

It is a trade-off. Small chunks match precisely but lose surrounding context. Large chunks carry context but dilute relevance and use more tokens.

A common start is a few hundred tokens with some overlap, split along the document's own structure such as headings. Then measure retrieval on real questions and adjust.

Hybrid search combines keyword search (BM25) with vector search, so exact terms such as product codes or names are not missed.

A reranker, usually a cross-encoder, then rescores the top candidates against the question, and only the best few go to the model.

Evaluate retrieval and generation separately.

For retrieval, build a labelled set of questions and measure whether the right passage appears in the top k results. For generation, check faithfulness to the retrieved context, correctness and citation accuracy, using human review or a model-graded check that you have calibrated against human judgements.

You describe each tool with a name, a description and a JSON schema for its parameters. The model returns a structured request to call a tool.

Your code runs the tool and sends the result back, and the model continues. The model never executes anything itself, so your code controls what is allowed.

The model chooses an action, your code runs the tool, the result is added to the conversation, and the loop repeats until the model gives a final answer.

Controls: a maximum number of steps, token and cost budgets, timeouts, an allow-list of tools, validation of tool arguments, and human approval before anything irreversible such as sending email or paying.

Prompt injection is text inside retrieved documents, web pages or tool output that tries to give the model new instructions.

There is no complete fix, so limit the damage: treat retrieved content as data and mark it clearly, give tools the least privilege they need, require confirmation for sensitive actions, and check outputs before acting on them.