Prompt is the full input you hand to a language model - everything the model sees before it generates a single word of output. In the simplest case that is a one-line question; in practice, a prompt usually bundles an instruction, any retrieved source passages, and the user's question together into a single piece of text.
Why it matters
A model can only work with what it is given. The quality, structure, and contents of a prompt directly determine the quality of the output: an ambiguous instruction produces a vague answer; a precise one, grounded in real source text, produces something you can actually trust and check.
This is especially important in document AI. When Sidenote answers a question about a PDF or article, it does not leave the model to draw on its own memory. It retrieves the most relevant passages from the document and places them directly in the prompt alongside your question, so the model is writing from text you can read, not text it once absorbed from somewhere on the web.
The way that prompt is constructed - which passages are chosen, how much context surrounds them, what the model is asked to do with them - is what separates a grounded, citable answer from a confident hallucination.
Vague prompt vs. effective prompt
The difference between a prompt that produces a generic answer and one that produces a specific, checkable answer usually isn't cleverness of wording - it's how much real information the prompt carries.
| Vague prompt | Effective prompt | |
|---|---|---|
| Example | "Tell me about the termination clause." | "Using the contract text below, does clause 14 allow either party to terminate without cause, and if so, what notice period is required?" |
| What the model has to work with | Its own training memory of what contracts "usually" say | The actual source passage, pasted or retrieved into the prompt |
| Task is explicit | No - the model guesses what "tell me about" means | Yes - a specific question with a specific expected answer shape |
| Answer is checkable against a source | No | Yes - the answer can be traced back to the passage it was built from |
| Most common failure mode | Confident, generic, possibly hallucinated detail | Rare - wrong answers are easier to catch because the source is visible |
The gap between these two isn't really about writing skill. It's about whether the prompt hands the model a real passage to reason over, or leaves it to reconstruct an answer from a blurred memory of similar documents it saw during training. The second row of that table - grounding the prompt in retrieved text - is doing almost all of the work.
Wording still matters at the margins - being specific about the desired format, length, or tone helps the model meet expectations more reliably - but it is a smaller lever than most people assume. A well-grounded prompt with plain, ordinary phrasing will consistently beat a cleverly worded prompt that gives the model nothing real to check its answer against.
Prompts can also be shaped by example rather than instruction alone. Few-shot prompting places a handful of worked examples in the prompt to steer the model's format or behavior, without any retraining. At the other end of the risk spectrum, prompt injection describes attempts to hijack a model by smuggling adversarial instructions into the prompt through the very documents it is reading.