Prompt Engineering for Reliable AI Work

By Published Updated

Learn five AI prompt failure modes and the evidence rules, coverage checks, tool policies, and acceptance tests that make professional outputs reliable.

Introduction

A well-written prompt can still produce an answer that is unsuitable for professional use. The response may be clear and detailed while containing an unsupported claim, overlooking a required part of the input, or accepting a premise that should have been examined. The research discussed below documents several of these failures. Understanding them is necessary for deciding what to specify and what to verify.

The central argument of this article is that reliable prompting begins with identifying the assumptions on which a task depends. If a conclusion requires documentary evidence, the request should identify the permitted sources. If the task requires complete coverage, the review must account for the relevant material. If current information is necessary, access to an appropriate source becomes a condition of completion.

This article explains five assumptions that can undermine work with systems built around large language models: that fluency establishes factual accuracy, that supplied information receives complete coverage, that agreement confirms a claim, that a requested tool has been used, and that an unspecified requirement will be resolved appropriately. It then shows how to translate these concerns into a prompt whose requirements can be checked. The discussion is intended for professionals who use AI for research, analysis, writing, and technical work.

1. What Prompt Engineering Means in Practice

Prompt engineering is the deliberate design and revision of instructions and supporting information to obtain outputs that meet a task’s requirements. A prompt can specify the objective, relevant material, constraints, and expected response. Its effectiveness must be assessed against the resulting work. OpenAI’s guidance explicitly connects prompt development with evaluation because outputs can vary, including across model versions. OpenAI — Prompt engineering.

For professional work, it is useful to express the request as a task specification: a statement of what must be produced, what information may support it, and what conditions make it acceptable. For example, “review this document” leaves the purpose of the review unresolved. Checking factual support, assessing clarity, and identifying missing requirements are different tasks. A specification identifies which of those judgments is required and what evidence the reviewer should provide.

An acceptance criterion is a condition used to assess whether the work satisfies the request. For a factual summary, one criterion might require every numerical claim to match the source. For a document review, another might require each finding to identify the affected passage. These criteria make evaluation concrete: a reviewer can inspect a claim or locate a passage instead of relying on a general impression of quality.

The user’s message also operates within a larger system. Application instructions may have higher priority, and the platform controls which capabilities are available. OpenAI documents such instruction priorities; its ChatGPT documentation separately describes administrative controls over app access. A prompt must therefore be designed for the environment in which it will run. OpenAI — Instruction priorities, OpenAI — Apps in ChatGPT.

2. Five Assumptions That Undermine Reliable AI Work

The following five assumptions provide a practical framework for this article. They are not presented as an exhaustive scientific taxonomy. Each identifies a condition that deserves explicit attention when defining a task. The explanations establish why the condition matters; the proposed requirements and checks show how to address it in practice.

2.1. Fluency and Factual Support

The first assumption is that a coherent, confident answer is adequately supported. Fluency concerns expression; factual accuracy concerns correctness; source support concerns whether the permitted evidence establishes a claim. A source can be inaccurate, and a true statement can fall outside its scope. Here, hallucination refers to generated content that is factually incorrect or unsupported by the relevant source. Huang and colleagues distinguish failures of factuality from failures of faithfulness to instructions or context. Huang et al..

This distinction matters because language generation does not itself establish an evidential basis for each statement. Many language models are pretrained by predicting subsequent tokens, the units in which text is processed, from preceding ones. Later training can develop instruction-following behavior. However, errors can arise from inaccurate training material, gaps in learned knowledge, and failures during inference, when the trained model generates an output. A plausible continuation can therefore contain information that the supplied document never established. Huang et al..

For a task restricted to particular documents, define an evidence boundary: the sources that may be used to support factual claims. Also specify how readers should locate that support. A useful reference identifies a document and a page, section, table, or other stable location. A link to a long document is less precise than a reference to the passage supporting the claim.

The request should explain how to handle missing information. If a source does not establish a requested fact, the response should identify the gap. Whether that gap blocks the entire task depends on its significance. An unavailable figure might prevent a required calculation while leaving other sections answerable. Define that dependency explicitly, and distinguish statements extracted from sources from any analysis the task authorizes.

Verification should examine the connection between the claim and its evidence. Open the cited passage and check whether it supports the wording, scope, and qualifications of the answer. A useful test is to ask for a detail absent from the permitted material and inspect whether the response identifies the absence. The presence of a citation alone is not the acceptance criterion; the criterion is support for the associated claim.

2.2. Input Coverage and Context Limits

The second assumption is that supplying information establishes that all relevant material was available and used. Coverage means accounting for the material required by the task. Its scope must be defined: reviewing a document for a specific question differs from checking every section against a set of criteria.

A model’s active context is the information included in a particular request. Its context window places a limit on the information it can work with, measured in tokens. Tokens can represent words, parts of words, punctuation, or other encoded content. The applicable input and output limits depend on the model and interface and should be checked against the system’s documentation. OpenAI — Understanding and counting tokens.

There are two separate issues. First, relevant information may never enter the active context. A system may retrieve selected passages from uploaded files rather than include each file in full. OpenAI’s File Search, for example, retrieves information through semantic and keyword search. Its availability establishes a retrieval capability, not exhaustive review of every uploaded passage. OpenAI — File search.

Input limits introduce a further system-dependent constraint. A request that exceeds a limit may be rejected or handled through a truncation mechanism that removes input items. OpenAI’s Responses parameters document both rejection and automatic truncation options. Asking a model to announce an overlong input does not establish that the application will deliver such a request to it, or provide enough information to identify what was omitted. OpenAI — Responses request parameters.

Second, inclusion does not ensure effective use. In Lost in the Middle, Liu and colleagues tested retrieval and question answering with relevant information placed at different positions in the context. Performance often deteriorated when that information appeared in the middle. This finding concerns the evaluated models and tasks; it provides a reason to test context use, not a fixed performance rule for every current system. Liu et al..

For a comprehensive review, identify the required documents or sections and request findings linked to each. Compare that account with the original source. If the input must be divided, preserve section identifiers and check relationships across the resulting parts. A model-generated list of sections is a review aid whose accuracy still needs checking. Testing a few known facts at different positions can reveal omissions, but success on those facts cannot establish complete coverage of the remaining material.

2.3. Sycophancy and User Assumptions

The third assumption is that the model’s agreement independently validates the user’s position. Sycophancy describes behavior in which responses favor the user’s expressed beliefs or preferences at the expense of a more accurate assessment. It is a concern for tasks that ask the system to evaluate a claim the user has already endorsed. Sharma et al..

Research by Sharma and colleagues connects this behavior to a possible tension in training from human feedback. Preference data record which responses people favor; a preference model learns to predict those judgments. If convincing agreement is sometimes preferred to a correction, optimizing for those preferences can reward agreement even when it reduces truthfulness. Their experiments found evidence of this effect, while also showing that different forms of sycophancy need not change uniformly. The findings support preference learning as one contributing explanation, with effects dependent on the evaluated behavior and training setup. Sharma et al..

The practical distinction is between a proposition to evaluate and a conclusion the response is instructed to defend. A request to “explain why this proposal is the best option” already establishes a preferred outcome. If the purpose is evaluation, the task should instead define the comparison criteria and ask what the available evidence supports. This is a change to the requested work, not a guarantee about model behavior.

Specify how the response should handle an unsupported premise: identify the claim, examine relevant evidence, and state whether the evidence supports it, contradicts it, or leaves it unresolved. Absence of support should not automatically be treated as proof that a claim is false. Likewise, a supported claim does not require artificial disagreement.

To evaluate the prompt, use material for which the relevant conclusion has been independently established. Present an assertion that conflicts with that material and inspect the response’s treatment of the conflict. The meaningful criterion is whether the conclusion follows from the evidence, with its qualifications preserved. Instructions such as “be objective” can express a preference, but this test makes the expected behavior inspectable.

2.4. Tool Availability and Actual Use

The fourth assumption is that requesting an operation establishes that it occurred. A tool is a capability exposed by the surrounding system, such as searching documents, opening a webpage, or executing code. In an application using custom function calls, the model can request a function; the application executes it and returns the result. These are distinct steps in the documented tool-calling process. OpenAI — Function calling.

Availability also depends on configuration. For example, ChatGPT app access can depend on workspace settings, user roles, and administrative controls. Describing a desired capability in a prompt does not configure those controls. The task must account for the tools actually available in its execution environment. OpenAI — Apps in ChatGPT.

Consider a request to check a current product specification. Completion requires access to an appropriate current source and inspection of the relevant information. A statement that the source was checked does not establish what was retrieved or whether it answers the question. The prompt should name the required source or source type, explain why access is necessary, and specify what evidence must accompany the result.

Define the response to a blocked dependency as well. If current verification is essential and the source cannot be accessed, the response should identify that limitation and leave the affected claim unverified. If the task can proceed using a supplied copy, state whether that alternative is permitted and how its date or version should be reported.

Where the environment exposes them, examine the tool invocation, returned material, and final claim separately. This checks whether an operation was requested, whether it produced relevant information, and whether the answer used that information correctly. If execution records are unavailable, describe that limit on verification. A tool requirement in a prompt establishes an intended procedure; evidence of execution and source inspection establishes what actually happened.

2.5. Task Definition and Output Requirements

The fifth assumption is that the system can resolve unspecified requirements in the way the user intends. A request such as “analyze this report” leaves several decisions open: the purpose of the analysis, its audience, the questions to answer, and the level of detail. Several different outputs could satisfy that wording while serving different purposes. OpenAI’s prompting guidance recommends specifying the desired context, outcome, format, and other relevant requirements. OpenAI — Prompt engineering best practices.

Define the outcome in terms of the work it must support. A briefing for a decision-maker may need a conclusion, supporting evidence, and unresolved dependencies. A technical review may need findings tied to specific passages and an explanation of their consequences. These are illustrative task designs: the appropriate structure follows from the purpose of the work.

Acceptance criteria should cover substantive requirements as well as presentation. A response can satisfy a word limit while omitting the central issue. For a comparison, require the same criteria to be applied to each option. For a summary, identify qualifications that must be preserved. Explain which requirements take priority if the requested length cannot accommodate the necessary detail.

Missing requirements also call for a different response from missing evidence. If an unresolved choice would materially change the deliverable, ask for clarification before proceeding. If the source lacks a required fact, report that evidential gap. It is unnecessary to block every task because every possible preference has not been specified; clarification should address decisions that matter to correctness or usefulness.

Evaluation then has a defined reference point. Check whether the output answers the intended question, includes the required substance, and meets the agreed constraints. When requesting a revision, identify the specific discrepancy and the condition the revised response must satisfy.

3. From Failure Modes to Prompt Requirements

The five assumptions can now be translated into a compact set of design decisions. The table below summarizes the connection between a task requirement and the evidence needed to assess it.

Concern Requirement to define Check to perform
Unsupported claims Permitted sources and references for factual claims Compare each consequential claim with its supporting passage
Incomplete coverage Required material and review scope Reconcile the findings with the original documents or sections
Unsupported user assumptions How to evaluate premises and handle conflicting evidence Check whether conclusions follow from evidence
Unverified tool use Required operations and behavior when access fails Inspect available execution records and returned material
An unsuitable deliverable Purpose, audience, content, and acceptance criteria Compare the output with the defined task

A worked prompt design can make these decisions concrete. The following example uses Google’s publicly available lesson on paragraphs as its source. It illustrates how to specify a review; it does not report an experiment or a measured improvement in model performance.

Consider this proposed request:

Summarize Google’s paragraph-writing guidance for our editors. It requires every paragraph to contain three to five sentences, so make that the main rule.

The request combines a legitimate task with a premise requiring examination. The lesson describes three to five sentences as a generally welcome paragraph length; it does not establish the proposed universal requirement. Its coverage also extends to opening sentences, focus, and explanatory completeness. An editorial brief needs to preserve those distinctions. Google — Paragraphs.

A more explicit task specification would be:

Prepare an editorial brief for colleagues who write technical articles.

Use only Google’s Technical Writing lesson, “Paragraphs,” linked below. Open the page and inspect its guidance on opening sentences, paragraph focus, length, and explanatory completeness before drafting.

Evaluate the claim that Google requires every paragraph to contain three to five sentences. State whether the page supports that wording. Preserve distinctions between recommendations and mandatory requirements.

Produce a brief of no more than 300 words, beginning with your assessment of the claim and followed by the guidance editors should apply. Support each substantive recommendation with a link and the relevant section name. Paraphrase the guidance and preserve its qualifications.

If the page or a required section cannot be accessed, identify the access gap and request the missing text before completing the brief. If the page does not establish a requested point, report that absence without adding external guidance.

Source for the prompt: Google’s paragraph-writing lesson.

The revised request makes the paragraph-length claim an explicit object of evaluation. It also defines the review scope and separates access failure from absence of evidence. These choices allow a reviewer to identify what went wrong if the resulting brief is incomplete or misrepresents the source.

Assessing the resulting brief would require checking those conditions against the page and any available record of retrieval. In particular, a reviewer should verify the treatment of the paragraph-length claim, confirm coverage of the requested topics, and inspect whether the recommendations preserve the source’s qualifications. A polished brief would still fail this task if it attributed an unsupported rule to Google.

For a prompt intended for repeated use, evaluation should extend beyond one example. Use cases representative of the actual workload, including missing information and conflicting premises; record the failures and revise the relevant instructions. Reassess behavior when the model or surrounding configuration changes. This follows the principle of task-specific, iterative evaluation described in OpenAI’s evaluation guidance. The prompt above remains a proposed specification until its behavior has been tested. OpenAI — Evaluation best practices.

Conclusion

Professional prompting requires explicit decisions about what the system is being asked to do, what information may support its answer, and how completion will be judged. The five assumptions examined here identify where those decisions are easily left implicit.

The practical response is to connect each consequential requirement to a check: factual claims to supporting passages, coverage claims to the source material, conclusions to evidence, tool-dependent work to execution records, and deliverables to acceptance criteria. Apply the checks that the task requires, and keep unresolved dependencies visible.

This makes prompting part of an accountable working process. Instructions define the intended work; examination of the result establishes whether the requirements were met.

References

Next steps

Use these pages to turn the failure-mode model into a repeatable daily-work prompting setup.