Start with the product decision, not the model demo
Choosing an AI product development partner is difficult because a polished demo can hide the parts that determine whether a product survives contact with users. A model may answer an ideal prompt beautifully while failing on incomplete data, ambiguous intent, permission boundaries, long sessions, or a provider outage. The buying decision is therefore not simply who can connect an API fastest. It is who can turn uncertain model behavior into a useful, operable product.
Begin with one business decision or workflow. Name the user, the action they are trying to complete, the cost of a wrong answer, and the acceptable recovery path. This forces every potential partner to discuss product behavior rather than perform a generic chatbot routine. If the proposal begins with a model name and only later discovers the user problem, the sequence is backwards.
What an AI product development partner should clarify
A useful discovery phase should reduce uncertainty before it expands scope. The output does not need to be a hundred-slide strategy deck. It should make the riskiest assumptions visible and define how they will be tested.
- The user task and the point where AI adds value beyond a deterministic feature.
- The data the system may read, retain, transform, or send to a model provider.
- The failure modes that require a refusal, a fallback, or human review.
- The quality threshold, test set, latency target, and cost envelope for the first release.
- The smallest production slice that can generate trustworthy evidence.
Ask for an evaluation plan before accepting a roadmap
Model output is probabilistic, so conventional acceptance criteria are necessary but insufficient. A button can work while the answer behind it is unreliable. OpenAI’s official guidance on working with evals recommends defining the objective, collecting representative data, setting metrics, and running comparisons continuously. The exact vendor is less important than the discipline: examples from the real workflow, explicit pass criteria, and regression checks whenever prompts, tools, retrieval, or models change.
Ask candidates to describe the first evaluation set they would build. Strong answers separate factual accuracy, task completion, policy compliance, tone, latency, and cost instead of compressing everything into a single subjective score. They also explain who reviews uncertain cases. A product team should be able to see why a release passed, not just hear that the newest model felt better in testing.
Inspect the interface for failure, not only success
AI product design is partly the design of uncertainty. Users need to know what the system understood, what it is doing, which source supports a claim, and how to correct it. Useful patterns include constrained inputs, previews before consequential actions, visible source references, editable drafts, progress states, and clear handoff to a person. The right pattern depends on the risk of the task.
Accessibility belongs in the same review. The W3C overview of WCAG 2.2 describes a testable standard for perceivable, operable, understandable, and robust web content. Ask how streamed answers are announced to assistive technology, how focus moves after an error, whether motion can be reduced, and whether every core action works by keyboard. An interface is not production-ready if the happy-path animation is elegant but the recovery path is invisible.
Make operating economics part of product design
A prototype budget covers only the visible build. A production budget also includes inference, retrieval, storage, observability, evaluation runs, human review, support, and future model changes. Ask for a cost model based on user actions rather than an impressive monthly token number. The useful unit might be a reviewed document, a resolved support case, or a completed research task.
The partner should identify which steps need a frontier model and which can use a smaller model, deterministic code, cached output, or no AI at all. This is not penny-pinching. It improves speed, predictability, and resilience. A model router can help, but only after each route has its own quality and latency checks. Cheap output that creates expensive rework is not an optimization.
Review security as a system property
No credible partner can promise zero hallucinations or absolute safety. They can define boundaries and controls. The OWASP project for LLM application risks documents issues such as prompt injection, insecure output handling, sensitive information disclosure, and excessive agency. The practical review should cover data classification, provider retention settings, tool permissions, secrets, tenant isolation, audit logs, incident response, and the actions an agent is never allowed to take without confirmation.
Ask for a threat model tied to the planned workflow. A research assistant that summarizes public pages and an agent that edits customer records should not share the same permission model. Capabilities should be granted narrowly, logged, reversible where possible, and tested with hostile inputs.
Evidence to request before you sign
Procurement becomes easier when evidence is concrete. Request a working prototype plan, architecture diagram, evaluation approach, data flow, delivery milestones, ownership terms, operating-cost assumptions, and the named risks that could change price or schedule. If case studies are offered, ask what the team actually delivered and which results can be verified. A screenshot is evidence of a screen, not evidence of adoption or business impact.
- A narrow first release with an explicit definition of done.
- Access to the repository, deployment configuration, prompts, and evaluation assets.
- A change process for models, providers, prompts, and data sources.
- Clear responsibility for monitoring, support, security updates, and handover.
- A written list of exclusions, dependencies, and decisions still owned by your team.
Three useful buying questions
Should we build in-house or hire a studio? Build in-house when the capability is core, the team can own it long term, and the learning loop is already available. A focused external partner can help when you need to reduce product and technical uncertainty quickly, establish the first operating system, or add skills that are not yet permanent hires.
How long should an AI prototype take? There is no honest universal number. Time depends on data access, workflow complexity, risk, integration, and evaluation. Ask what uncertainty each phase removes and what evidence triggers the next investment.
Which model should we choose? Choose against your own evaluation set, latency needs, data constraints, and cost envelope. Preserve the ability to change providers where the economics justify it. Model selection is a repeatable product decision, not a permanent brand allegiance.
A practical next step
Before contacting an AI product development partner, write a one-page brief with the user, workflow, available data, unacceptable failures, desired evidence, and decision deadline. The brief will expose vague proposals quickly and give good teams something useful to challenge. If you are shaping an AI-enabled product and want a sober review of the product, interface, and delivery risks, contact Seuil Digital with that page. No theatre required.