Summary

• Cortex AI Function Evaluation lets customers benchmark quality, cost, token use, consistency, and alternative implementations against repeatable datasets.
• Function Optimization can test prompts and model candidates automatically to identify configurations meeting quality requirements at lower cost.
• Cortex Agents Compact summarises earlier conversation state to reduce token consumption and context pressure in longer-running agent sessions.

Snowflake has added evaluation and optimisation tools to its Cortex AI platform, giving organisations a more systematic way to decide whether model outputs are good enough — and economical enough — before they become part of production data workflows.

Cortex AI Function Evaluation, now in public preview, lets teams run AI functions and model calls against a consistent dataset while measuring output quality, cost, token usage, and variation. Different prompts, models, or implementations can therefore be compared against the same examples instead of being judged through a handful of manually selected demonstrations.

A second public-preview feature, Cortex AI Function Optimization, can automatically search across alternative prompts and model candidates. The system scores those combinations against a defined dataset and metric, allowing customers to compare whether a cheaper or smaller model performs adequately against a more capable alternative.

Snowflake has also introduced a preview Compact API for Cortex Agents. It summarises earlier conversation history into a smaller representation that can be passed into later agent calls, reducing token consumption and helping longer conversations remain inside model context limits.

Evaluation becomes part of AI engineering

The releases address a persistent production problem: a model that performs impressively in a demonstration can still behave unevenly across the less convenient inputs found in real workloads. Small changes to prompts, model versions, input data, or output requirements can alter results, while open-ended generation makes conventional software testing difficult to apply directly.

Repeatable evaluation provides a way to turn that ambiguity into a measurable engineering process. Snowflake customers can use labelled datasets, generate synthetic examples, or create expected outputs and then run candidate implementations against the same test material.

For classification or constrained outputs, rule-based metrics can determine whether an answer meets the expected result. More open-ended tasks can use an LLM-based judge or custom functions, although those approaches introduce another model into the evaluation chain and therefore require scrutiny of their own.

The underlying principle is less exotic than the technology being tested. Conventional software teams do not normally change a database engine, compiler, or business rule and then judge the result from three convenient examples. AI systems are gradually acquiring equivalent disciplines around regression tests, baselines, release criteria, and performance monitoring.

That becomes more important when an AI function sits inside SQL or a data workflow because its output can feed downstream applications without a person examining every result. A subtle quality decline that would be irritating in a chatbot can become a data-quality problem when thousands of records are classified, extracted, or enriched automatically.

Model economics depend on the workload

Snowflake’s optimisation feature also tackles a cost question that becomes harder as model catalogues grow. Larger frontier models may produce stronger results on demanding tasks, but paying for that capability is wasteful when a smaller model can satisfy the workload’s actual quality threshold.

The platform can compare several candidates against the same metric and dataset, estimating cost alongside quality. The selected combination can then be used directly in an AI call or materialised as a reusable custom AI function with a stable SQL interface.

That selection problem is becoming a recurring enterprise discipline rather than a one-time procurement decision. An organisation might need a high-capability model for complex reasoning but use cheaper models for classification, extraction, moderation, or structured generation, with the preferred choice changing as pricing and performance move.

Techopia has previously examined how lower model prices can be offset by increasingly elaborate agent workflows. Evaluation and optimisation address one part of that problem by making quality-versus-cost decisions testable rather than intuitive.

The Compact API tackles another source of spend. Conversational agents commonly send part or all of an earlier interaction back to a model so it has enough context to answer the next request. As those conversations lengthen, repeated history consumes more tokens and can eventually exceed the context available to the model.

Compaction summarises that earlier state before subsequent calls. The trade-off is that compression can discard detail, leaving developers to decide which information must be retained exactly and which can safely be represented by a shorter summary.

Production AI acquires conventional controls

Taken together, the releases show a data platform treating AI outputs less like opaque creative artefacts and more like components that can be tested, benchmarked, costed, and revised. That shift becomes necessary as models move from optional assistants towards systems that transform data or initiate downstream work.

The difficult questions remain around the metrics themselves. A repeatable benchmark is valuable only if the dataset resembles real operating conditions, the scoring method captures the errors the business actually cares about, and teams revisit those tests as inputs and models change.

A single aggregate score can also hide failure modes affecting smaller but important groups of inputs. Production evaluation therefore needs more than one leaderboard number, particularly in regulated or high-impact use cases where a rare class of error may carry disproportionate consequences.

Even with those limitations, enterprise AI adoption is moving into a phase where “the model looked good in testing” is no longer an adequate implementation standard. Organisations increasingly need to know what was tested, against which data, at what cost, using which version, and how much performance changed before an AI function becomes routine infrastructure.