Cortex adds context, prompt prefix and exact response reuse, delivering up to 150× faster prefill, 11× faster response delivery and 2.25× throughput.
SAN FRANCISCO , CA, UNITED STATES, September 22, 2026 — Pervaziv AI today announced its 3-Tier Cortex Inference Cache Architecture, a new performance and governance layer designed to reduce repeated work across enterprise AI without weakening the controls that keep context current, permissions intact and results trustworthy.
The architecture separates caching into three forms of reuse: application context, model prompt prefix and exact completed response. Each tier has a different validity test and security boundary.
Recent Cortex measurements show why that separation matters. In a repeated conversation style workload, prompt processing fell from 2,913 milliseconds cold to 19.3 milliseconds warm, approximately 150 times faster. In a separate approved read only text route, an initial request completed in 3,132 milliseconds and an identical repeat returned a reported cache hit in 281 milliseconds, approximately 11 times faster with about 91 percent lower end to end latency. A separate serving workload matrix completed 240 requests with zero request failures and reached 2.25 times the throughput for one mid length workload under moderate concurrency.
These measurements describe different parts of the system, not one universal speed claim. Prompt prefix reuse accelerates repeated input processing while the model still generates a new answer. Exact response reuse can avoid generation for an approved identical request. Cortex keeps those distinctions visible so performance can be measured without confusing speed with correctness.
## Reuse What Is Safe. Recompute What Changed
——————————————————————–
Enterprise AI rarely begins from an empty prompt. A developer may continue a coding task across many turns. A security analyst may investigate changing evidence. A user in Cortex Discover may reason across browser tabs, then continue the objective in Visual Studio Code or Cortex Cloud. The next request often shares instructions, context and task state with what came before.
Repeating all of that work can waste time and compute. Reusing it indiscriminately can create a different problem: stale context, outdated permissions or a fast answer that no longer reflects the current task.
“The next step in enterprise AI performance is not simply caching more,” said Anoop Jaishankar, Founder and CEO of Pervaziv AI. “It is knowing exactly what can be reused, what changed, who is still authorized to use it, and when fresh computation is required. Cortex is turning inference caching into a governed capability, where speed comes from removing repeated work without removing the checks that make the result trustworthy.”
## A Different Performance Layer for Cortex
———————————————————–
The announcement follows a sequence of Cortex releases that expanded where enterprise AI work can happen. Cortex Connect established continuity across mobile, browser experiences and Visual Studio Code. Cortex Cloud added durable managed execution for eligible work in a controlled remote environment. Cortex Discover brought Cortex into a dedicated agentic AI browser with current page awareness, selected tab context, visual understanding and supported actions.
The inference cache architecture addresses a different layer: what happens when Cortex encounters work that overlaps with what it has already processed.
Rather than treating caching as one feature, Cortex asks which portion of the earlier work is still valid for the current request. Sometimes that is prepared application context. Sometimes it is an identical beginning of the model input. In a narrower set of approved cases, it may be the completed response itself.
## Tier One: Reuse Current Context
———————————————–
Before a model generates a response, Cortex may need to gather conversation state, selected files, repository material, retrieved passages, browser content, workspace state and governing instructions. That preparation can require finding relevant material, checking whether sources remain available and assembling a bounded working context.
The first cache tier is designed to reuse eligible context preparation when source state and authorization remain current. The unit of reuse is application context, not a saved final answer. The model still receives the current request and performs inference.
Consider a developer who asks Cortex to explain a repository change, then asks how it affects testing, then requests a migration plan. Much of the project context may remain relevant, but the reasoning task is different each time. Context reuse can avoid rebuilding unchanged preparation while still allowing the model to answer the new question.
Freshness is the controlling condition. A document can be replaced. A repository branch can move. A browser page can navigate or update. A user can lose access to a workspace. Context that was valid moments earlier cannot remain eligible simply because it still exists in memory.
Cortex therefore ties reusable context to the source and authorization state that made it valid. When a meaningful revision or permission changes, Cortex can rebuild the affected context and continue through the normal path.
This also separates caching from context compaction. Compaction decides what information should remain active in a long running task. Caching decides how much valid work needs to be repeated. The two can work together, but they solve different problems.
## Tier Two: Reuse Model Preparation
—————————————————
Before a model produces its first output token, it processes the input it has been given. For long enterprise prompts, that input processing can represent a meaningful part of user wait time.
Many ongoing workflows contain a stable beginning. Governing instructions, tool definitions, a task framework or earlier conversation context may remain identical while new material is appended.
For eligible inference environments, Cortex can use prompt prefix reuse when the beginning of the model request remains identical under the correct isolation boundary. The model still runs, processes the changed portion and generates a new response. This is a computation optimization, not a replay of the old answer.
In one inference test, prompt processing fell from 2,693 milliseconds cold to 19.3 milliseconds warm, approximately 140 times faster. In a separate conversation style test of the same workload, prompt processing fell from 2,913 milliseconds to 19.3 milliseconds, approximately 150 times faster. Warm total latency in those runs was 547 to 554 milliseconds, and repeat runs kept warm total latency within approximately 6 percent.
The 150 times figure applies to prompt processing in that tested workload, not complete response time. Generation still occurs, and total user visible latency includes additional work.
Cortex also avoids manufacturing certainty when a serving environment does not expose a reliable cache hit signal. In those cases, the system can record the state as unknown and report timing evidence rather than turn a fast request into an assumed hit.
## Tier Three: Reuse an Exact Response, Selectively
——————————————————————–
The third tier is the most direct form of reuse and also the most restrictive.
When an approved route receives a completely identical request, Cortex can return a previously completed response without running model generation again. Because that behavior can bypass new reasoning, the eligibility standard is deliberately narrow.
Exact response reuse is designed for carefully reviewed, read only routes where the complete effective request matches and the relevant source, authorization, model and execution state remain valid.
A live test used an approved read only text task. The first request was an exact response cache miss and completed in 3,132 milliseconds. An identical repeat returned an explicitly reported hit in 281 milliseconds. The responses matched. For that request pair and deployment, the repeat was approximately 11 times faster and had about 91 percent lower latency.
That result demonstrates the measured value of the route. It is not a guarantee that every Cortex request will see the same improvement.
The narrow scope matters because enterprise agents inspect live information, call tools, write code, prepare changes and can participate in workflows with external effects. A cached response cannot prove that a deployment completed now, that a browser page still shows the same state or that a new action was actually performed.
This is also why Cortex does not treat semantic similarity as proof that two requests are equivalent. Similar questions can differ in permissions, source evidence, tool definitions, model configuration or requested action. Exact response reuse therefore favors exact identity and explicit route eligibility.
## Governance Across All Three Tiers
————————————————–
The three tiers define what can be reused. Cortex governance determines whether that reuse is allowed for the current request.
Enterprise information does not follow one uniform ownership model. Two people at the same company can have different access to repositories, cases or customer records. A single request can combine public documentation with a private project file. Access can also change while reusable state remains in memory.
Cortex is designed to carry authorization into the reuse boundary. The applicable scope follows the most restrictive material in the request. Public information can be broadly reusable only when it is actually classified as public. Shared workspace material requires current permission. Personal or sensitive material requires narrower isolation. If the system cannot establish the required boundary, the architecture favors fresh computation or disables that reuse path.
Authorization remains separate from caching. A cache does not decide whether a user can read a document. Cortex must establish access before that document participates in the request. Reuse is an optimization behind that decision, not a replacement for it.
## Freshness Is More Than a Timer
———————————————–
Traditional caches often rely heavily on expiration time. Enterprise AI needs a more complete definition of validity.
A context entry might be scheduled to live for an hour, but the source document could change five minutes later. A user could lose access before the timer expires. A tool contract, model revision or governing instruction could also change how the request should be processed.
Cortex treats validity as a relationship among the request, its inputs, current authorization and execution configuration. Changes to source material, model behavior, instructions, tools, policy or task context can make earlier work ineligible even before expiration.
The architecture also distinguishes invalidation from eviction. Invalidation means earlier state can no longer be selected, while eviction reclaims capacity. Correctness depends on stale or unauthorized state becoming unavailable for selection.
## A Warm Cache Must Not Become a Dependency
———————————————————————
Caching is designed to accelerate Cortex, not become the only way it can function.
Inference processes can restart. Reusable state can be evicted under memory pressure. Requests can move to different instances. Policy changes can invalidate earlier scopes.
When safe reuse is unavailable or uncertain, Cortex should continue through an authorized cold path and recompute the required work.
Important task state therefore cannot live only inside a model cache. Conversation continuity, source state, approvals and workflow outcomes remain governed by the systems that own them. A cached prefix is not proof that a current tool result was observed. A cached context snapshot is not authorization after permissions change. A cached response is not evidence that an external action happened.
## Throughput, Tail Latency and Fairness
——————————————————–
The architecture also looks beyond the speed of one warm request.
A workload matrix completed 240 requests with zero request failures. For one mid length workload, moderate concurrency reached 2.25 times the throughput, while 95th percentile latency increased by 19 percent. At larger inputs, additional concurrency produced little throughput benefit and much worse tail latency.
Those results show why enterprise inference cannot optimize for hit rate or peak throughput alone. Reusable state consumes memory. Long context can create greater reuse opportunities but also greater pressure on shared serving capacity.
Cortex therefore evaluates preparation time, time to first token, generation rate, end to end latency, throughput, tail latency, memory pressure and fairness together. The goal is not the largest possible cache. It is to complete more useful, eligible work within latency, capacity and safety requirements.
## Built for Long Running, Cross Surface Work
————————————————————–
Cortex Discover gives users a dedicated browser where AI can work with current pages and selected tabs. Cortex Connect helps maintain continuity as users move among mobile, browser and development environments. Cortex Cloud gives eligible tasks a managed place to continue remote execution. Cortex routing coordinates models, search and engineering skills around the task.
Across those experiences, repeated information is common. A developer can research a library change in Cortex Discover, continue implementation in Visual Studio Code and move eligible work to Cortex Cloud. Stable instructions and project context may persist while the question, file revision, browser state or execution result changes.
The three tier model lets Cortex reuse work at the layer where it remains valid. Context can be reused while sources and access remain current. Model preparation can be reused while the model facing prefix remains identical. A completed response can be reused only when the full effective request remains an approved exact match. Everything else proceeds as fresh work.
## A Controlled Path to Broader Deployment
————————————————————-
Pervaziv AI is treating inference caching as a route specific capability rather than a blanket switch.
A read only analysis route, a coding task, a live security investigation and a workflow that can change external state should not inherit the same cache policy. Candidate deployments can therefore be evaluated independently across correctness, privacy, cold and warm behavior, source and permission invalidation, memory use, mixed workloads, failover, fairness and tail latency.
One tier can be enabled without assuming the others are appropriate. Context reuse can provide value even if exact response reuse remains disabled.
This incremental model keeps performance claims grounded in observed workloads rather than extrapolated from one benchmark.
## The Larger Cortex Direction
——————————————
The 3-Tier Cortex Inference Cache Architecture advances the broader direction behind recent Cortex releases: enterprise AI should become more capable without asking organizations to give up visibility or control.
Cortex Discover expanded the execution surface into the browser. Cortex Cloud added durable managed execution. Cortex Connect linked intent and progress across devices and development environments. Smarter context compaction addressed how long running work can carry the right information forward more efficiently.
Inference caching now addresses repeated computation itself.
For users, that can mean less waiting during iterative work. For administrators, it creates a clearer framework for where reuse is permitted and when it must stop. For platform operators, it separates context preparation, prompt processing, generation and completed response reuse so each can be measured honestly.
The principle is simple: reuse context when its sources and access remain valid, reuse model preparation when the prefix actually matches, and reuse a finished answer only when the complete request is an approved exact match.
If those conditions are not satisfied, recompute.
That is the performance model Pervaziv AI is building into Cortex: faster where reuse is safe, fresh where the work has changed, and governed across both paths.
## About Pervaziv AI
—————————-
Pervaziv AI builds Cortex, an Enterprise AI Control Layer for secure, governed AI work across browser, mobile, development and cloud environments. Cortex coordinates specialized AI models and agents with model routing, search routing, skill routing, security, privacy, verification and managed execution capabilities. The platform is designed to help organizations move from isolated AI assistance toward trusted workflows that connect context, reasoning, action and evidence across the places where enterprise work happens.
