AWS Lambda Memory Allocation and Performance Tuning: The Complete Guide
Tune AWS Lambda performance: the memory-to-CPU model, benchmarking with Power Tuning, cost analysis, and adaptive allocation patterns.
After optimizing cold starts in the first part, the next challenge is making your Lambda functions run efficiently once they are warm. Memory allocation is the configuration knob with the widest blast radius, because one value sets the CPU share, the execution time, and the bill at once.
The working default: start at 1024 MB, then move only where a measured curve tells you to. The 128 MB default is almost always wrong for anything that computes, and the reflex fix of jumping straight to 3008 MB usually buys capacity that single-threaded code cannot spend.
Memory size is only one input. The model worth carrying is the relationship between allocated memory, the CPU that comes with it, and the GB-seconds you are billed for.
Lambda’s Memory-CPU Model
How CPU Scales with Memory
Lambda exposes one resource dial. The memory setting also decides how much CPU the function gets:
| Memory | Approx. vCPU | What that buys |
|---|---|---|
| 128 MB | ~0.07 | Slowest execution; CPU-bound work crawls |
| 512 MB | ~0.29 | Common baseline for light API handlers |
| 1024 MB | ~0.58 | Balanced starting point for most workloads |
| 1769 MB | 1.00 | One full vCPU |
| 3008 MB | ~1.70 | More than one core; only parallel code benefits |
| 10240 MB | ~5.79 | Maximum allocation |
CPU scales in proportion to memory the whole way up, so the vCPU column is the memory value divided by 1769. Past that line you own more than one core, and single-threaded Node.js code stops getting faster while the per-millisecond price keeps climbing.
The Memory-Duration Curve
Every function has its own curve, and the only numbers worth acting on are the ones you measure. Lambda Power Tuning is a Step Functions state machine that invokes the same payload at several memory settings and reports duration and cost for each:
{
"lambdaARN": "arn:aws:lambda:us-east-1:123456789012:function:image-resize",
"powerValues": [128, 512, 1024, 1769, 3008],
"num": 50,
"payload": { "key": "sample.jpg" },
"parallelInvocation": true,
"strategy": "balanced"
}
Two shapes turn up repeatedly. CPU-bound work (image resizing, compression, hashing) falls steeply until it approaches one vCPU, and cost per invocation stays close to flat along that stretch, because the shorter duration offsets the higher per-millisecond rate. I/O-bound work flattens far earlier: past the knee you are paying more per millisecond to wait on the same network call.
That is why 1024 MB is where to start. It usually sits on the flat part of the cost curve while giving CPU-bound code most of a core, and the measured curve tells you which way to move from there.
Building a Benchmarking Framework
Comprehensive Performance Testing Setup
Power Tuning gives you the curve between configurations. In-function instrumentation tells you where the time goes inside a single invocation:
// comprehensive-benchmark.ts
import { performance, PerformanceObserver } from 'perf_hooks';
interface BenchmarkResult {
memoryUsed: number;
executionTime: number;
coldStart: boolean;
gcEvents: number;
cpuIntensive: boolean;
}
// Module scope survives across invocations in the same environment
let isWarm = false;
export class LambdaBenchmark {
private results: BenchmarkResult[] = [];
private coldStart = !isWarm;
private gcCount = 0;
constructor() {
isWarm = true;
this.monitorGC();
}
private monitorGC() {
// PerformanceObserver reports every collection the engine runs.
// Patching global.gc would only count the ones you trigger yourself,
// and global.gc needs --expose-gc, which Lambda does not enable.
const observer = new PerformanceObserver((list) => {
this.gcCount += list.getEntries().length;
});
observer.observe({ entryTypes: ['gc'] });
}
async benchmark<T>(
operation: () => Promise<T>,
label: string
): Promise<{ result: T; metrics: BenchmarkResult }> {
this.gcCount = 0;
const startMemory = process.memoryUsage();
const startTime = performance.now();
const result = await operation();
const endTime = performance.now();
const endMemory = process.memoryUsage();
const metrics: BenchmarkResult = {
memoryUsed: endMemory.heapUsed - startMemory.heapUsed,
executionTime: endTime - startTime,
coldStart: this.coldStart,
gcEvents: this.getGCEvents(),
cpuIntensive: this.detectCPUIntensiveOperation(endTime - startTime)
};
console.log(`Benchmark [${label}]:`, metrics);
this.results.push(metrics);
return { result, metrics };
}
private detectCPUIntensiveOperation(duration: number): boolean {
// Operations taking >100ms are likely CPU-bound
return duration > 100;
}
private getGCEvents(): number {
return this.gcCount;
}
}
// Usage in your Lambda
export const handler = async (event: any) => {
const benchmark = new LambdaBenchmark();
const { result } = await benchmark.benchmark(async () => {
return await processLargeDataset(event.data);
}, 'data-processing');
return result;
};
Production Benchmarking Strategy
If you want the same sweep without the state machine, move one function through the memory settings and read the REPORT line, which carries billed duration and max memory used:
for memory in 512 1024 1536 1769 3008; do
aws lambda update-function-configuration \
--function-name image-resize \
--memory-size "${memory}" > /dev/null
# Configuration updates are asynchronous; invoking too early hits the old value
aws lambda wait function-updated-v2 --function-name image-resize
echo "== ${memory} MB =="
aws lambda invoke \
--function-name image-resize \
--cli-binary-format raw-in-base64-out \
--payload file://test-payload.json \
--log-type Tail \
--query 'LogResult' --output text \
"response-${memory}.json" | base64 --decode | grep REPORT
done
One invocation per setting measures noise. Repeat the loop and take a median before you trust the ordering.
Memory Optimization Strategies
Strategy 1: Right-Sizing for Workload Types
Different workloads have different optimal memory allocations:
// Memory allocation by workload type
const workloadOptimization = {
// API Gateway proxy functions
simpleAPI: {
memoryMB: 512,
reason: "Low CPU, fast response time priority"
},
// Database operations
databaseIntensive: {
memoryMB: 1024,
reason: "Balanced CPU for query processing + connection overhead"
},
// Image/file processing
fileProcessing: {
memoryMB: 1769,
reason: "CPU-intensive, benefits from full vCPU"
},
// ML inference
machineLearning: {
memoryMB: 3008,
reason: "Memory for model + multi-core for inference"
},
// Data transformation
dataETL: {
memoryMB: 1769,
reason: "CPU-bound operations, optimal cost/performance"
}
};
Strategy 2: Memory Leak Prevention
A background watchdog is the wrong shape here. Lambda freezes the execution environment between invocations, so a setInterval timer only ticks while a request is in flight and its ticks land on whichever invocation happens to be running. Sample at known points in the handler instead:
// Memory pressure sampling, inside the invocation
export class MemoryManager {
private readonly allocatedBytes =
parseInt(process.env.AWS_LAMBDA_FUNCTION_MEMORY_SIZE || '512', 10) * 1024 * 1024;
private readonly threshold = 0.8; // 80% of allocated memory
check(label: string): void {
const usage = process.memoryUsage();
// Lambda bills and kills on the whole sandbox, so rss is the number
// that matches "Max Memory Used" in the REPORT line, not heapUsed
const ratio = usage.rss / this.allocatedBytes;
if (ratio > this.threshold) {
console.warn('High memory usage detected:', {
label,
rss: Math.round(usage.rss / 1024 / 1024) + 'MB',
heapUsed: Math.round(usage.heapUsed / 1024 / 1024) + 'MB',
external: Math.round(usage.external / 1024 / 1024) + 'MB',
usage: Math.round(ratio * 100) + '%'
});
}
}
}
// Usage pattern
const memoryManager = new MemoryManager();
export const handler = async (event: any) => {
memoryManager.check('invocation-start');
const result = await processEvent(event);
memoryManager.check('invocation-end');
return result;
};
A leak announces itself as invocation-start climbing over successive requests on the same environment. A single expensive request shows up only at invocation-end.
Strategy 3: Garbage Collection Optimization
NODE_OPTIONS accepts only a small subset of V8 flags. --expose-gc, --gc-interval and --optimize-for-size are rejected, and the runtime refuses to start rather than ignoring them, so anything guarded by if (global.gc) is dead code on the managed runtime:
// Set as a Lambda environment variable
const gcOptimizations = {
NODE_OPTIONS: [
'--max-old-space-size=1024', // keep the heap under the configured memory
'--max-semi-space-size=32' // larger young generation, fewer scavenges
].join(' ')
};
// Bound the working set instead of reaching for a manual collection
const processLargeDataset = async (data: any[]) => {
const results = [];
for (const chunk of chunkArray(data, 1000)) {
results.push(await processChunk(chunk));
// Nothing above holds a reference to the previous chunk's
// intermediates, so the collector reclaims them on its own
}
return results;
};
Chunking bounds the transform step, not the input. If the dataset itself is what fills the sandbox, stream it out of S3 rather than buffering it and then slicing.
Cost Analysis Framework
The Real Cost of Memory Allocation
Build a comprehensive cost analysis that factors in all variables:
// cost-calculator.ts
interface LambdaCostParams {
memoryMB: number;
avgExecutionMs: number;
invocationsPerMonth: number;
region: 'us-east-1' | 'us-west-2' | 'eu-west-1';
}
interface CostBreakdown {
computeCost: number;
requestCost: number;
totalMonthlyCost: number;
costPerInvocation: number;
performanceRating: number;
}
export class LambdaCostCalculator {
// x86 on-demand rates; the same numbers apply in these three regions.
// arm64 is cheaper per GB-second, so the architecture is a lever too.
private pricing: Record<LambdaCostParams['region'], {
computePerGBSecond: number;
requestPer1M: number;
}> = {
'us-east-1': { computePerGBSecond: 0.0000166667, requestPer1M: 0.20 },
'us-west-2': { computePerGBSecond: 0.0000166667, requestPer1M: 0.20 },
'eu-west-1': { computePerGBSecond: 0.0000166667, requestPer1M: 0.20 }
};
calculateCost(params: LambdaCostParams): CostBreakdown {
const { memoryMB, avgExecutionMs, invocationsPerMonth, region } = params;
const pricing = this.pricing[region];
// Convert memory to GB and execution time to seconds
const memoryGB = memoryMB / 1024;
const executionSeconds = avgExecutionMs / 1000;
// Calculate compute cost
const gbSeconds = memoryGB * executionSeconds * invocationsPerMonth;
const computeCost = gbSeconds * pricing.computePerGBSecond;
// Calculate request cost (pricing is per 1M requests)
const requestCost = (invocationsPerMonth / 1000000) * pricing.requestPer1M;
const totalMonthlyCost = computeCost + requestCost;
const costPerInvocation = totalMonthlyCost / invocationsPerMonth;
// Performance rating (lower execution time = higher rating)
const performanceRating = Math.max(1, 10 - (avgExecutionMs / 100));
return {
computeCost,
requestCost,
totalMonthlyCost,
costPerInvocation,
performanceRating
};
}
findOptimalMemory(
baseParams: Omit<LambdaCostParams, 'memoryMB'>,
performanceProfile: { memory: number; executionMs: number }[]
): { memory: number; cost: number; savings: number } {
const scenarios = performanceProfile.map(profile => ({
...profile,
cost: this.calculateCost({
...baseParams,
memoryMB: profile.memory,
avgExecutionMs: profile.executionMs
})
}));
// Find the configuration with the best cost-performance ratio
const optimal = scenarios.reduce((best, current) =>
(current.cost.totalMonthlyCost / current.cost.performanceRating) <
(best.cost.totalMonthlyCost / best.cost.performanceRating)
? current : best
);
const baseline = scenarios[0]; // Assuming first is baseline
const savings = baseline.cost.totalMonthlyCost - optimal.cost.totalMonthlyCost;
return {
memory: optimal.memory,
cost: optimal.cost.totalMonthlyCost,
savings
};
}
}
// Usage example
const calculator = new LambdaCostCalculator();
// Durations come from your own Power Tuning run, not from this page
const performanceData = [
{ memory: 512, executionMs: 2100 },
{ memory: 1024, executionMs: 1300 },
{ memory: 1769, executionMs: 900 },
{ memory: 3008, executionMs: 800 }
];
const optimal = calculator.findOptimalMemory({
avgExecutionMs: 0, // Will be overridden
invocationsPerMonth: 1000000,
region: 'us-east-1'
}, performanceData);
console.log(`Optimal configuration: ${optimal.memory}MB`);
console.log(`Monthly savings: $${optimal.savings.toFixed(2)}`);
Advanced Performance Patterns
Pattern 1: Adaptive Memory Allocation
Dynamically adjust processing based on available memory:
// adaptive-processing.ts
export class AdaptiveProcessor {
private availableMemoryMB: number;
private processingStrategy: 'small' | 'medium' | 'large';
constructor() {
this.availableMemoryMB = parseInt(process.env.AWS_LAMBDA_FUNCTION_MEMORY_SIZE || '512');
this.processingStrategy = this.determineStrategy();
}
private determineStrategy(): 'small' | 'medium' | 'large' {
if (this.availableMemoryMB >= 3008) return 'large';
if (this.availableMemoryMB >= 1024) return 'medium';
return 'small';
}
async processData(data: any[]): Promise<any[]> {
switch (this.processingStrategy) {
case 'large':
// Process everything in memory with parallel operations
return await this.parallelProcessing(data);
case 'medium':
// Batch processing with moderate memory usage
return await this.batchProcessing(data, 1000);
case 'small':
// Stream processing to minimize memory usage
return await this.streamProcessing(data, 100);
}
}
private async parallelProcessing(data: any[]): Promise<any[]> {
// Node runs one thread: this overlaps I/O, it does not use extra cores.
// CPU-bound work above 1769 MB needs worker_threads to reach the second core.
const chunks = this.chunkArray(data, Math.ceil(data.length / 4));
const promises = chunks.map(chunk => this.processChunk(chunk));
const results = await Promise.all(promises);
return results.flat();
}
private async batchProcessing(data: any[], batchSize: number): Promise<any[]> {
const results = [];
for (let i = 0; i < data.length; i += batchSize) {
const batch = data.slice(i, i + batchSize);
const processed = await this.processChunk(batch);
results.push(...processed);
}
return results;
}
private async streamProcessing(data: any[], chunkSize: number): Promise<any[]> {
const results = [];
for (let i = 0; i < data.length; i += chunkSize) {
const chunk = data.slice(i, i + chunkSize);
const processed = await this.processChunk(chunk);
results.push(...processed);
}
return results;
}
private chunkArray<T>(array: T[], size: number): T[][] {
return Array.from({ length: Math.ceil(array.length / size) }, (_, i) =>
array.slice(i * size, i * size + size)
);
}
private async processChunk(chunk: any[]): Promise<any[]> {
// Your actual processing logic here
return chunk.map(item => ({ ...item, processed: true }));
}
}
Pattern 2: Memory-Aware Caching
Implement intelligent caching based on available memory:
// memory-aware-cache.ts
export class MemoryAwareCache {
private cache = new Map<string, any>();
private maxMemoryUsage = 0.6; // Use max 60% of available memory for cache
private availableMemoryBytes: number;
constructor() {
this.availableMemoryBytes = parseInt(process.env.AWS_LAMBDA_FUNCTION_MEMORY_SIZE || '512') * 1024 * 1024;
}
set(key: string, value: any): void {
const currentUsage = process.memoryUsage().heapUsed;
const maxCacheMemory = this.availableMemoryBytes * this.maxMemoryUsage;
if (currentUsage < maxCacheMemory) {
this.cache.set(key, {
value,
timestamp: Date.now(),
size: this.estimateObjectSize(value)
});
} else {
// Cache is full, implement LRU eviction
this.evictLeastRecentlyUsed();
this.cache.set(key, {
value,
timestamp: Date.now(),
size: this.estimateObjectSize(value)
});
}
}
get(key: string): any {
const entry = this.cache.get(key);
if (entry) {
// Update timestamp for LRU
entry.timestamp = Date.now();
return entry.value;
}
return null;
}
private evictLeastRecentlyUsed(): void {
let oldestKey = '';
let oldestTime = Date.now();
for (const [key, entry] of this.cache.entries()) {
if (entry.timestamp < oldestTime) {
oldestTime = entry.timestamp;
oldestKey = key;
}
}
if (oldestKey) {
this.cache.delete(oldestKey);
}
}
private estimateObjectSize(obj: any): number {
// Rough estimation of object size in memory
return JSON.stringify(obj).length * 2; // Rough approximation
}
getCacheStats(): {
entries: number;
estimatedMemoryMB: number;
memoryUsagePercent: number;
} {
let totalSize = 0;
for (const entry of this.cache.values()) {
totalSize += entry.size;
}
return {
entries: this.cache.size,
estimatedMemoryMB: totalSize / 1024 / 1024,
memoryUsagePercent: (totalSize / this.availableMemoryBytes) * 100
};
}
}
Production Monitoring and Profiling
Advanced CloudWatch Custom Metrics
Track the numbers Lambda does not publish for you. Each putMetricData is a synchronous API call on the invocation path, so above a few requests per second, write CloudWatch Embedded Metric Format lines to stdout and let the log agent extract the metrics instead:
// performance-monitor.ts
import { CloudWatch } from '@aws-sdk/client-cloudwatch';
export class PerformanceMonitor {
private cloudWatch: CloudWatch;
private functionName: string;
constructor() {
this.cloudWatch = new CloudWatch({});
this.functionName = process.env.AWS_LAMBDA_FUNCTION_NAME || 'unknown';
}
async trackPerformanceMetrics(
executionTime: number,
memoryUsed: number,
cpuIntensive: boolean
): Promise<void> {
const metrics = [
{
MetricName: 'ExecutionTime',
Value: executionTime,
Unit: 'Milliseconds',
Dimensions: [
{ Name: 'FunctionName', Value: this.functionName },
{ Name: 'MemorySize', Value: process.env.AWS_LAMBDA_FUNCTION_MEMORY_SIZE || '512' }
]
},
{
MetricName: 'MemoryUtilization',
Value: memoryUsed,
Unit: 'Bytes',
Dimensions: [
{ Name: 'FunctionName', Value: this.functionName }
]
},
{
MetricName: 'CPUIntensiveOperations',
Value: cpuIntensive ? 1 : 0,
Unit: 'Count',
Dimensions: [
{ Name: 'FunctionName', Value: this.functionName }
]
}
];
await this.cloudWatch.putMetricData({
Namespace: 'Lambda/Performance',
MetricData: metrics
});
}
async trackCostMetrics(estimatedCost: number): Promise<void> {
await this.cloudWatch.putMetricData({
Namespace: 'Lambda/Cost',
MetricData: [
{
MetricName: 'EstimatedCost',
Value: estimatedCost,
Unit: 'None',
Dimensions: [
{ Name: 'FunctionName', Value: this.functionName }
]
}
]
});
}
}
X-Ray Performance Profiling
Use X-Ray for detailed performance insights:
// x-ray-profiling.ts
import * as AWSXRay from 'aws-xray-sdk-core';
export const handler = async (event: any) => {
// Lambda opens the facade segment for you. captureAsyncFunc would execute
// the body at module load rather than returning a wrapped handler.
const segment = AWSXRay.getSegment();
// Memory allocation tracking
const memorySubsegment = segment?.addNewSubsegment('memory-tracking');
const initialMemory = process.memoryUsage();
memorySubsegment?.addAnnotation('initial_memory_mb', Math.round(initialMemory.heapUsed / 1024 / 1024));
try {
// Your business logic with subsegments
const processingSegment = segment?.addNewSubsegment('data-processing');
const result = await processData(event.data);
processingSegment?.close();
// Memory usage after processing
const finalMemory = process.memoryUsage();
memorySubsegment?.addAnnotation('final_memory_mb', Math.round(finalMemory.heapUsed / 1024 / 1024));
memorySubsegment?.addAnnotation('memory_delta_mb', Math.round((finalMemory.heapUsed - initialMemory.heapUsed) / 1024 / 1024));
return result;
} finally {
memorySubsegment?.close();
}
};
const processData = async (data: any) => {
const segment = AWSXRay.getSegment();
const subsegment = segment?.addNewSubsegment('data-transformation');
try {
// Add metadata for performance analysis
subsegment?.addMetadata('input_size', JSON.stringify(data).length);
subsegment?.addAnnotation('cpu_intensive', true);
const result = await heavyProcessingOperation(data);
subsegment?.addMetadata('output_size', JSON.stringify(result).length);
return result;
} finally {
subsegment?.close();
}
};
Practical Lessons: When Memory Optimization Goes Wrong
The Over-Allocation Trap
Over-allocation is the most common way a tuning pass backfires on the bill. Moving a function from 1024 MB to 3008 MB on the assumption that “more is always better” nearly triples the per-millisecond rate. That only pays for itself if the duration falls by roughly the same proportion, which it will not once the function is past one vCPU and single-threaded.
The rule: after any performance change, recompute GB-seconds. A duration graph alone will approve a change that raises the bill.
The Memory Leak That Appeared at Scale
Leaks hide at low concurrency. Every concurrent request gets its own execution environment, and each one accumulates independently, so a slow leak in a logging or tracing library is invisible over ten invocations and fatal over ten thousand.
The signal is Max Memory Used in the REPORT line. If it climbs across successive invocations of the same environment instead of settling, something is retaining references between requests.
// An OOM kill cannot be caught inside the function: the runtime is
// terminated, not thrown at. Guard the expensive path before it starts.
class MemoryGuard {
private readonly allocatedBytes =
parseInt(process.env.AWS_LAMBDA_FUNCTION_MEMORY_SIZE || '512', 10) * 1024 * 1024;
private readonly ceiling = 0.75;
async run<T>(operation: () => Promise<T>): Promise<T> {
const { rss } = process.memoryUsage();
if (rss / this.allocatedBytes > this.ceiling) {
// Fail with a retryable error instead of dying mid-write
throw new Error(
`Memory ceiling reached: ${Math.round(rss / 1024 / 1024)}MB already in use`
);
}
return operation();
}
}
The False Economy of Under-Allocation
The mirror mistake is a blanket memory cut. Halving memory halves the rate per millisecond, but if duration more than doubles, GB-seconds go up and you have paid extra for a slower function. Under-allocation also moves every invocation closer to the timeout, and a timeout that triggers a retry bills the work twice.
What’s Next: Production Monitoring Deep Dive
A memory setting holds only as long as the traffic that shaped it. The next part of this series covers the monitoring, error tracking, and debugging techniques that tell you when the curve has moved underneath you.
That part covers:
- Advanced CloudWatch dashboards and alerts
- X-Ray trace analysis and performance insights
- Error handling and circuit breaker patterns
- Production debugging tools and techniques
The Default and Its Limits
1024 MB with a measured curve behind it covers most Node.js functions: API handlers, database-backed reads, moderate transforms. Three cases justify overriding it. Go lower when the function is a thin proxy whose duration is dominated by a downstream call, because extra CPU buys nothing while you wait on the network. Go above 1769 MB only when the work is genuinely parallel (worker threads, or native libraries that thread internally) or when a model or dataset has to fit in memory. Before going wider, try arm64: Graviton costs about 20% less per GB-second at the same memory setting.
Re-run Power Tuning after a dependency upgrade, a runtime bump, or a shift in payload size. The target is not the fastest possible execution; it is the lowest GB-second figure that still clears your latency budget.
References
- Configure Lambda function memory - Official reference on the 128 MB - 10,240 MB memory range and the proportional CPU allocation model.
- Memory and computing power - AWS Lambda - How Lambda allocates vCPU proportionally to memory, with 1,769 MB equaling one full vCPU.
- Profiling functions with AWS Lambda Power Tuning - Using the open-source Lambda Power Tuning tool (Step Functions-based) to find the optimal memory setting.
- Lambda cost and performance optimization - Serverless Applications Lens - Well-Architected guidance on balancing cost and performance through memory allocation and ARM/Graviton processor options.
- Best practices for working with AWS Lambda functions - Official Lambda best practices on initialization patterns, connection reuse, and avoiding dependency bloat.
- Using CloudWatch metrics with Lambda - Lambda metrics available in CloudWatch for tracking memory utilization, duration, and invocation patterns.
- AWS Lambda pricing - Per-GB-second and per-request rates by architecture, including the x86 and arm64 difference used in the cost model above.
AWS Lambda Production Guide: 5 Years of Real-World Experience
A comprehensive guide to AWS Lambda based on 5+ years of production experience, covering cold start optimization, performance tuning, monitoring, and cost optimization with real war stories and practical solutions.
All Posts in This Series
Related posts
A technical guide to choosing and implementing AWS edge computing for global apps, with practical examples and cost optimization strategies.
Production-tested strategies for cutting AWS Lambda cold starts: runtime selection, provisioned concurrency, and practical optimization techniques.
Advanced AWS Lambda patterns and cost optimization: Lambda Layers, VPC configuration, cross-account execution, and architectural decisions.
Run Bun and Deno on AWS Lambda with custom runtimes: performance benchmarks, cost analysis, and production deployment patterns.
Prompt caching, model routing, token budgets, and semantic caching: how to keep production LLM spend predictable without giving up answer quality.