AI Engineering
Inference cost is a design problem, not a billing problem
By Pavan Jegurupati, Founder, InferLearn
The bill is a symptom
By the time finance notices, the architecture has already decided your costs.
Count calls before you count tokens
Most expensive pipelines are expensive because they call a model more often than they need to, not because the model is pricey.
Cache the deterministic parts
A surprising share of any pipeline is not probabilistic at all.
Taggedcostinferencearchitecture
Keep reading
