ML systems
Production ML Serving Architecture
A hybrid batch and real-time architecture turning model scores, live behavior, and campaign rules into low-latency personalization decisions.
The problem
Personalized messaging required fresh behavioral context and low-latency decisions, but many useful ML features and scores were most reliable and economical to compute in batch. The serving design needed to connect both worlds without making campaign delivery depend on slow analytical queries.
My role
I designed the production serving architecture and its decision flow, aligning model computation, storage, APIs, session events, and product rules around the latency and reliability requirements of push and email delivery.
System design
batch scoring→ Key-value store
low latency→ Prediction APIs
decision access→ Session engine
real-time events→ Push & email
personalized action
- Batch jobs produced stable model scores in BigQuery.
- Scores were loaded into a low-latency key-value store instead of queried from the warehouse at request time.
- Prediction APIs exposed the decision inputs to downstream services.
- A real-time event-streaming session engine combined stored predictions with live behavior to decide audience, timing, and content.
Operating principles
The architecture separated expensive model computation from latency-sensitive decisioning. It also provided clear boundaries for monitoring data freshness, API availability, decision volume, and model behavior.
Outcome
The system served personalized push and email decisions to approximately 12 million monthly users across eight countries, supporting the experimentation and causal measurement program described in the remarketing case study.
This is a conceptual representation. Infrastructure names, internal schemas, and proprietary implementation details are intentionally omitted.