Machine Learning Engineer Interview Questions and Answers

I start by pinning the objective and the constraint: what counts as a good recommendation, how many requests per second arrive, and what latency the page can tolerate, because those decide everything downstream. Then I map the data flow — impression and click logs, catalog attributes, user history — and mark which features must be computed online and therefore live in a feature store. For the model I would split retrieval from ranking: a cheap generator narrows millions of items to a few hundred candidates, then a heavier ranker scores them inside the budget. I define offline metrics such as recall at k and ranking quality, add online guardrails including click-through and dwell time, and plan a shadow deployment before any traffic split. Finally I describe the monitoring and retraining loop, because a recommender decays as the catalog and user taste change.

I treat skew as a class of bug with known causes and I check them in order. First I compare the raw feature distributions the model sees at serving time against the distributions recorded at training time, which catches upstream pipeline changes and silent schema shifts. Second I check the transformation code itself: if training uses a pandas path while serving uses a different implementation, the two will disagree on edge cases such as nulls, time zones and default values, so I push both onto one shared transformation library and one feature store definition. Third I check point-in-time correctness, because a feature joined without an as-of timestamp leaks future information into training. I also log raw feature values at request time and alert when live values drift outside the training band.

I begin with the budget as a hard number and work backward. If the endpoint must answer within a fixed number of milliseconds at p95, I split that across network, feature retrieval, preprocessing and model inference, leaving deliberate headroom for the tail. That usually forces the real decisions: whether features come from an in-process cache or Redis, whether the model fits comfortably in memory, whether I can batch concurrent requests, and whether a smaller or quantized model is worth a small quality loss. I benchmark the deployed service under production-like concurrency rather than trusting a notebook timing, and I load test before launch. If the budget still is not met, I would shrink the model or move heavy computation offline into a precomputed feature instead of promising an impossible speed.

A deployed model needs three layers of monitoring. The first is service health: request rate, error rate, latency and saturation, the same signals every endpoint needs. The second is data and feature health: null rates, range violations, schema changes and drift between live feature distributions and the training baseline, because these move before quality does. The third is model quality: where labels arrive later I join them back and track the live metric against the offline estimate, and where they never arrive I watch proxy signals and the prediction distribution. I wire alerts to the symptom a human should act on rather than to every metric, and each alert links to a runbook that says whether to roll back, retrain or investigate an upstream pipeline.

I do not retrain on a fixed calendar alone; I combine a schedule with triggers and an explicit decision rule. The schedule depends on how fast the world changes — a fraud model may need weekly retraining while a demand model may hold for a quarter. The triggers are drift in input features, a fall in a live quality metric, rising prediction error once labels arrive, or a major upstream change such as a new catalog or pricing system. Before automating anything, I make retraining reproducible and evaluated: the candidate is scored against a frozen holdout and against the current champion on identical data. I promote it only if it wins on the primary metric without regressing guardrails, and a human reviews the evaluation report until the process has earned trust.

I A/B test a model the same way I would test any product change, with extra care about interference. First I define one primary metric, guardrails such as latency and error rate, and a minimum detectable effect so the test has a sample size instead of an arbitrary duration. I randomize by user, not by request, because the same person seeing two different models produces an inconsistent experience and contaminates the comparison. I run a shadow or canary phase first to catch serving bugs and confirm the artifact loads correctly. During the test I watch for novelty effects and for feedback loops, since a ranking model changes the very data it will later train on. I state the ship rule in advance and treat a flat result as a real outcome rather than rerunning until something looks positive.

I choose based on when the prediction is needed and how expensive the input features are. If a score can be computed hours ahead and consumed later, such as a nightly churn list or a daily demand forecast, I batch it, because batch jobs are simpler, cheaper and easier to backfill. If the decision must reflect the current request context, such as fraud scoring at checkout or ranking a live search page, I serve online and accept the latency and infrastructure cost. Many systems are hybrid: precompute the expensive static parts in batch, then combine them with request-time features in a lightweight online model. The deciding questions are the freshness requirement, the cost per prediction, and whether a slightly stale answer would change the user's outcome at all.

I reduce inference cost in layers, starting with measurement. I break down cost per prediction into compute, memory, network and storage, then find the dominant term. Common wins are batching concurrent requests to raise GPU utilization, right-sizing instance types instead of over-provisioning, caching predictions or features that repeat, and moving static computation into precomputed features. On the model side, quantization, pruning, distillation and compiling to a runtime such as ONNX can cut compute with little quality loss if I verify on a holdout. I also revisit the serving topology: autoscaling driven by queue depth, a cheaper tier for low-priority traffic, and dropping the model entirely when a simple heuristic performs close enough. Every change is checked against quality guardrails, because the cheapest model that fails is not a saving.

I make rollback a designed path rather than an improvisation. The model is versioned in a registry, the serving layer reads a pointer instead of a hardcoded artifact, and the previous version stays warm so reverting is a configuration change and a reload rather than a rebuild. Before launch I define the trigger: an error-rate spike, a latency breach, or a guardrail metric falling outside its expected band. On trigger, the on-call engineer reverts the pointer, confirms the metric recovers, and then preserves the failing artifact along with the request logs for analysis. I practice this in a game-day exercise, because a rollback that has never been rehearsed tends to fail at the worst possible moment. Afterward I write a short review and add the check that would have caught the problem earlier.

I handle imbalance by first questioning whether the metric is the problem. Accuracy is useless when positives are rare, so I switch to precision, recall, F1 or a cost-weighted measure that matches the business decision, and I fix the operating threshold from that cost rather than leaving the model default. On the data side, I try class weights or focal loss before resampling, because naive oversampling duplicates rare events and naive undersampling discards signal. I keep the evaluation set at the natural prevalence so offline numbers reflect what production will see, and I split by time or by entity rather than randomly when the same entity appears repeatedly. In production I monitor the positive rate and the score distribution, since a shift in either usually changes the right threshold.

Be explicit about what reached production: how many models you served, who consumed the predictions, and what monitoring and rollback existed around them.

Say how you kept one transformation path for offline and online features, and name the check or alert that caught skew before it reached users.

Use a placeholder such as [X]% only when you can describe the measurement method, the baseline and your own contribution to the result.