ML Case-study Interview Question: Graph Neural Networks for Personalized Notifications: Re-engaging Food Delivery Users
Browse all the ML Case-Studies here.
Case-Study question
How would you design a system to generate personalized notifications for a food delivery platform using Graph Neural Networks to improve user re-engagement, especially for users who do not frequently open the app?
Explain your strategy for building and deploying this system. Include your plan for data collection, graph construction, modeling approach, offline evaluation, integration with an existing pipeline, and impact measurement. Describe how you would handle edge cases, such as users with minimal interaction history, and how you would balance scalability with model accuracy.
Detailed solution
Data collection and graph construction
Construct a graph from user-restaurant interactions, where each user and each restaurant is a node, and an order or view is an edge. Use data over a period of about one year. Include user features (e.g., historical ordering patterns), restaurant features (e.g., cuisine type, rating), and edge information (e.g., frequency of orders).
V represents all users and restaurants as nodes. E represents relationships like orders or views. Each node has attributes (user or restaurant metadata). Each edge indicates an interaction between a user node and a restaurant node. This graph representation captures shared user preferences and restaurant similarities.
GNN model architecture
Message passing shares node features with neighboring nodes. Each node aggregates these incoming messages and updates its own representation. Stacking multiple layers allows deeper patterns. At each layer, a user node learns from its direct neighbors and the neighbors' attributes. This iterative approach highlights collaborative filtering patterns.
An Identity-aware Graph Neural Network (ID-GNN) can capture the center node's identity in the process. For a given user node, define an ego network that includes the node and its neighbors. Apply different sets of parameters to the center node and neighbor nodes. This approach preserves the user node's unique identity while leveraging neighbor information.
Model training
The core task is link prediction between user and restaurant. Train the model to predict the probability that a user will order from a restaurant in the next few days. The encoder produces latent embeddings of users and restaurants. The decoder uses these embeddings to generate a probability score. Optimize using a suitable loss function (e.g., binary cross-entropy).
Train with batches of user-restaurant pairs labeled as positive (user made an order) or negative (randomly sampled user-restaurant pair that was not observed). After training, the model yields user embeddings and restaurant embeddings.
Offline evaluation
Compare the GNN model performance against a baseline model. Evaluate metrics like MAP@k (Mean Average Precision at k) and MRR@k (Mean Reciprocal Rank at k) with multiple values of k, focusing on top-k recommendations. Track how many times the correct restaurant ranks among the top recommendations.
Integration with notification pipeline
Run the GNN-based system on a daily schedule. Compute user and restaurant embeddings offline. For each user, fetch restaurants within a certain geographical radius. Use the embeddings to score each user-restaurant candidate. Apply post-processing filters to maintain diversity, remove low-rated options, and remove redundant brands. Send notifications highlighting the top recommendations.
Observed impact
Notifications with personalized “Try Something New” suggestions led to improved engagement metrics. Users reopened the app more frequently, indicating that the recommendations aligned with their preferences. Retention also improved among previously inactive users.
Handling cold-start and minimal data
Use GNNs that naturally incorporate neighbor information. Even when a user has sparse history, the neighborhood context from similar users can inform meaningful recommendations. For new restaurants, incorporate metadata and external signals (e.g., location popularity) for initialization.
Scalability considerations
Limit the size of the graph by focusing on recent data or top interactions. Pre-compute embeddings offline. Restrict candidate restaurants to a user’s last known delivery areas. Maintain an efficient pipeline to handle embedding generation, retrieval, and scoring within daily or weekly intervals.
Follow-up question 1
How would you address the potential issue of user embeddings becoming stale if a user's preferences change rapidly?
Answer Retrain or refresh embeddings on a frequent schedule. If retraining daily is expensive, implement incremental or mini-batch updates to only refresh embeddings of the users with significant new interactions. Combine GNN outputs with a short-term preference model capturing sudden shifts in tastes (e.g., if a user starts ordering vegan meals, highlight more vegan restaurants next day).
Follow-up question 2
How would you handle the risk of recommending the same restaurant too often, which can annoy users?
Answer Diversify restaurant suggestions. Track how often a user sees a certain restaurant. Penalize repetitions in the ranking function. Add constraints on categories or cuisines to rotate new options. Combine GNN scores with diversity penalties to ensure variety in top recommendations.
Follow-up question 3
If the real-time notification system is a batch process (once per day) but a user’s new interaction data is generated throughout the day, how would you incorporate those new signals to keep recommendations relevant?
Answer Capture new data in a temporary buffer. For that user, combine stored GNN embeddings with lightweight real-time features (e.g., new ordering patterns) using a simpler model at serving time. This can override or adjust stale signals. The next day’s offline GNN run incorporates the new data for a more comprehensive update.
Follow-up question 4
How would you explain the GNN's decisions to business stakeholders or product teams that want clear visibility into why certain restaurants are recommended?
Answer Highlight which users or restaurants are most influential in updating a target user node's embedding. Show relevant graph relationships, such as “Similar users ordered from these restaurants.” Provide localized interpretations that describe user feature similarities or shared past ordering patterns. Emphasize that GNN uses message passing to gather signals from neighbors, and indicate which neighbors contributed most.
Follow-up question 5
What operational checks would you institute to ensure model performance does not degrade over time?
Answer Continuously monitor engagement metrics like click-through rate and orders resulting from notifications. Run offline evaluations on rolling time windows. Establish alerts if performance drops below a threshold. Periodically retrain and test with fresh data. Compare against stable baseline models to ensure that the GNN approach maintains or improves performance.
Follow-up question 6
How would you handle differing user contexts, like a user on vacation or using a different address?
Answer Maintain embeddings for multiple addresses per user. Incorporate location-based context in the embedding retrieval step. If a user is on vacation in a new city, fall back to generic restaurant popularity for that region, or do a partial synergy between existing preference embeddings and local top restaurants. Capture these ephemeral contexts in the post-processing filters.
Follow-up question 7
How would you adapt the GNN approach to generate recommendations for other types of notifications, like special promotions or new feature announcements?
Answer Use the same user embeddings from the GNN framework. Create additional edge types for promotions clicked or feature usage. Retrain or fine-tune the GNN to predict user engagement with promotional content. Filter the final candidate set to target specific user segments that match the promotion category. Keep the same message passing and aggregation logic, but adjust the link prediction task to incorporate promotional edges.