ML Interview Q Series: Bayesian Probability Update for Rain Given Multiple Unreliable Testimonies.
Browse all the Probability Interview Questions here.
Three people in Seattle each claim it is raining, with each of them having a 1/3 chance of lying. If the prior probability of rain on any day is 0.25, what is the probability that it is actually raining, given that all three said it was raining?
Short Compact solution
By applying Bayes’ theorem, the probability that it is raining given all three friends say “yes” is found through:
Comprehensive Explanation
The problem is a straightforward application of Bayes’ theorem, which states that the posterior probability of an event (in this case, rain) given some observed evidence (three “yes” responses) is proportional to the likelihood of observing that evidence if the event is true, multiplied by the prior probability of the event, all divided by the total probability of observing the evidence under both the event and its complement.
Finally, we form the posterior probability by dividing the numerator (the joint probability under R) by this total probability. The arithmetic simplifies neatly to 8/11.
The result makes intuitive sense because hearing unanimous “yes” answers significantly increases the chance of rain relative to the prior of 0.25. Each friend’s statement is moderately reliable (only a 1/3 chance of lying), and it is unlikely that all three would coordinate the same lie if it is not actually raining. Hence, the posterior probability jumps substantially above the prior.
A short Python snippet to verify this numerically:
p_rain = 0.25
p_lie = 1/3
p_truth = 1 - p_lie
# Probability all say "yes" if it's raining
p_all_yes_if_rain = p_truth**3
# Probability all say "yes" if it's not raining
p_all_yes_if_not_rain = p_lie**3
numerator = p_all_yes_if_rain * p_rain
denominator = numerator + p_all_yes_if_not_rain * (1 - p_rain)
posterior = numerator / denominator
print(posterior) # Should print 0.7272727... which is 8/11
What if each friend had a different probability of lying?
How would the result change if the prior probability of rain were higher or lower?
The posterior probability is heavily influenced by the prior. If the prior for rain were extremely low, even unanimous agreement might not move the final probability as high. Conversely, if the prior is higher, the posterior probability given unanimous “yes” would be even larger. In all such cases, the formula stays the same: changing P(R) changes the numerator and denominator in exactly the same approach, but numerically you might see different final outcomes.
How do we extend this to more than three friends?
To generalize to n friends each with probability p of telling the truth, you would raise p to the power of n for the scenario where it is raining, and similarly raise (1−p) to the power of n for the scenario where it is not raining. The full Bayes’ calculation remains consistent: multiply each by its respective prior, add them in the denominator, and divide.
Why might the posterior probability be so high given the prior is only 0.25?
What if the assumption of independence is violated?
All of these calculations rely on the assumption that each friend’s truth-telling or lying is independent of the others. If their responses are correlated (for example, two friends might be more likely to give the same answer if they consult with each other), the straightforward multiplication of probabilities would not hold. One would need a joint probability model for their responses. Correlation often makes the combined evidence less powerful than it appears under independence, so the posterior probability could be lower or higher than our calculation, depending on how the correlation manifests.
Below are additional follow-up questions
What if the reliability of each friend changes day by day or hour by hour?
If each friend’s probability of lying is not fixed but varies over time, the straightforward product approach no longer holds for the current day’s statements. You would need to know the specific probability of lying on that particular day (or hour) to properly compute the conditional probabilities. A Bayesian update can still be done, but the prior for each friend’s truth-telling or lying would need to be adapted to that day’s (or hour’s) reliability. One subtle pitfall is that if you try to average the reliability over a longer period, you might end up with an inaccurate estimate for the day in question if the day’s conditions are unusual. In real-world scenarios, one might use historical data, personal knowledge, or other metadata (for example, if a particular friend is more truthful in the morning vs. the evening) to dynamically set their lying probabilities.
How does the model change if we suspect that the three friends are coordinating their answers?
How might we handle a scenario in which a friend occasionally says “I don’t know” instead of “yes” or “no”?
A friend providing an ambiguous or “I don’t know” response introduces a third possible outcome rather than just a binary yes/no. To use Bayes’ theorem, you must define the probability of each friend giving “I don’t know” under both raining and non-raining scenarios. You might decide that “I don’t know” could be due to uncertainty, or perhaps they are unsure or refusing to answer. Mathematically, you would update your likelihood function to accommodate three possible responses from each friend (yes/no/don’t know) and build a more complex likelihood model. A common pitfall is improperly lumping “I don’t know” with “no” and thus losing potentially valuable information about the friend’s genuine uncertainty.
What if the friends have different biases about rain, such as exaggerating or understating certain weather conditions?
If some friends have a bias to over-report rain (e.g., a tendency to say it’s raining even when they are unsure) or under-report (they tend to say it’s not raining even if it might be), these biases shift their personal “lying” or “error” probabilities in ways that are not symmetric. In this situation, each friend would have a distinct likelihood function for the evidence, for example:
Friend A might say “yes” with higher probability even on days it’s only lightly drizzling (so effectively they rarely say “no”),
Friend B might rarely say “yes” unless it’s really pouring.
You would need separate parameters for each friend’s personal tendencies, and your Bayesian update would treat each friend’s statement with its own conditional probabilities. Failing to account for these biases can lead to overconfident or underconfident posterior probabilities because you assume a uniform lying probability when in reality each friend’s reliability might vary with the weather intensity or their personal traits.
Can we incorporate extra data, such as a weather forecast, into this Bayesian calculation?
Absolutely. You can treat the weather forecast (for instance, a meteorological model that provides a 40% chance of rain) as an updated prior or as an additional evidence source. If it’s an updated prior, you would replace the baseline P(R)=0.25 with something else like P(R)=0.40 from the forecast. Alternatively, if you want to treat the forecast as additional evidence rather than a prior, you can combine it with the friends’ statements in a larger Bayesian network. The subtlety is that forecasts might themselves be uncertain, correlated with your friends’ knowledge, or rely on the same underlying weather data. A real-world pitfall arises if you double-count the same source of information (e.g., your friends heard the same forecast you’re using to adjust your prior).
What if we only trust a friend’s report if they are physically located in Seattle at that very moment?
The reliability of a statement might also depend on the friend’s current context—specifically whether they can directly observe the weather. If one friend is traveling and only hears about the rain secondhand, their statement might be less reliable. In a Bayesian sense, you need to condition each friend’s probability of telling the truth on whether they can directly observe the weather. If a friend isn’t in Seattle, you could lower the probability of truth or consider the possibility that they’re guessing. A pitfall is forgetting to adjust for location-based knowledge, leading to an overly optimistic weighting of that friend’s statement.
Could the prior probability for rain differ depending on the season or location in Seattle?
In reality, the probability of rain in Seattle is not a fixed 25% year-round; it varies across seasons and weather patterns. If this question is considered over a longer timescale, you should use a seasonally adjusted prior. During a typically wet season, the prior might be much higher (e.g., 50%), whereas during a dry season it might be lower. One must ensure to update the prior accordingly, or else the posterior might systematically underestimate or overestimate the true probability. An edge case is if the climate itself is changing over the years, so the historical record might not reflect the current or future conditions accurately.
What if the event has multiple states, such as drizzle, moderate rain, or heavy rain?
How do we prevent numerical underflow or overflow if the probabilities are extremely small for large numbers of friends?
Could we apply a frequentist approach to this problem, or is a Bayesian approach more suitable?
A frequentist approach might frame the question differently, often focusing on the probability of seeing this pattern of “yes” responses if the true state is rain or no rain. This might be akin to a hypothesis test: “Is it more likely that it’s raining or not, given we observed three ‘yes’ answers?” But to get a probability that it is actually raining, you typically need a prior in some form—hence the natural fit for a Bayesian perspective. A common pitfall is mixing frequentist p-values (which measure data likelihood under a null hypothesis) with a desire to get a posterior probability of an event being true, which requires a Bayesian approach.
Below are additional follow-up questions
What if the three statements arrive one after the other, and we want to update our belief incrementally as each statement comes in?
When statements are delivered sequentially rather than simultaneously, you can apply Bayesian updating after each new piece of evidence. You start with the prior (for example, 25% chance of rain), then update that prior using the first friend’s statement to get a posterior. That posterior becomes your new “prior” for the second friend’s statement, and so on. Concretely, if the first friend says “yes,” you compute:
The probability the first friend would say “yes” if it is raining, multiplied by your prior P(R).
The probability the first friend would say “yes” if it is not raining, multiplied by P(R′).
Normalize to get the updated posterior P(R∣First says yes).
Then for the second friend, you take that updated posterior as your new baseline. A subtle pitfall is assuming each statement is conditionally independent given “rain” or “no rain.” If the second friend is influenced by the first friend’s answer, then you can’t simply multiply probabilities. One must incorporate the conditional dependencies carefully. In a real-world scenario, if the friends overhear each other, or if you reveal what previous friends said before asking the next, independence is broken, and your sequential update must reflect that correlation.
How do we handle a situation where a friend might be motivated to mislead you deliberately, rather than lying randomly?
In adversarial settings, the notion of a fixed “one-third chance of lying” becomes too simplistic. A friend may lie whenever it serves a particular goal—for instance, if they want you to carry an umbrella unnecessarily or to stay indoors. This could shift their lying probability depending on context, turning it from a simple constant to a strategy-based model. A Bayesian framework can still be used, but you need a game-theoretic perspective or an adversarial model that estimates how likely a friend is to lie in different situations. A subtle trap is assuming independence or a constant lying probability when the friend’s motivation is dynamic. In practice, you’d look for patterns in that person’s statements over time or other signals that might indicate strategic deception.
What if friends can give partial or ambiguous statements, such as “I see some clouds, but I’m not entirely sure it’s raining”?
When statements are not strictly “yes” or “no” but contain some level of uncertainty, you can model each statement as having a probability distribution over possible interpretations. For instance, “I see some clouds” might correspond to a higher probability of rain than a pure “no,” but not as high as a pure “yes.” In Bayesian terms, you would assign conditional probabilities of hearing that ambiguous statement given it is raining or not raining. Then the total likelihood of that statement would factor into your posterior update. A real pitfall arises if you treat the partial statement simply as a “yes” or “no,” throwing away nuance. In practice, you might keep a more granular representation, for example, P(Ambiguous∣R) vs. P(Ambiguous∣R′), and carefully update your belief about R.
Could we incorporate different costs or utilities for believing it is raining when it is not, versus disbelieving it is raining when it is?
Yes, you can extend this framework to a decision-theoretic viewpoint. Even after computing P(R∣All say yes), you might not strictly pick the hypothesis with the higher probability. Instead, you choose the action that maximizes expected utility or minimizes expected loss. For instance, maybe the “cost” of wrongly assuming no rain is very high (e.g., getting drenched), whereas the cost of wrongly assuming rain is relatively low (e.g., carrying an umbrella unnecessarily). Then you weigh these potential costs in your final decision. The subtlety is that a correct Bayesian posterior is only the first step. You must multiply that posterior by the payoff (or loss) associated with each decision outcome. Ignoring cost asymmetries can lead to suboptimal real-world decisions, such as always trusting the statements or always ignoring them.
What if we try to model each friend’s statement as a small logistic regression, where “yes” or “no” is predicted from some features (e.g., time, location, friend’s history), and then combine outputs?
In a machine-learning context, each friend could be a probabilistic classifier that outputs P(yes∣rain) and P(yes∣no rain) based on observed features. You’d then treat these outputs as likelihoods, and apply Bayes’ rule similarly. In practice, you might learn each friend’s parameters via historical data—how often does friend ii say “yes” when it is indeed raining? The subtlety is ensuring you have an unbiased dataset of real “rain” vs. “no rain” conditions to train on. If your training data is incomplete or skewed (say you only recorded outcomes in certain seasons), your logistic regression estimates might be inaccurate. Additionally, logistic regression typically assumes independence among features, so if friends’ features overlap or are correlated, you might need a more sophisticated model such as a hierarchical Bayesian approach.
What if the event “rain” is actually a continuous or near-continuous variable, like rainfall intensity, and each friend’s statement is a thresholded observation?
If rainfall is measured as a continuous variable (e.g., millimeters per hour), each friend’s “yes” might mean “rainfall is above a certain personal threshold.” In this scenario, you could model each friend’s statement as the outcome of comparing the continuous rainfall variable X to that friend’s threshold. Bayesian updating would then integrate over the distribution of X, using the probability that a friend sees rainfall above or below their threshold. One pitfall is that each friend’s threshold might differ (one friend might say “yes” at even a light drizzle, another only says “yes” if it’s pouring). If you incorrectly assume a uniform threshold, your posterior can be skewed, either overestimating or underestimating the true rainfall intensity.
How do we incorporate a dynamic prior that changes hour by hour as the weather conditions shift?
Instead of a static 25% prior, you can use a time-varying prior P(R∣time t), perhaps from a real-time weather model or meteorological data. Then, at each hour t, you compute P(R∣Friends’ statements, time t). The posterior at hour t might then serve as the prior for hour t+1 if you assume the weather transitions from one hour to the next. For example, you could have a Markov chain describing how rain transitions to no rain (and vice versa) with certain probabilities. A subtle trap is double-counting your friends’ statements when building both the prior and the likelihood. If your dynamic prior already incorporates some of their earlier statements, you must be careful not to incorporate the same evidence again.
Could we approach this problem using multi-armed bandits if we repeatedly query friends about the weather on different days?
In multi-armed bandits, you have multiple options (“arms”) and you want to pick the arm with the best expected reward. Here, each “arm” might represent asking a particular friend, or relying on a specific combination of friends. You learn over repeated trials (different days) which friend is more reliable and allocate more “pulls” (i.e., queries) to that friend. Eventually, you converge on a strategy that picks the friend(s) with the highest likelihood of giving a correct answer. The subtlety is that the reliability might change over time, so you must balance exploration (testing reliability again) with exploitation (trusting the historically most reliable friend). Another challenge is that the “reward” (i.e., correctness of the friend’s statement) might only be verifiable if you have a ground truth—like a weather report at day’s end—so you need a feedback mechanism to update your bandit approach.
How would we test a hypothesis that all three friends are always truthful or always lying by collecting data over many days?
If we ask each friend multiple times on the same day, how should we combine their repeated statements?
Multiple statements from the same friend on the same day can be correlated. If the weather has not changed, a friend who decides to lie might repeat that lie consistently. Conversely, a friend telling the truth might give the same correct response multiple times. If you treat these repeated statements as independent confirmations, you risk inflating your confidence. One way to model this is with a hierarchical approach: each friend either decides to tell the truth or lie for that day, and then all statements from that friend follow accordingly. This means the repeated statements on a single day are not truly independent events. The subtle pitfall is double-counting the same friend’s statements as separate evidence when, in reality, it’s essentially a single decision repeated multiple times.
