ML Interview Q Series: Approximating Gambling Probabilities Using Logarithms and the Central Limit Theorem
Browse all the Probability Interview Questions here.
Consider again Problem 10E-29. Assume now that the casino owner makes an offer only to the most famous gambler in town. The casino owner lends $1,000 to the gambler and proposes that he make 100 bets on red with this starting capital. The gambler is allowed to stake any amount of his bankroll at any bet. The gambler gets one-fourth of the staked money back if he loses the bet, while he gets double the staked money back if he wins the bet. As reward, the gambler can keep to himself any amount in excess of $1,000 when stopping after 100 bets. Suppose that the gambler decides to stake each time a fixed percentage of 5% of his current bankroll. Use the normal distribution to approximate the probability that the gambler takes home more than d dollars for d = 0, 500, 1,000, 2,500. Hint: consider the logarithm of the size of the bankroll of the gambler.
Short Compact solution
We denote the gambler’s bankroll after the n-th bet by V_n. If the gambler stakes a fraction alpha = 0.05 of the current bankroll on each bet, then at each bet:
With probability 19/37, the gambler loses and retains 1 - 0.75 alpha of the bankroll (because losing three-fourths of the stake leaves 1 - 0.75 alpha = 0.9625 of the previous bankroll).
With probability 18/37, the gambler wins and multiplies the bankroll by 1 + alpha = 1.05.
Thus V_n/V_{n-1} is 0.9625 with probability 19/37 and 1.05 with probability 18/37. Let X_i = ln(V_i / V_{i-1}). Then
E(X_i) = (19/37) ln(0.9625) + (18/37) ln(1.05) = 0.0041086
and
sigma(X_i) = 0.0434898
By the Central Limit Theorem, ln(V_{100} / V_0) is approximately Normal with:
For a > 0, the probability that V_{100} exceeds a V_0 is
P(V_{100} > a V_0) = 1 - Phi( (ln(a) - mu)/sigma )
Plugging in a = 1, 1.5, 2, 3.5 (which correspond to d = 0, 500, 1,000, 2,500 if the initial capital V_0 = $1{,}000) gives approximate probabilities 0.8276, 0.5494, 0.2581, and 0.0264 respectively.
Comprehensive Explanation
Lognormal Modeling of the Bankroll
When a gambler stakes a fixed fraction alpha of the current bankroll and the outcome is a win or loss with certain probabilities, we can model each bet’s multiplier as a random variable. Specifically:
If the gambler loses, the fraction alpha of the bankroll is bet, but only one-fourth of that wager is returned. This yields a bankroll multiplier of 1 - 0.75 alpha.
If the gambler wins, that same fraction alpha is bet but is returned doubled, thus yielding a bankroll multiplier of 1 + alpha.
Because each bet is independent, the overall bankroll V_{100} after 100 bets can be written (starting from V_0) as:
V_{100} = (V_0) * (multiplier_1) * (multiplier_2) * ... * (multiplier_{100})
Taking the natural log transforms products into sums:
ln(V_{100}/V_0) = ln(multiplier_1) + ln(multiplier_2) + ... + ln(multiplier_{100})
Define each X_i = ln(multiplier_i). Since multiplier_i is 0.9625 with probability 19/37 and 1.05 with probability 18/37, we can compute:
E(X_i) = (19/37)*ln(0.9625) + (18/37)*ln(1.05)
Var(X_i) = E(X_i^2) - [E(X_i)]^2, and sigma(X_i) is the square root of Var(X_i).
From the given calculations, these turn out to be
E(X_i) = 0.0041086 sigma(X_i) = 0.0434898
Applying the Central Limit Theorem
By the Central Limit Theorem, the sum of 100 i.i.d. random variables X_i converges in distribution to a normal random variable. Hence
ln(V_{100}/V_0) ~ Normal( mu, sigma^2 )
where
mu = 100 * E(X_i) = 0.41086
sigma = sqrt(100) * sigma(X_i) = 0.434898
Probability of Exceeding a Threshold
We want P(V_{100} > d + V_0). Since V_0 = $1000, “taking home more than d dollars” means V_{100} > 1000 + d. Define a = 1 + (d / 1000). So:
P(V_{100} > a V_0) = P( ln(V_{100}/V_0) > ln(a) )
Because ln(V_{100}/V_0) is approximately Normal(mu, sigma^2),
P( ln(V_{100}/V_0) > ln(a) ) = 1 - Phi( [ln(a) - mu] / sigma )
Substitute mu = 0.41086 and sigma = 0.434898. For example:
For d = 0, a = 1, ln(a) = 0 => Probability ~ 1 - Phi( (0 - 0.41086)/0.434898 )
For d = 500, a = 1.5 => Probability ~ 1 - Phi( [ln(1.5) - 0.41086]/0.434898 )
And similarly for d = 1,000 (a = 2) and d = 2,500 (a = 3.5).
The final approximate probabilities are 0.8276, 0.5494, 0.2581, and 0.0264 for d = 0, 500, 1,000, and 2,500 respectively.
Lognormal Mean and Standard Deviation
Since ln(V_{100}/V_0) is approximately Normal(mu, sigma^2), V_{100}/V_0 is approximately lognormally distributed. For a lognormal random variable:
Mean = exp( mu + sigma^2/2 )
Standard Deviation = exp( mu + sigma^2/2 ) * sqrt( exp( sigma^2 ) - 1 )
In this problem:
mu = 0.41086
sigma^2 = 0.434898^2
Hence we can compute the expected final bankroll and its standard deviation in lognormal terms if needed.
Possible Follow-up Questions
Why do we model the logarithm of the bankroll rather than the bankroll itself?
When the gambler’s capital evolves as a product of random multipliers, taking the logarithm converts multiplicative changes into additive terms. This makes the Central Limit Theorem applicable, so ln(V_{100}/V_0) can be approximated by a normal distribution. Without the logarithm, the distribution of V_{100} can be heavily skewed and not well-approximated by a normal distribution.
What if the fraction alpha were larger than 5%?
If alpha increases, the swings become more pronounced. The loss multiplier 1 - 0.75 alpha and the win multiplier 1 + alpha move further from 1, so the variance of ln(V_{n}/V_{n-1}) grows. A larger alpha typically increases both the upside potential and downside risk. The expected log-growth might even decline if alpha is set too large, because extremely negative outcomes (large fractional losses) pull down the average of the logs.
Could we use the Kelly criterion here?
The Kelly fraction maximizes the gambler’s expected logarithmic growth. If alpha=0.05 is smaller or larger than the Kelly fraction, the gambler is not strictly optimizing growth according to the Kelly framework. One could compute the Kelly fraction by solving for the alpha that maximizes E( ln(1 - 0.75 alpha) * p_lose + ln(1 + alpha) * p_win ), where p_lose = 19/37 and p_win = 18/37.
How do we interpret the normal approximation's accuracy for only 100 bets?
The CLT is an asymptotic result, so exactness is not guaranteed at just 100 bets. Nonetheless, 100 independent bets often suffices for a decent approximation, especially for the sum of i.i.d. random variables whose distribution is not too extreme. One might also compare the normal approximation with a small-scale Monte Carlo simulation for validation.
How might we implement this simulation in Python?
Below is a simple Python code snippet that repeatedly simulates the process and estimates the final probabilities:
import numpy as np
n_simulations = 10_000_00
alpha = 0.05
p_win = 18/37
p_lose = 19/37
V0 = 1000
results = []
for _ in range(n_simulations):
bankroll = V0
for __ in range(100):
bet = alpha * bankroll
if np.random.rand() < p_win:
# Win
bankroll = bankroll - bet + 2 * bet
else:
# Lose
bankroll = bankroll - bet + 0.25 * bet
results.append(bankroll)
results = np.array(results)
for d in [0, 500, 1000, 2500]:
prob = np.mean(results > V0 + d)
print(f"Probability of ending with more than {d} profit: {prob:.4f}")
This brute-force approach gives an empirical estimate of the probability of exceeding each threshold d. We expect the values to be close to those obtained via the normal approximation.
Below are additional follow-up questions
What if the gambler’s bet outcomes are not identically distributed for all 100 rounds?
If the gambler faces changing probabilities, payouts, or stake fractions from bet to bet, the assumption of i.i.d. (independent and identically distributed) multipliers no longer holds. In such a scenario, the Central Limit Theorem (CLT) in its simplest form might be inaccurate. The gambler’s overall capital path would then be modeled as a product of different random factors M_1, M_2, …, M_n, each potentially with its own distribution. One approach is to compute the mean and variance of ln(M_i) for each i separately, sum them, and then rely on a generalized version of the CLT if there is still some underlying independence among the bets. However, if these bets are correlated (e.g., changes in the casino’s payout structure or changes in the probability of winning), then we would need more advanced methods such as the Lindeberg CLT or a delta method approach for correlated variables, or potentially a Monte Carlo simulation if an analytic form is difficult to derive.
A subtle real-world concern is that the house might alter rules or the gambler might start placing side bets that have different odds. This would mean the entire bet distribution and expected log return can evolve over time. In practice, you would track each round’s multiplier distribution precisely, summing up ln(M_i) with the correct mean and variance for that round and ensuring you incorporate any dependencies that might exist between successive rounds.
How could a minimum bet size or table limit affect the analysis?
In many casinos, there is a minimum or maximum bet size (a table limit). If the gambler’s chosen fraction alpha of the bankroll falls below the table’s minimum bet at some point, the gambler can no longer strictly follow the “stake 5%” rule. Conversely, if 5% of the gambler’s bankroll exceeds the table maximum, the gambler is forced to bet less. These constraints break the exact assumption that the gambler always bets exactly alpha times the current bankroll.
A key edge case is if the gambler’s bankroll becomes very small, so 5% of that bankroll is below the minimum bet. At that point, the gambler is compelled to bet an amount larger than 5% (just to meet the minimum). This can increase risk and accelerate ruin. On the other hand, if the bankroll becomes very large, the gambler may be capped at some maximum bet; the resulting fraction of the bankroll bet might be less than 5%, reducing volatility and possibly preventing exponential growth from continuing at the same rate.
Mathematically, the product distribution approach must be adjusted: whenever the fraction alpha times V_n is less than the minimum or above the maximum, the multiplier changes. In a simulation, you could handle this constraint by clipping the bet amount to [min_bet, max_bet]. Analytically, you might approximate the effect or rely on partial sums with piecewise distributions.
How would outcomes change if the gambler can change alpha dynamically based on past results?
If alpha is not fixed but instead adaptively chosen after each bet—maybe the gambler bets more aggressively after a win or more conservatively after a loss—then the multiplier each round depends on the history. The sequence of X_i = ln(1 - 0.75 alpha_i) or ln(1 + alpha_i) is no longer identically distributed and may not even be independent if alpha_i depends on the path of previous wins and losses.
To handle adaptive betting:
Analytical Difficulty: A closed-form expression for the distribution of V_{n} becomes harder to obtain. One might still compute expected log growth by conditioning on possible states of the bankroll, but it is complex.
Stochastic Control or Dynamic Programming: In advanced scenarios, you can frame the problem as a stochastic control problem, optimizing alpha_i subject to the gambler’s risk preferences.
Potential Pitfalls: Adaptive strategies can reduce risk but might also reduce expected returns if they deviate from a mathematically optimal fraction (e.g., Kelly criterion). The gambler also risks overreacting to short-term variance, which might degrade long-run performance.
What if the probability of winning each bet changes during a losing streak?
In reality, some individuals erroneously believe in phenomena like the “gambler’s fallacy”—believing that a long losing streak makes a win more likely. If the gambler changes the betting strategy under the false assumption that the win probability is shifting, the distribution of outcomes is not the same as assumed. Even if the house modifies the actual odds (for instance, promotional bets after a losing streak), that modifies p_win or the payoff structure.
From a mathematical standpoint, if p_win changes to reflect the gambler’s losing or winning streak, the multiplier distribution for each subsequent bet would need to incorporate these new probabilities. The gambler’s final distribution might have heavier tails if an artificially inflated or deflated probability leads to bigger misalignments between actual probability and bet sizing. In such a scenario, an in-depth model of how p_win transitions from state to state is required (a Markov chain approach could be used, or repeated re-estimation of p_win each round).
How does this approach handle the possibility of the bankroll hitting zero (or near zero)?
When using log transforms, a major pitfall is ignoring the fact that the bankroll cannot go negative, and if it hits zero, the gambler is essentially “ruined.” If alpha remains 5% but the bankroll is so small that a single bet might reduce the bankroll to below 1 cent, you are in a discrete domain where eventually the gambler can’t continue at all. The log transform also becomes problematic if the bankroll hits exactly zero because ln(0) is undefined.
In real-world terms, if the gambler’s bankroll is extremely close to zero, the smallest actual currency unit (e.g., 1 cent) might effectively put the gambler out of the game. Strictly speaking, the normal approximation might overestimate the probability of bouncing back from near ruin. A more precise approach for low bankroll values might be:
Absorbing States: Treat zero bankroll as an absorbing boundary in a Markov chain model.
Truncation or Discretization: If the gambler can’t place a bet smaller than 1 cent, the system is discrete and can be simulated or solved exactly (though that might be computationally intensive).
Monte Carlo: A simulation approach explicitly accounts for the possibility of small or zero bankroll and can track the gambler’s ruin more precisely than a continuous lognormal approximation.
Could the distribution of ln(V_{100}/V_0) be significantly skewed or have higher moments that deviate from normal?
Yes. Even though the CLT helps approximate the sum of i.i.d. variables as normal, 100 bets may still be borderline for capturing higher moments accurately. Skewness and kurtosis (heavy tails) can make the normal approximation less accurate. Empirically, if the probability of a big downward jump or an outlier upward jump is non-trivial, the normal distribution might underestimate tail probabilities.
In practice, we often see:
Skew: Log returns can be skewed if one multiplier is significantly larger or smaller than the other.
Kurtosis: If rare but extremely large (positive or negative) multipliers occur, heavier tails result.
Impact: Risk calculations (like VaR or expected shortfall) based on a normal approximation might understate the real tail risk.
For more precise modeling, one could use the Gamma distribution for each multiplier (when appropriate) or a more flexible distribution, then apply exact or numerical convolution or a simulation-based approach.
How would insurance or side bets affect the distribution?
Sometimes gamblers buy side bets or insurance bets, effectively paying a premium that partially offsets the chance of a large loss. From a mathematical standpoint, such additions alter the random multiplier for each round: you lose a small fraction for the insurance premium but possibly recoup part of the loss if the main bet fails. This might reduce the variance of X_i, although it may also reduce the expected value of X_i if the premium is high. The gambler has to reevaluate:
The new E(X_i): It might go down because paying for insurance every bet is a cost.
The new sigma(X_i): It might shrink if the insurance dampens extremes.
The new alpha: The gambler might stake a different fraction alpha to optimize the new distribution.
An edge case is if the insurer can cap certain bets or if the insurance is only valid for a limited number of bets. That changes the distribution in a time-dependent manner, making an i.i.d. assumption inaccurate. Then a piecewise approach or simulation is more suitable.
How do we ensure numerical stability in a computational or simulation approach?
When repeatedly multiplying or taking logs, numerical underflow and overflow can occur:
Underflow: If the bankroll becomes extremely small, floating-point representation might push it to zero, halting further simulation or leading to NaNs when taking the log.
Overflow: If the bankroll grows extremely large, multiplying further can exceed floating-point ranges.
Mitigation: One common technique is to store only ln(V_n). Instead of physically multiplying V_n by a factor, you add ln(factor) to ln(V_n). This way, the exponentiation can be deferred until the end, or partially exponentiated if needed for intermediate steps.
In real-world modeling, you might want to safeguard your code:
Use double precision (float64) carefully.
Check for extremely small or large values and handle them with logical conditions or by storing logs.
How can a Monte Carlo simulation be validated to confirm the analytical approximation?
To validate the normal approximation with a simulation, you would:
Implement the exact random process for 100 bets, with the gambler staking 5% of the bankroll each time.
Repeat for a large number of simulations, e.g., a few million trials, to obtain an empirical distribution of ln(V_{100}/V_0).
Compare the sample mean and variance of ln(V_{100}/V_0) to the analytically derived mu and sigma.
Assess normality by looking at histograms, Q-Q plots, or statistical tests (e.g., Shapiro–Wilk or Kolmogorov–Smirnov).
Examine tail probabilities (e.g., 1 minus the CDF for large values). If the simulated tail differs notably from the theoretical normal tail, you know the normal approximation is less accurate in the extremes.
A key pitfall is ensuring the number of simulations is large enough to capture rare tail events. Otherwise, you might underestimate or overestimate the probabilities in the distribution’s tails.
When might we prefer exact or partial exact methods over the CLT?
Small Number of Bets: If the gambler only places, say, 10 bets (not 100), the CLT approximation may be unreliable. Then you might compute the probability distribution of the bankroll exactly by enumerating all possible outcome combinations or by convolving discrete distributions for each round.
High Bet Count But Highly Skewed Multipliers: Even with 100 bets, if the multipliers have extreme values (e.g., a 0.01 probability of losing 90%), the normal approximation might not capture the tail well. A more specialized stable distribution or a heavy-tailed approach might be more appropriate.
Regulatory or Risk Management Scenarios: In settings like financial risk compliance, regulators may demand that you either use robust stress testing or exact enumeration for certain worst-case scenarios, rather than relying purely on a normal approximation that can underestimate risk.
Could the casino’s perspective on the gambler’s strategy lead to different probabilities or constraints?
From the casino’s viewpoint, if the gambler’s potential for a large win is considered too high, the casino might impose table limits or change payouts. This means that the distribution of V_{n+1}/V_{n} can be dynamic if the casino observes large swings and reacts to them (e.g., restricting bet sizes mid-game). That dynamic environment invalidates the i.i.d. assumption. Additionally:
The casino might demand collateral if large losses are possible.
The gambler might have a limited time window or a mandatory break after certain large bets.
House edge changes: In real casinos, the house usually adjusts payoffs so they retain a definite edge, which might differ from the theoretical simplified probabilities used in the problem.
Any of these real-world constraints drastically alter the gambler’s distribution of outcomes relative to a fixed fraction bet with fixed odds.