On Predictive Models and the Problem of Choice
The setup is deceptively simple. There are two boxes in front of you. Box A is clear and contains $1,000. Box B is opaque. A predictor has already placed either $1,000,000 or nothing inside Box B based on a prediction of what you will choose. You get to pick either just Box B or both Box A and Box B. If the predictor correctly guessed you would take only Box B, it put $1,000,000 inside. If it guessed you would take both boxes, Box B is empty. The predictor has been right nearly every time in every description of this problem. So what do you do?
The Newcomb Collins Paradox Why Predictive Algorithms Force Us To Question Free Will And Optimal Strategy
This is Newcomb's Paradox, and it tears apart the foundation of expected utility theory. The two schools of thought don't just disagree on the answer. They disagree on what the question is even asking. Evidential decision theory says: take only Box B. Given that the predictor has an almost perfect track record, the correlation between your choice and the contents of Box B is extremely strong. Choosing one box evidences that the million dollars is in there. Choosing both boxes evidences that it isn't. The math pushes you toward one box every time. Causal decision theory says: take both boxes. The predictor has already made its move. The money is either in Box B or it isn't. Your choice now cannot causally affect what was placed there hours or days ago. You are deciding between getting $1,000 more (by also taking Box A) or not. Two-boxing strictly dominates regardless of what the predictor predicted.
I spent roughly three years working on incentive design for recommendation systems before I really encountered the practical side of this problem. The issue came up when we were building a feature that predicted user churn and then adjusted content delivery accordingly. The model was accurate enough that we could reliably predict which users would stay and which would leave based on early engagement signals. And then a product manager asked a question that basically collapsed into Newcomb's problem: if the model predicted someone was going to churn, should we invest retention resources in them? The causal argument is that investing in a user the model predicts will leave is wasteful because the prediction is based on observable behavior, not an immutable fate. But the evidential argument is equally compelling: the same signals that predict churn are the ones that retention efforts actually counteract. Users who show early disengagement patterns are precisely the ones most responsive to intervention. The model isn't reading the future. It's reading the present. And the present is malleable. What actually happened in production is that we built a system where the prediction and the intervention were deliberately decoupled. We used one model trained on historical data to generate predictions, and a separate causal inference layer to estimate the treatment effect of our retention actions. This is essentially a meta-level workaround for the paradox. We accepted that the raw prediction was evidentially useful but causally inert, and we layered a second model on top that answered a different question: what changes outcomes, not what forecasts them.
Get the Full Details
Here is the counter-intuitive part that most people miss. The paradox isn't really about free will. It's about the difference between type-1 and type-2 predictions in any system where the predictor is part of the environment the agent inhabits. When the predictor is external and fixed, causal decision theory wins cleanly. The money is already there or it isn't. When the predictor is internal to the system—when your decision process and the prediction mechanism share information—the evidential approach often produces better real-world outcomes because it treats the prediction as a correlated signal rather than a causally irrelevant artifact. Another thing beginners consistently get wrong is assuming the predictor must be supernatural for the paradox to matter. It doesn't. A sufficiently sophisticated behavioral model trained on enough data functions as a predictor for all practical purposes. I've seen this exact structure emerge in credit scoring, insurance underwriting, and hiring pipelines without anyone explicitly invoking decision theory. The system predicts an outcome, acts on that prediction, and then the prediction becomes self-fulfilling or self-defeating depending on how the agent responds. That's Newcomb's Paradox in a suit. The honest limitation is that this framework breaks down completely when the predictor's accuracy drops below a certain threshold. Once the correlation between prediction and outcome becomes weak enough, evidential decision theory starts recommending choices that are objectively worse. I'd estimate the cutoff is somewhere around 55 to 60 percent predictive accuracy, below which two-boxing becomes the rational move regardless of which decision theory you subscribe to. Most real-world models hover well above that, which is why the paradox stays relevant in practice.
There is also a scenario where the paradox dissolves entirely and most people don't realize it. If you can communicate with the predictor before making your choice, or if the predictor can observe your decision process in real time, the whole tension evaporates. The predictor simply outputs whatever you're about to do, and you're left with a trivial optimization problem rather than a deep philosophical one. This is why the paradox is almost always presented with the constraint that the prediction is sealed before your choice. Remove that constraint and the problem becomes boring. In practice, the takeaway is straightforward but uncomfortable. When you're building or using predictive systems, your first question shouldn't be what the model predicts. It should be whether your actions can change what the model predicted, and whether the model's prediction is being used as evidence of a fixed future or as a snapshot of a malleable present. The answer to that question determines which version of the paradox you're actually dealing with, and which decision rule will actually serve you.
