TLDR: The research paper by Morgan Allen and Dr. Paul Savala investigates how machine learning models not only predict MLB game winners but also infer the “strength” of a win, measured by score differential. They train various models on a comprehensive dataset, demonstrating a clear relationship between predicted win probability and the final score margin. The study also explores the application of these probabilities to run-line betting, showing that optimized betting strategies, which involve selective wagering based on predicted win probabilities, can yield positive returns, unlike naive approaches.
Major League Baseball (MLB) has long been a sport deeply intertwined with statistics and analytics. From Billy Beane’s pioneering work with the 2002 Oakland Athletics to today’s complex algorithms, teams leverage advanced models for everything from player development to ticket sales. A recent research paper, “Assessing win strength in MLB win prediction models”, delves deeper into this analytical realm, exploring not just which team will win, but by how much.
Authored by Morgan Allen and Dr. Paul Savala from St. Edward’s University, this study extends previous work in MLB win prediction by training a comprehensive set of machine learning models on a vast dataset. Unlike some earlier studies that used smaller datasets or had methodological issues, Allen and Savala ensured a robust approach by collecting extensive player and team statistics from 2001 to 2019, carefully splitting data by season to avoid overlap between training and testing sets.
Beyond Simple Wins: Understanding Win Strength
The core innovation of this research lies in its focus on “win strength,” measured by score differential. Intuitively, if a model predicts a team has a 90% chance of winning, one might expect a significant victory margin. Conversely, a 51% win probability suggests a much closer game. This paper rigorously investigates whether machine learning models can indeed capture this relationship between predicted win probability and the actual strength of the win.
The researchers trained six different families of models: logistic regression (LogR), support vector machines (SVM), K-nearest neighbors (KNN), gradient boosting decision trees (XGB), and artificial neural networks (ANN). They also compared their models against a baseline model that always predicts a home-team win and the well-known FiveThirtyEight (FTE) ELO-based model. The models were trained on data from the 2001-2015 seasons and tested on the held-out 2016-2019 seasons, excluding playoffs.
Key Findings: Models Understand the Game Deeper
All developed models significantly outperformed the simple home-team win baseline. Logistic Regression, in particular, showed top performance across various metrics like accuracy, AUROC (Area Under the Receiver Operating Characteristic curve), log-loss, and Brier score. Importantly, the study confirmed a positive relationship between a model’s predicted win probability and the final score differential. This means that when a model predicts a higher chance of winning, it generally correlates with a larger victory margin.
For games predicted to be “toss-ups” (win probability near 50%), the models accurately correlated with close final scores. While there was still variation in individual game outcomes, the average score differential for these games was consistently low. Conversely, when models predicted a dominant win (e.g., 75-100% probability), the average score differential was significantly higher, often between two and four runs, and even higher for dominant away team predictions.
The research also touched upon ensembling, a technique where multiple models’ predictions are combined. It was noted that while individual models achieved around 63% accuracy, theoretical maximum accuracy through optimal ensembling could reach approximately 75%, suggesting further potential for improvement by combining diverse, high-performing models like XGB.
Practical Application: Run-Line Betting
A significant practical application explored in the paper is run-line betting, where a team must win by a certain margin (typically 1.5 runs). The study demonstrated that a naive betting strategy, simply betting on the team with a predicted win probability of 0.5 or greater, resulted in substantial losses. However, by optimizing the betting strategy with specific high and low probability cutoffs – only betting on games where the predicted win probability was confidently high or low, and abstaining from betting on toss-up games – positive returns were achieved, even into double digits.
This optimized approach, using the LogR model, showed that positive returns were possible by wagering on a small percentage of games (0.5% to 5% of all games, or roughly 5 to 120 games per season), with the highest returns often realized when betting on 1% to 2% of games.
Also Read:
- Predicting Transit Delays: How AI Learns from Unstructured Alerts
- AI for Game Design: Automating Monster Level Estimation in Pathfinder RPG
Conclusion
Allen and Savala’s research provides compelling evidence that machine learning models can develop a nuanced understanding of MLB game outcomes, moving beyond simple win/loss predictions to infer the strength of a win. This deeper insight has significant implications for both sports analytics and strategic betting, demonstrating that with careful model selection and optimized strategies, positive returns in run-line betting are achievable. The findings pave the way for future work in refining ensembling techniques and exploring more sophisticated betting strategies.
