Failed Approaches on Problem Solving#
Even with careful planning, data-driven projects sometimes fail or underperform. Below are three examples of failed approaches, the datasets used, and how I improved my thinking based on those experiences.
1. Overfitting with Limited Data#
Scenario: Attempting to build a classification model on a small dataset with insufficient validation.
Dataset Link: Titanic - Machine Learning from Disaster (Kaggle)
What Happened#
Trained a complex neural network on a very small subset of training data.
Achieved near-perfect accuracy on training data but performed poorly on unseen data (classic overfitting).
Lessons Learned#
Cross-Validation: Always use proper cross-validation to estimate out-of-sample performance.
Simpler Models First: Sometimes a simpler model (e.g., logistic regression) generalizes better.
Data Quantity & Quality: Acquire more data or perform data augmentation if possible.
2. Ignoring Missing Data#
Scenario: Building a house price prediction model but dropping columns with missing values outright.
Dataset Link: House Prices - Advanced Regression Techniques (Kaggle)
What Happened#
Many records had partial information, and ignoring them reduced the dataset size significantly.
Important features were lost, leading to a biased and underperforming model.
Lessons Learned#
Handle Missingness: Use imputation strategies (mean, median, mode, or advanced imputation).
EDA: Thorough exploratory data analysis often reveals patterns in missingness.
Domain Expertise: Sometimes data is “missing” because it’s simply not recorded or irrelevant; domain experts can clarify.
3. Using the Wrong Metric for Credit Risk#
Scenario: Evaluating a credit risk model solely on accuracy.
Dataset Link: Give Me Some Credit (Kaggle)
What Happened#
Because the majority of borrowers did not default, the model had high accuracy but did a poor job identifying those who would default.
This led to missed detections of risky borrowers.
Lessons Learned#
Imbalanced Datasets: In cases like fraud detection or credit risk, accuracy is not enough.
Alternative Metrics: Precision, recall, F1-score, or ROC-AUC might be more relevant.
Business Context: Always align metrics with the actual cost of false positives vs. false negatives.
How I Improved My Thinking#
Iterative Approach
I now iterate on model design, data preprocessing, and metric selection early and often, incorporating feedback loops.Metric Alignment
I choose metrics that reflect real-world objectives—whether it’s reducing false negatives in fraud detection or capturing critical edge cases in healthcare.Continuous Learning & Validation
I retrain and revalidate models to ensure they remain accurate over time. Business environments and data distributions can change, so static models often fail.Domain Collaboration
Engaging domain experts helps clarify nuances in data collection, business objectives, and regulatory constraints.
By embracing these lessons, I’ve become more effective at building robust, ethical, and business-aligned ML solutions.