Exploratory Data AnalysisFriday 08 January 2021
When I start working on a machine learning project my first impulse is always to try to fit some models. At the end of the project I always remember how important exploratory data analysis is, and wish I had remembered sooner. Even on things where EDA doesn't seem necessary it usually is.
I have been working on an instance detection challenge and what use will EDA be on a dataset of annotated images ? It turns out a lot. After doing some EDA I found that many of the annotations were wrong, and simply by correcting them I was able to greatly increase my model's performance.
In addition, by doing some EDA on the predictions from a fitted model I was able to identify some common causes of errors and attempt to address them.