The COST (change one single thing) approach to experimentation has be shown, at least 100 years ago, to be an ineffective approach to experimentation. Yet it persists. Why? The A/B testing approach is a variant of COST, but with fewer drawbacks, though still inefficient in capturing all there is to learn. Let’s dive in by introducing common terminology to see why. If design of experiments is a familiar topic for you, please skip on to section 3.
Experiments can be applied wherever you might imagine, so the examples used below come from baking, learning a new language, the sciences…
R² can be calculated before even fitting a regression model, which doesn’t make sense then to use it for judging prediction ability. Also you get the same R² value if you flip the input and output around. Again, this is nonsensical for a prediction metric.
The intention of your regression model is the important factor for choosing an appropriate metric, and a suitable metric is probably not R².
This article explains which better alternatives exist: the standard error, confidence intervals and prediction intervals.
Historical data is collected and a relationship between the input 𝑥 and the output 𝑦 is…
To eliminate this waste, consider 2 points.
Trust educational science, which has shown spread-out learning to be more effective than cramming in new concepts.
Secondly, business decisions are often based on return on investment (ROI). Training choices based on ROI will make you more accountable and force you to consider more effective approaches.
Training your colleagues, or employees seems to be a checkbox item — something part of their development plan or performance review.
What is the manager to do when faced with a group of staff that need skills developed in a rapidly changing and digitizing corporate world? …
Kevin is an engineer & data scientist with 20+ years experience across a variety of industries. Still continually learning new ways to extract value from data.