FAQ: Outlier Detection and Handling
- When there is an extreme outlier on a particular day (e.g., a data entry error, a one-off promotional event, or an unexpected surge/drop), how does the ML model determine it's an outlier, and how does it handle it for forecasting purposes? Does it ignore it, smooth it, or incorporate it in a specific way? How does the system differentiate between a true outlier and a genuine, but unusual, data point that should be factored into the forecast (e.g., a legitimate peak during a successful promotion)?
- The system doesn’t differentiate between a true outlier and a genuine, but unusual, data point. Both are factored into the forecast, and their impact depends on their duration and magnitude. The current means of handling these situations is the scheduling events feature.The ML treats all historical data points as valid information for predicting future trends. It doesn’t have a built-in mechanism to distinguish between a data point that is a result of a one-time error (a true outlier) and one that represents a genuine, albeit rare, event (a legitimate peak). When a forecast is generated, the system considers the entire history. If a significant peak or dip occurs, it can influence the forecast for subsequent periods, especially if the forecasting model is sensitive to recent data.To prevent a genuine, but unusual, data point from distorting the general forecast, you can use the scheduling events feature. This tool allows you to manually mark specific dates as being affected by a known event, such as a successful promotion, a holiday, or a store-wide sale. Once identified as a known event, the ML will give different weighting to that date when generating predictions, with matching events receiving a higher weighting and average days receiving a lower weighting.