When reviewing your prediction scorecard, you see a prediction score. If you’re not satisfied, here are some ways to improve the prediction quality.
If your prediction quality is Great, that's great! No need for improvement, but check your scorecard again from time to time to keep an eye on the quality. If the quality is Good when your prediction runs on live and not estimated data, you’re all set. If the prediction quality is OK, Too Low, or Too High, read on.
First, notice whether the Prediction Quality title says “Estimated” or not. If so, it means that you haven’t enabled your prediction yet, or it’s enabled but not yet running on live data. Either way, the quality score is estimated, based on a subset of your example data. It’s typical for the quality to go down after the prediction starts running on live data. For example, if your prediction quality is Great but estimated, check the scorecard again in a month to see how it’s performing with live data.
Here are some considerations for improving a prediction quality.
- Collect more data. Not all quality problems are due to the amount of data. In some cases, you can improve the quality when you have more data in your dataset.
- Try changing the way your data is segmented. For example, make sure your segment includes only records that are relevant to your prediction. If you have two distinct datasets that would each benefit from having its own prediction, create a separate prediction for each segment.
- Make sure your example set contains relevant records for both yes and no examples. Look for possible sources of bias in your data that influence your prediction.
- Make sure you’re not excluding any fields that could provide additional, useful data.
- If your prediction uses formulas, check to make sure they’re correct.
- If your prediction quality is Too High when estimated (before it’s running on live data), it’s likely too good to be true. Review the fields included in your prediction. Do any of them look wrong, or contain data that couldn’t have been known until after the event being predicted? If so, exclude those fields. This type of problem is called hindsight bias. Also look over the Top Predictors on the scorecard. Do they make sense? If any of them look wrong, consider excluding those fields.