KnowlegeSEEKER provides an extensive set of model performance evaluation and comparison capabilities. Model Analyzer capabilities perform the following tasks:
- Analysis of how well your model fits a set of observations
- Model sensitivity evaluation of true positive rate vs. false positive rate
- Model calibration testing
- Model comparison based on different data samples
- Model comparison based on 2 or more models on the same data
- Model comparison results over time
- Estimate Return on Investment (ROI) based on model performance and business-specific costs and expected returns
Charts are both versatile and clear, and multiple models can be added to the same chart to allow easy visual comparison. Charts are customizable and can be easily copied and pasted into Microsoft Office® applications.
These tasks are enabled with supported Model diagnostics algorithms and charts:
- Lift and Cumulative Lift charts
- Goodness-of-Fit measures: Kolmogorov-Smirnov (K-S) test charts and Hosmer-Lemeshow tests
- Relative Operating Characteristic (ROC) curve
- Automatic calculation of Gini Index and area under the curve
- Profit curve
- Accuracy, Bias and Error charts, and Scatter plots for models with continuous dependent variables
Additional KnowledgeSEEKER validation options and capabilities are:
- Model Analyzer accepts inputs in the form of models and in the form of datasets with validation results—to allow comparison of results of models developed in KnowledgeSEEKER with those developed in tools such as SAS and SPSS.
- All models can be validated against a validation partition or dataset to produce a report with comparison statistics and confusion matrices.
- Validation results of Decision Tree models display differences among the training and validation samples on a node-by-node basis.