Browse Library
Sign In Start Free Trial
Close icon Search icon
Account
Browse Library Advanced Search Sign In Start Free Trial

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • IBM SPSS Modeler Cookbook
  • Toc
  • feedback
IBM SPSS Modeler Cookbook

IBM SPSS Modeler Cookbook

By : Keith McCormick, Abbott
4.4 (20)
close
IBM SPSS Modeler Cookbook

IBM SPSS Modeler Cookbook

4.4 (20)
By: Keith McCormick, Abbott

Overview of this book

IBM SPSS Modeler is a data mining workbench that enables you to explore data, identify important relationships that you can leverage, and build predictive models quickly allowing your organization to base its decisions on hard data not hunches or guesswork. IBM SPSS Modeler Cookbook takes you beyond the basics and shares the tips, the timesavers, and the workarounds that experts use to increase productivity and extract maximum value from data. The authors of this book are among the very best of these exponents, gurus who, in their brilliant and imaginative use of the tool, have pushed back the boundaries of applied analytics. By reading this book, you are learning from practitioners who have helped define the state of the art. Follow the industry standard data mining process, gaining new skills at each stage, from loading data to integrating results into everyday business practices. Get a handle on the most efficient ways of extracting data from your own sources, preparing it for exploration and modeling. Master the best methods for building models that will perform well in the workplace. Go beyond the basics and get the full power of your data mining workbench with this practical guide.
Table of Contents (11 chapters)
close
close
Preface
Preface
What is CRISP-DM?
Data mining is a business process
The IBM SPSS Modeler workbench
Historical introduction to scripting
What this book covers
Who this book is for
Conventions
Reader feedback
Customer support
Free Chapter
1
1. Data Understanding
chevron up
1. Data Understanding
Introduction
Using an empty aggregate to evaluate sample size
Evaluating the need to sample from the initial data
Using CHAID stumps when interviewing an SME
Using a single cluster K-means as an alternative to anomaly detection
Using an @NULL multiple Derive to explore missing data
Creating an Outlier report to give to SMEs
Detecting potential model instability early using the Partition node and Feature Selection node
2
2. Data Preparation – Select
2. Data Preparation – Select
Introduction
Using the Feature Selection node creatively to remove or decapitate perfect predictors
Running a Statistics node on anti-join to evaluate the potential missing data
Evaluating the use of sampling for speed
Removing redundant variables using correlation matrices
Selecting variables using the CHAID Modeling node
Selecting variables using the Means node
Selecting variables using single-antecedent Association Rules
3
3. Data Preparation – Clean
3. Data Preparation – Clean
Introduction
Binning scale variables to address missing data
Using a full data model/partial data model approach to address missing data
Imputing in-stream mean or median
Imputing missing values randomly from uniform or normal distributions
Using random imputation to match a variable's distribution
Searching for similar records using a Neural Network for inexact matching
Using neuro-fuzzy searching to find similar names
Producing longer Soundex codes
4
4. Data Preparation – Construct
4. Data Preparation – Construct
Introduction
Building transformations with multiple Derive nodes
Calculating and comparing conversion rates
Grouping categorical values
Transforming high skew and kurtosis variables with a multiple Derive node
Creating flag variables for aggregation
Using Association Rules for interaction detection/feature creation
Creating time-aligned cohorts
5
5. Data Preparation – Integrate and Format
5. Data Preparation – Integrate and Format
Introduction
Speeding up merge with caching and optimization settings
Merging a lookup table
Shuffle-down (nonstandard aggregation)
Cartesian product merge using key-less merge by key
Multiplying out using Cartesian product merge, user source, and derive dummy
Changing large numbers of variable names without scripting
Parsing nonstandard dates
Parsing and performing a conversion on a complex stream
Sequence processing
6
6. Selecting and Building a Model
6. Selecting and Building a Model
Introduction
Evaluating balancing with Auto Classifier
Building models with and without outliers
Using Neural Network for Feature Selection
Creating a bootstrap sample
Creating bagged logistic regression models
Using KNN to match similar cases
Using Auto Classifier to tune models
Next-Best-Offer for large datasets
7
7. Modeling – Assessment, Evaluation, Deployment, and Monitoring
7. Modeling – Assessment, Evaluation, Deployment, and Monitoring
Introduction
How (and why) to validate as well as test
Using classification trees to explore the predictions of a Neural Network
Correcting a confusion matrix for an imbalanced target variable by incorporating priors
Using aggregate to write cluster centers to Excel for conditional formatting
Creating a classification tree financial summary using aggregate and an Excel Export node
Reformatting data for reporting with a Transpose node
Changing formatting of fields in a Table node
Combining generated filters
8
8. CLEM Scripting
8. CLEM Scripting
Introduction
Building iterative Neural Network forecasts
Quantifying variable importance with Monte Carlo simulation
Implementing champion/challenger model management
Detecting outliers with the jackknife method
Optimizing K-means cluster solutions
Automating time series forecasts
Automating HTML reports and graphs
Rolling your own modeling algorithm – Weibull analysis
9
A. Business Understanding
A. Business Understanding
Introduction
Define business objectives by Tom Khabaza
Assessing the situation by Meta Brown
Translating your business objective into a data mining objective by Dean Abbott
Produce a project plan – ensuring a realistic timeline by Keith McCormick
10
Index
Index

Using CHAID stumps when interviewing an SME

In this recipe we will learn how to use the interactive mode of the CHAID Modeling node to explore data. The name stump comes from the idea that we grow just one branch and stop. The exploration will have the goal of answering five questions:

  1. What variables seem predictive of the target?
  2. Do the most predictive variables make sense?
  3. What questions are most useful to pose to the Subject Matter Experts (SMEs) about data quality?
  4. What is the potential value of the favorite variables of the SMEs?
  5. What missing data challenges are present in the data?

Getting ready

We will start with a blank stream.

How to do it...

To use CHAID stumps:

  1. Add a Source node to the stream for the cup98lrn reduced vars2.txt file. Ensure that the field delimiter is Tab and that the Strip lead and trail spaces option is set to Both.
  2. Add a Type node and declare TARGET_B as flag and as the target. Set TARGET_D, RFA_2, RFA_2A, and RFA_2F, RFA_2R to None.
    How to do it...
  3. Add a CHAID Modeling node and make sure that it is in interactive mode.
    How to do it...
  4. Click on the Run button. From the menus choose Tree | Grow Branch with Custom Split. Then click on the Predictors button.
    How to do it...
  5. Allow the top variable, LASTGIFT, to form a branch. Note that LASTGIFT does not seem to have missing values.
    How to do it...
  6. Further down the list, the RAMNT_ series variables do have missing values. Placing the mouse on the root node (Node 0) choose Tree | Grow Branch with Custom Split again.
    How to do it...
  7. The figure shows RAMNT_8, but your results may differ somewhat as CHAIDtakes an internal partition and therefore does not use all of the data. The slight differences can change the ranking of similar variables. Allow the branch to grow on your selected variable.
    How to do it...
  8. Now we will break away the missing data into its own category. Repeat the steps leading up to this branch, but before clicking on the Grow button, select Custom and at the bottom, set Missing values into as Separate Node.
    How to do it...
  9. Sometimes SMEs will have a particular interest in a variable because it has been known to be valuable in the past, or they are invested in the variable in some way. Even though it is well down the list, choose the variable Wealth2 and force it to branch while ensuring that missing values are placed into a Separate node.
    How to do it...

How it works...

There are several advantages to exploring data in this way with CHAID. If you have accidentally included perfect predictors it will become obvious in a hurry. This recipe is dedicated to this phenomenon. Another advantage is that most SMEs find CHAID rather intuitive. It is easy to see what the relationships are without extensive exposure to the technique. Meanwhile, as an added benefit, the SMEs are becoming acquainted with a technique that might also be used during the modeling phase. As we have seen, CHAID can show missing data as a Separate node. This feature is shown to be useful in the Binning scale variables to address missing data recipe in Chapter 3, Data Preparation – Clean. By staying in interactive mode, the trees are kept simple; also, we can force any variable to branch even if it is not near the top of the list. Often SMEs can be quite adamant that a variable is important, while the data shows them otherwise. There are countless reasons why this might be the case, and the conversation should be allowed to unfold. One is likely to learn a great deal trying to figure out why a variable that seemed promising is not performing well in the CHAID model.

Let's examine the CHAID tree a bit more closely. The root node shows the total sample size and the percentage in each of the two categories. In the figures in this recipe, the red group is the donors group. Notice that the more recent their LASTGIFT was, the more likely that they donated. Starting with 8.286 percent for the less than or equal to 9 group, dropping down to 3.476 percent for the less than 19 group. Note that when you add up the child nodes, you get the same number as the number in the root node.

How it works...

It is recommended that you take a screenshot of at least the top 10 or so variables of interest to management or SMEs. It is a good precaution to place the images on slides, since you will be able to review and discuss without waiting for Modeler to process. Having said that, it is an excellent idea to be ready to further explore the data using this technique on live data during the meeting.

See also

  • The Using the Feature Selection node creatively to remove or decapitate perfect predictors recipe in Chapter 2, Data Preparation – Select
  • The Binning scale variables to address missing data recipe in Chapter 3, Data Preparation – Clean
End of Section 5
Next Section
bookmark search playlist font-size

Change the font size

margin-width

Change margin width

day-mode

Change background colour

Close icon Search
Country selected
Close icon Your notes and bookmarks

Delete Bookmark

Modal Close icon
Are you sure you want to delete it?
Cancel
Yes, Delete