Browse Library
Sign In Start Free Trial
Close icon Search icon
Account
Browse Library Advanced Search Sign In Start Free Trial

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • IBM SPSS Modeler Cookbook
  • Toc
  • feedback
IBM SPSS Modeler Cookbook

IBM SPSS Modeler Cookbook

By : Keith McCormick, Abbott
4.4 (20)
close
IBM SPSS Modeler Cookbook

IBM SPSS Modeler Cookbook

4.4 (20)
By: Keith McCormick, Abbott

Overview of this book

IBM SPSS Modeler is a data mining workbench that enables you to explore data, identify important relationships that you can leverage, and build predictive models quickly allowing your organization to base its decisions on hard data not hunches or guesswork. IBM SPSS Modeler Cookbook takes you beyond the basics and shares the tips, the timesavers, and the workarounds that experts use to increase productivity and extract maximum value from data. The authors of this book are among the very best of these exponents, gurus who, in their brilliant and imaginative use of the tool, have pushed back the boundaries of applied analytics. By reading this book, you are learning from practitioners who have helped define the state of the art. Follow the industry standard data mining process, gaining new skills at each stage, from loading data to integrating results into everyday business practices. Get a handle on the most efficient ways of extracting data from your own sources, preparing it for exploration and modeling. Master the best methods for building models that will perform well in the workplace. Go beyond the basics and get the full power of your data mining workbench with this practical guide.
Table of Contents (11 chapters)
close
close
Preface
chevron up
Preface
What is CRISP-DM?
Data mining is a business process
The IBM SPSS Modeler workbench
Historical introduction to scripting
What this book covers
Who this book is for
Conventions
Reader feedback
Customer support
Free Chapter
1
1. Data Understanding
1. Data Understanding
Introduction
Using an empty aggregate to evaluate sample size
Evaluating the need to sample from the initial data
Using CHAID stumps when interviewing an SME
Using a single cluster K-means as an alternative to anomaly detection
Using an @NULL multiple Derive to explore missing data
Creating an Outlier report to give to SMEs
Detecting potential model instability early using the Partition node and Feature Selection node
2
2. Data Preparation – Select
2. Data Preparation – Select
Introduction
Using the Feature Selection node creatively to remove or decapitate perfect predictors
Running a Statistics node on anti-join to evaluate the potential missing data
Evaluating the use of sampling for speed
Removing redundant variables using correlation matrices
Selecting variables using the CHAID Modeling node
Selecting variables using the Means node
Selecting variables using single-antecedent Association Rules
3
3. Data Preparation – Clean
3. Data Preparation – Clean
Introduction
Binning scale variables to address missing data
Using a full data model/partial data model approach to address missing data
Imputing in-stream mean or median
Imputing missing values randomly from uniform or normal distributions
Using random imputation to match a variable's distribution
Searching for similar records using a Neural Network for inexact matching
Using neuro-fuzzy searching to find similar names
Producing longer Soundex codes
4
4. Data Preparation – Construct
4. Data Preparation – Construct
Introduction
Building transformations with multiple Derive nodes
Calculating and comparing conversion rates
Grouping categorical values
Transforming high skew and kurtosis variables with a multiple Derive node
Creating flag variables for aggregation
Using Association Rules for interaction detection/feature creation
Creating time-aligned cohorts
5
5. Data Preparation – Integrate and Format
5. Data Preparation – Integrate and Format
Introduction
Speeding up merge with caching and optimization settings
Merging a lookup table
Shuffle-down (nonstandard aggregation)
Cartesian product merge using key-less merge by key
Multiplying out using Cartesian product merge, user source, and derive dummy
Changing large numbers of variable names without scripting
Parsing nonstandard dates
Parsing and performing a conversion on a complex stream
Sequence processing
6
6. Selecting and Building a Model
6. Selecting and Building a Model
Introduction
Evaluating balancing with Auto Classifier
Building models with and without outliers
Using Neural Network for Feature Selection
Creating a bootstrap sample
Creating bagged logistic regression models
Using KNN to match similar cases
Using Auto Classifier to tune models
Next-Best-Offer for large datasets
7
7. Modeling – Assessment, Evaluation, Deployment, and Monitoring
7. Modeling – Assessment, Evaluation, Deployment, and Monitoring
Introduction
How (and why) to validate as well as test
Using classification trees to explore the predictions of a Neural Network
Correcting a confusion matrix for an imbalanced target variable by incorporating priors
Using aggregate to write cluster centers to Excel for conditional formatting
Creating a classification tree financial summary using aggregate and an Excel Export node
Reformatting data for reporting with a Transpose node
Changing formatting of fields in a Table node
Combining generated filters
8
8. CLEM Scripting
8. CLEM Scripting
Introduction
Building iterative Neural Network forecasts
Quantifying variable importance with Monte Carlo simulation
Implementing champion/challenger model management
Detecting outliers with the jackknife method
Optimizing K-means cluster solutions
Automating time series forecasts
Automating HTML reports and graphs
Rolling your own modeling algorithm – Weibull analysis
9
A. Business Understanding
A. Business Understanding
Introduction
Define business objectives by Tom Khabaza
Assessing the situation by Meta Brown
Translating your business objective into a data mining objective by Dean Abbott
Produce a project plan – ensuring a realistic timeline by Keith McCormick
10
Index
Index

Historical introduction to scripting

By the time Clementine Version 4 was released in 1997, the workbench had gained substantial market traction. Its revolutionary visual programming interface had enabled a more business-focused approach to analytics than ever before—all the major families of algorithms were represented in an easy-to-use form, ODBC had enabled integration with a comprehensive range of data, and commercial partners were busy rebadging Clementine to reach a wider audience through new market channels.

The workbench lacked one major kind of functionality, that of automation, to enable the embedding of data mining within other applications. It was therefore decided that automation would form the centre piece of Version 5, and it would be provided by two major features: batch mode and scripting. Batch mode enabled running the workbench without the user interface so that streams could be run in the background, could be scheduled to run at a given time or at regular intervals, and could be run as part of a larger application. Scripting enabled the user to gain automated control of stream execution, even without the user being present; this was also a prerequisite for any complex operation executed in batch mode.

The motivation behind scripting was to provide a number of capabilities:

  • Gain control of the order of stream execution where this matters, that is, when using the Set Globals node
  • Automate repetitive processes, for example, cross-validation or the exploration of many different sets of fields or options
  • Remove the need for user intervention so that streams could run in the background
  • Manipulate complex streams, for example, if the need arose to create 1000 different Derive nodes

These motives led to an underlying philosophy of scripting, that is, scripts replace the user, not the stream. This means that the operations of scripting should be at the same level as the actions of the user, that is, they would create nodes and link them, control their settings, execute streams, and save streams and models. Scripts would not be used to implement data manipulation or algorithms directly; these would remain in the domain of the stream itself. This reflects a fundamental fact about technologies—they are defined by what they cannot do as by what they can. These principles are not inflexible, for example, cross-validation might be considered as part of an algorithm but was one of the first scripts to be written; however, they guided the design of the scripting language. A consequence of this philosophy was that there could be no interaction between script and data; the restriction was lifted only later with the introduction of access to output objects.

A number of factors influenced the design of the scripting language in addition to the above philosophy:

  • In line with the orientation towards nontechnical users, the language should be simple
  • The timescale for implementation was short, so the language should be easy to implement
  • The language should be familiar, and so should use existing programming concepts and constructs, and not attempt to introduce new ones

These philosophical and practical constraints led to a programming language influenced by BASIC, with structured features taken from POP-11 and an object-oriented approach to nodes taken from Smalltalk and its descendants.

Next Section
bookmark search playlist font-size

Change the font size

margin-width

Change margin width

day-mode

Change background colour

Close icon Search
Country selected
Close icon Your notes and bookmarks

Delete Bookmark

Modal Close icon
Are you sure you want to delete it?
Cancel
Yes, Delete