MatchUp Object:Incremental:Order of Operations

← MatchUp Object Reference

MatchUp Object Incremental Interface Navigation
Overview
Order of Operations
Functions Initialization Mapping Match Key Comparison Key File Transaction

Basic Steps

These are the basic steps of a typical implementation of the Incremental interface.

1. Initialize the Incremental interface.

   After creating an instance of the Incremental interface, point the object toward its supporting data file, select a matchcode and key file to use, and initialize these files.

2. Create field mappings.

   In order to build a key to compare to the key file, the Incremental interface needs to know which types of data the program will be passing to the interface and in what order.

3. Read the record from the data source.

   This can be a new address passed from a website, a single record from a newly acquired list or data source, to be compared against the master list.

4. Build a match key for the incoming record.

   This consists of passing the actual data to the interface in the same order used when creating a field mapping. After passing the necessary fields (usually a small subset of the fields from each record) via the AddField function, the Incremental interface uses this information to generate a match key.

5. Compare the match key to the key file.

   The MatchRecord function searches the key file for any keys that match the new record. If it finds a match, it provides information on the duplicate records in the key file.

6. Write new records to the key file.

   The new key, whether or not it is unique, can then be written to the key file, so it can be used for future deduping operations. The program code must also write the new address record to the database separately.

Pseudocode Implementation

This is a common implementation of the Incremental interface using pseudocode for maximum clarity. Working sample programs in several programming languages can be found on the MatchUp Object install disc.

Initialize the Incremental interface

After creating an instance of the Incremental interface, point the object toward its supporting data file, select a matchcode and key file to use, and initialize these files.

First, create a new instance of the Incremental interface.

SET mu = NEW mdMUIncremental

In order to successfully initialize this new instance, point it toward its data files and supply a valid license string.

CALL mu.SetLicenseString with LicenseString
CALL mu.SetPathToMatchUpFiles with PathToMatchUpFiles

Before initialization, specify which matchcode and key file will be used for the current deduping operation.

CALL mu.SetMatchcodeName with MatchCodeName
CALL mu.SetKeyFile with PathToKeyFile

If all of the above have been set correctly, calling the InitializeDataFiles function should return a value of NoError. If it does not, call the GetInitializeErrorString function to determine the reason for the failure to initialize.

CALL mu.InitializeDataFiles RETURNING initResult
IF initResult is not NoError THEN
  PRINT "Initialization Error: " + mu.GetInitializeErrorString
  EXIT PROGRAM
END IF

If the initialization was successful, call the following functions to display version and expiration information about the instance of MatchUp Object currently in use on the local computer.

PRINT "Confirming Initialization: " + mu.GetInitializeErrorString
PRINT "Build Number: " + mu.GetBuildNumber
PRINT "Database Date: " + mu.GetDatabaseDate
PRINT "Database Expiration Date: " + mu.GetDatabaseExpirationDate
PRINT "License Expiration Date: " + mu.GetLicenseExpirationDate

Create field mappings

Field mappings define a valid incoming type of data for each matchcode component. For example, a typical matchcode may include a five-digit ZIP Code, a last name, and a street address. But the data coming in, however, may contain the city, state, and ZIP as a single character field and the person’s full name as a single field as well.

Even if the fields in a database do not exactly match the components required by the matchcode, MatchUp Object is able to extract only the information it needs.

CALL mu.ClearMappings

After clearing any mappings from a previous use of the Incremental interface, call the AddMapping function once for each field being considered.

CALL mu.AddMapping with mu.CityStZip RETURNING mapOK
CALL mu.AddMapping with mu.FullName RETURNING mapOK
CALL mu.AddMapping with mu.Address RETURNING mapOK

Get the record from the data source

Regardless of the source, the object only needs to read the fields containing the data that the Incremental interface needs for comparison.

READ Record FROM database RETURNING userInfo, CityStateZip, TheFullName, StreetAddress

The userInfo field is any identifying character string that is unique to the current record.

Note that the Incremental interface does not handle database input and output. This must be done programmatically using whatever database interface is being used.

Build the match key for the current record

Now we use the data from the previous step to construct a match key to use in the next step.

CALL mu.ClearFields

After clearing any data from a previous use of the Incremental interface, call the AddField function once for each field being considered.

CALL mu.AddField with CityStateZip
CALL mu.AddField with TheFullName
CALL mu.AddField with StreetAddress

Pass the userInfo to the interface using the SetUserInfo function.

CALL mu.SetUserInfo with userInfo

The BuildKey function constructs the match key out of the information passed via the AddField function calls.

CALL mu.BuildKey

Compare the match key to the key file

The MatchRecord function compares the match key to the key file and determines if the key already exists in the key file.

CALL mu.MatchRecord

Check the GetResults function to determine if the MatchRecord call produced a match, meaning that the current record is a duplicate to another one in the key file.

CALL mu.GetResults RETURNING ResultsCodes

If the record is a duplicate, this code retrieves information about the other duplicate records in the database.

IF ResultsCodes contains “MS03” THEN // Record is a duplicate
  PRINT "This record is in the database."
  PRINT "There are " + mu.GetCount + " records in dupe group #" + mu.GetDupeGroup
  PRINT "It matches these records:"
WHILE mu.NextMatchingRecord
  PRINT "#" + mu.GetEntry + " is Record: " + mu.GetUserInfo
ENDWHILE

“Dupe Group” is a number that gets assigned to each unique record. Duplicate records are assigned the same number.

===Add the new key to the key file===
In this example, duplicate records are being rejected while unique records are being added to the database. Depending on the end user needs, the program may handle duplicate records differently.

<pre>
ELSE
  CALL mu.AddRecord
  Add New Record to master database
ENDIF

The AddRecord function only adds the new key to the key file. To add the data to the database, the developer would need to implement that in code, according to whatever database engine is in use.

Using the Transaction Functions

The transaction functions allow MatchUp Object to delay writing changes to the historical database until all records have been compared and all duplicates detected. After a call to the BeginTransaction function, the Incremental interface will cache all calls to the AddRecord function until a call to the CommitTransaction function, which writes all changes to the keyfile in a single operation, significantly speeding up processing.

If any errors are detected, the RollbackTransaction function flushes all AddRecord function calls since the call to the BeginTransaction function and no changes will be written to the historical database.

Below is a simplified example of how transactions work with the Incremental interface.

CALL BeginTransaction
FOR EACH Record in DatabaseTable
  READ Record
  Build Match Key
  CALL MatchRecord
  IF Record Is Unique THEN
    CALL AddRecord
  ENDIF
NEXT Record
IF ERROR
  CALL RollbackTransaction
  Return
ENDIF
CALL CommitTransaction