MatchUp Object:Incremental:Order of Operations
|MatchUp Object Incremental Interface Navigation|
|Order of Operations|
These are the basic steps of a typical implementation of the Incremental interface.
- Initialize the Incremental interface.
- After creating an instance of the Incremental interface, point the object toward its supporting data file, select a matchcode and key file to use, and initialize these files.
- Create field mappings.
- In order to build a key to compare to the key file, the Incremental interface needs to know which types of data the program will be passing to the interface and in what order.
- Read the record from the data source.
- This can be a new address passed from a website, a single record from a newly acquired list or data source, to be compared against the master list.
- Build a match key for the incoming record.
- This consists of passing the actual data to the interface in the same order used when creating a field mapping. After passing the necessary fields (usually a small subset of the fields from each record) via the AddField function, the Incremental interface uses this information to generate a match key.
- Compare the match key to the key file.
- The MatchRecord function searches the key file for any keys that match the new record. If it finds a match, it provides information on the duplicate records in the key file.
- Write new records to the key file.
- The new key, whether or not it is unique, can then be written to the key file, so it can be used for future deduping operations. The program code must also write the new address record to the database separately.
This is a common implementation of the Incremental interface using pseudocode for maximum clarity. Working sample programs in several programming languages can be found on the MatchUp Object install disc.
Initialize the Incremental interface
After creating an instance of the Incremental interface, point the object toward its supporting data file, select a matchcode and key file to use, and initialize these files.
First, create a new instance of the Incremental interface.
SET mu = NEW mdMUIncremental
In order to successfully initialize this new instance, point it toward its data files and supply a valid License Key.
CALL mu.SetLicenseString with LicenseString CALL mu.SetPathToMatchUpFiles with PathToMatchUpFiles
Before initialization, specify which matchcode and key file will be used for the current deduping operation.
CALL mu.SetMatchcodeName with MatchCodeName CALL mu.SetKeyFile with PathToKeyFile
If all of the above have been set correctly, calling the InitializeDataFiles function should return a value of NoError. If it does not, call the GetInitializeErrorString function to determine the reason for the failure to initialize.
CALL mu.InitializeDataFiles RETURNING initResult IF initResult is not NoError THEN PRINT "Initialization Error: " + mu.GetInitializeErrorString EXIT PROGRAM END IF
If the initialization was successful, call the following functions to display version and expiration information about the instance of MatchUp Object currently in use on the local computer.
PRINT "Confirming Initialization: " + mu.GetInitializeErrorString PRINT "Build Number: " + mu.GetBuildNumber PRINT "Database Date: " + mu.GetDatabaseDate PRINT "Database Expiration Date: " + mu.GetDatabaseExpirationDate PRINT "License Expiration Date: " + mu.GetLicenseExpirationDate
Create field mappings
Field mappings define a valid incoming type of data for each matchcode component. For example, a typical matchcode may include a five-digit ZIP Code, a last name, and a street address. But the data coming in, however, may contain the city, state, and ZIP as a single character field and the person’s full name as a single field as well.
Even if the fields in a database do not exactly match the components required by the matchcode, MatchUp Object is able to extract only the information it needs.
After clearing any mappings from a previous use of the Incremental interface, call the AddMapping function once for each field being considered.
CALL mu.AddMapping with mu.CityStZip RETURNING mapOK CALL mu.AddMapping with mu.FullName RETURNING mapOK CALL mu.AddMapping with mu.Address RETURNING mapOK
Get the record from the data source
Regardless of the source, the object only needs to read the fields containing the data that the Incremental interface needs for comparison.
READ Record FROM database RETURNING userInfo, CityStateZip, TheFullName, StreetAddress
The userInfo field is any identifying character string that is unique to the current record.
Note that the Incremental interface does not handle database input and output. This must be done programmatically using whatever database interface is being used.
Build the match key for the current record
Now we use the data from the previous step to construct a match key to use in the next step.
After clearing any data from a previous use of the Incremental interface, call the AddField function once for each field being considered.
CALL mu.AddField with CityStateZip CALL mu.AddField with TheFullName CALL mu.AddField with StreetAddress
Pass the userInfo to the interface using the SetUserInfo function.
CALL mu.SetUserInfo with userInfo
Compare the match key to the key file
The MatchRecord function compares the match key to the key file and determines if the key already exists in the key file.
CALL mu.GetResults RETURNING ResultsCodes
If the record is a duplicate, this code retrieves information about the other duplicate records in the database.
IF ResultsCodes contains “MS03” THEN // Record is a duplicate PRINT "This record is in the database." PRINT "There are " + mu.GetCount + " records in dupe group #" + mu.GetDupeGroup PRINT "It matches these records:" WHILE mu.NextMatchingRecord PRINT "#" + mu.GetEntry + " is Record: " + mu.GetUserInfo ENDWHILE “Dupe Group” is a number that gets assigned to each unique record. Duplicate records are assigned the same number. ===Add the new key to the key file=== In this example, duplicate records are being rejected while unique records are being added to the database. Depending on the end user needs, the program may handle duplicate records differently. <pre> ELSE CALL mu.AddRecord Add New Record to master database ENDIF
The AddRecord function only adds the new key to the key file. To add the data to the database, the developer would need to implement that in code, according to whatever database engine is in use.
Using the Transaction Functions
The transaction functions allow MatchUp Object to delay writing changes to the historical database until all records have been compared and all duplicates detected. After a call to the BeginTransaction function, the Incremental interface will cache all calls to the AddRecord function until a call to the CommitTransaction function, which writes all changes to the keyfile in a single operation, significantly speeding up processing.
If any errors are detected, the RollbackTransaction function flushes all AddRecord function calls since the call to the BeginTransaction function and no changes will be written to the historical database.
Below is a simplified example of how transactions work with the Incremental interface.
CALL BeginTransaction FOR EACH Record in DatabaseTable READ Record Build Match Key CALL MatchRecord IF Record Is Unique THEN CALL AddRecord ENDIF NEXT Record IF ERROR CALL RollbackTransaction Return ENDIF CALL CommitTransaction