SSIS:MatchUp:Algorithms - Revision history

Admin at 00:22, 14 November 2015

2015-11-14T00:22:28Z

@@ Line 1: / Line 1: @@
-[[SSIS:Reference|&larr; SSIS Reference]]
+{{SSISMatchUpNav
+|MatchcodeEditorCollapse=
-{| class="mw-collapsible" cellspacing="2" style="background-color:#f9f9f9; border:1px solid #aaaaaa; font-size:9pt; color:#0645ad; padding:8px 8px;"
+}}

Admin at 19:08, 20 May 2015

2015-05-20T19:08:16Z

← Older revision		Revision as of 19:08, 20 May 2015
Line 87:		Line 87:
	Determines whether two strings are identical.		Determines whether two strings are identical.

	===~~Jaro~~===		===Soundex===
	~~Gathers common~~ characters (~~in order~~) ~~between~~ the ~~two strings~~, ~~then counts transpositions between~~ the ~~two common strings~~.		SoundEx is a string transformation and comparison-based algorithm. For example, JOHNSON would be transformed to "J525" and JHNSN would also be transformed to "J525" which would then be considered a SoundExing match after evaluation.
			If the original strings are identical, SoundEx will return 100%. If the SoundEx'd strings are equal, the algorithm returns 99%. Otherwise, SoundEx will return 0%.

			===Phonetex===
			A variation of the SoundEx Algorithm. PhonetEx takes into account letter combinations that sound alike, particularly at the start of the word (such as 'PN' = 'N', 'PH' = 'F').

			===Containment===
			Matches when one record's component is contained in another record. For example, “Smith” is contained in “Smithfield.”

			===Frequency===
			Matches the characters in one record’s component to the characters in another without any regard to the sequence. For example “abcdef” would match ”badcfe.”

			===Fast Near===
			A typographical matching algorithm. It works best in matching words that don’t match because of a few typographical errors. Exactly how many errors is specified on a scale from 1(Tight) to 4(Loose). The Fast Near algorithm is a faster approximation of the Accurate Near algorithm described below. The tradeoff for speed is accuracy; sometimes Fast Near will find false matches or miss true matches.

			===Accurate Near===
			An implementation of the Levenshtein algorithm. It is a typographical matching algorithm. The Accurate Near algorithm produces better results than the Fast Near algorithm, but is slower.

			===Frequency Near===
			The Frequency algorithm will match the characters of one string to the characters of another without any regard to the sequence. For example "abcdef" would be considered a 100% match to "badcfe."

			===Vowels===
			Only vowels will be compared. Consonants will be removed.

			===Consonants===
			Only consonants will be compared. Vowels will be removed.

			===Alphabetic===
			Only alphabetic characters will be compared.

	===~~Jaro-Winkler~~===		===Numeric===
	~~A variation to the Jaro algorithm. Strings that have matching~~ characters ~~at the beginning~~ will be ~~accounted for~~ and are ~~given additional weight to similarity~~.		Only numeric characters will be compared. Decimals and signs are considered numeric.

	===N-Gram===		===N-Gram===
	Counts the number of common sub-strings (grams) ~~of a specified length~~ between the two strings.		Counts the number of common sub-strings (grams) between the two strings. Substring size ‘N’, is currently defaulted as 2 in MatchUp.

	===~~Dice's Coefficient~~===		===Jaro===
	~~A variation of~~ the ~~N-Gram algorithm. Dice's Coefficient~~ counts ~~matching n-Grams but does not count extra duplicate n-Grams~~.		Gathers common characters (in order) between the two strings, then counts transpositions between the two common strings.

	===~~Jaccard Similarity~~===		===Jaro-Winkler===
	~~A variation~~ of the ~~N-Gram algorithm. The Jaccard Similarity is identical~~ to ~~the N-Gram algorithm but uses a different formula for similarity computation~~.		Just like Jaro, but gives added weight for matching characters at the start of the string (up to 4 characters).

	===~~Overlap Coefficient~~===		===Longest Common Substring (LCS)===
	~~A variation of~~ the ~~N-Gram algorithm. The Overlap Coefficient is identical to~~ the ~~N-Gram algorithm but uses a different formula for similarity computation~~.		Finds the longest common substring between the two strings.

	===~~Levenshtein~~===		===Needleman-Wunch===
	~~The~~ Levenshtein algorithm ~~computes for the similarity of two strings by taking into account the amount of~~ character mistakes. ~~Mistakes are based off the number~~ of ~~incorrect characters, inserted characters~~, and ~~deleted characters~~.		A variation of the Levenshtein algorithm. Levenshtein and Needleman-Wunsch are identical except that character mistakes are given different weights depending on how far two characters are on a standard keyboard layout. For example: A to S is given a mistake weight of 0.4, while A to D is a 0.6 and A to P is a 1.0.

	===~~Needleman-Wunsch~~===		===MD Keyboard===
	~~A variation of the Levenshtein~~ algorithm~~. Levenshtein and Needleman~~-~~Wunsch are identical except that character mistakes are given different weights depending~~ on ~~how far two characters are on a standard keyboard layout. For example: A to S is given a mistake weight~~ of ~~0.4, while A to D is a 0.6~~ and ~~A to P is~~ a 1.0.		An algorithm developed by Melissa Data which counts keyboarding mis-hits with a weighted penalty based on the distance of the mis-hit and assigns a percentage of similarity between the compared strings.
			This effectively adds the "understanding" that the keyboarder may have typed in one character before another.

	===Smith-Waterman-Gotoh===		===Smith-Waterman-Gotoh===
Line 115:		Line 144:
	This effectively adds the "understanding" that the keyboarder may have tried to abbreviate one of the words.		This effectively adds the "understanding" that the keyboarder may have tried to abbreviate one of the words.

	===~~MDKeyboard~~===		===Dice’s Coefficient===
	A variation of the ~~Smith~~-~~Waterman-Gotoh~~ algorithm. ~~Smith~~-~~Waterman~~-~~Gotoh and MDKeyboard are identical except that character transpositions are given a different weight.~~		A variation of the N-Gram algorithm. Dice's Coefficient counts matching n-Grams but does not count extra duplicate n-Grams.
	~~This effectively adds the "understanding" that the keyboarder may have typed in one character before another~~.

	===~~Longest Common Substring (LCS)~~===		===Jaccard Similarity Coefficient===
	The ~~LCS~~ algorithm ~~counts~~ for ~~the longest common set of adjacent characters between 2 strings~~.		A variation of the N-Gram algorithm. The Jaccard Similarity is identical to the N-Gram algorithm but uses a different formula for similarity computation.

	===~~Containment~~===		===Overlap Coefficient===
	~~The Containment algorithm will return 100% if one string is a subset of another.~~ A ~~0% is returned otherwise.~~		A variation of the N-Gram algorithm. The Overlap Coefficient is identical to the N-Gram algorithm but uses a different formula for similarity computation.

	~~===Frequency===~~
	~~The Frequency algorithm will match the characters~~ of ~~one string to~~ the ~~characters of another without any regard to the sequence. For example "abcdef" would be considered a 100% match to "badcfe."~~

	~~===SoundEx===~~
	~~SoundEx is a string transformation and comparison~~-~~based~~ algorithm. ~~For example, JOHNSON would be transformed to "J525" and JHNSN would also be transformed~~ to ~~"J525" which would then be considered a SoundExing match after evaluation.~~
	~~If the original strings are identical, SoundEx will return 100%. If the SoundEx'd strings are equal,~~ the algorithm ~~returns 99%. Otherwise, SoundEx will return 0%.~~

	~~===PhonetEx===~~
	~~A variation of the SoundEx Algorithm. PhonetEx takes into account letter combinations that sound alike, particularly at the start of the word (such as 'PN' = 'N', 'PH' = 'F')~~.

	===Double ~~Metaphone~~===		===Double MetaPhone===
	A variation of the PhonetEx Algorithm. Double Metaphone performs 2 different PhonetEx-style transformations. It creates two PhonetEx-like strings (primary and alternate) for both strings.		A variation of the PhonetEx Algorithm. Double Metaphone performs 2 different PhonetEx-style transformations. It creates two PhonetEx-like strings (primary and alternate) for both strings.
	The logic used for Double Metaphone Similarity works as follows:		The logic used for Double Metaphone Similarity works as follows:

Admin at 22:09, 21 November 2014

2014-11-21T22:09:22Z

← Older revision		Revision as of 22:09, 21 November 2014
Line 28:		Line 28:
	\|-		\|-
	\|[[SSIS:MatchUp:Options\|Options]]		\|[[SSIS:MatchUp:Options\|Options]]
			\|-
			\|[[SSIS:MatchUp:Golden Record\|Golden Record]]
	\|-		\|-
	\|[[SSIS:MatchUp:Survivorship Pass-Through\|Survivorship/Pass-Through]]		\|[[SSIS:MatchUp:Survivorship Pass-Through\|Survivorship/Pass-Through]]

Admin: Created page with "← SSIS Reference {| class="mw-collapsible" cellspacing="2" style="background-color:#f9f9f9; border:1px solid #aaaaaa; font-size:9pt; color:#0645ad; pa..."

2014-11-18T01:55:45Z

Created page with "← SSIS Reference {| class="mw-collapsible" cellspacing="2" style="background-color:#f9f9f9; border:1px solid #aaaaaa; font-size:9pt; color:#0645ad; pa..."

New page

[[SSIS:Reference|← SSIS Reference]]

{| class="mw-collapsible" cellspacing="2" style="background-color:#f9f9f9; border:1px solid #aaaaaa; font-size:9pt; color:#0645ad; padding:8px 8px;"
! style="text-align:left; color:black; border-bottom:1px solid #aaaaaa;"|'''MatchUp Navigation'''
|-
| style="padding-right:220px;"|
|-
|[[SSIS:MatchUp|Overview]]
|-
|[[SSIS:MatchUp Tutorial|Tutorial]]
|-
|
{| class="mw-collapsible mw-collapsed" style="width:100%;"
|- align="left"
! style="color:black;"|[[SSIS:MatchUp:Advanced Configuration|Advanced Configuration]]
|-
|[[SSIS:MatchUp:Advanced Configuration#On-Premise|On-Premise]]
|}
|-
|
{| class="mw-collapsible mw-collapsed" style="width:100%;"
|- align="left"
! style="color:black;"|MatchUp Tabs
|-
|[[SSIS:MatchUp:Matchcode|Matchcode]]
|-
|[[SSIS:MatchUp:Field Mapping|Field Mapping]]
|-
|[[SSIS:MatchUp:Options|Options]]
|-
|[[SSIS:MatchUp:Survivorship Pass-Through|Survivorship/Pass-Through]]
|-
|[[SSIS:MatchUp:Lookup Pass-Through|Lookup Pass-Through Columns]]
|-
|[[SSIS:MatchUp:Output Filter|Output Filter]]
|}
|-
|
{| class="mw-collapsible" style="width:100%;"
|- align="left"
! style="color:black;"|[[SSIS:MatchUp:Matchcode Editor|Matchcode Editor]]
|-
|[[SSIS:MatchUp:Matchcode List|Matchcode List]]
|-
|[[SSIS:MatchUp:Component List|Component List]]
|-
|[[SSIS:MatchUp:Component Properties|Component Properties]]
|-
|[[SSIS:MatchUp:Algorithms|Algorithms]]
|}
|-
|
{| class="mw-collapsible mw-collapsed" style="width:100%;"
|- align="left"
! style="color:black;"|[[SSIS:MatchUp:Matchcodes:Overview|Matchcodes Overview]]
|-
|[[SSIS:MatchUp:Matchcodes:Component Combinations|Component Combinations]]
|-
|[[SSIS:MatchUp:Matchcodes:Blank Field Mapping|Blank Field Mapping]]
|-
|[[SSIS:MatchUp:Matchcodes:Mapping|Mapping]]
|-
|[[SSIS:MatchUp:Matchcodes:Optimization|Optimization]]
|-
|[[SSIS:MatchUp:Matchcodes:Swap Matching|Swap Matching]]
|}
|-
|
{| class="mw-collapsible mw-collapsed" style="width:100%;"
|- align="left"
! style="color:black;"|[[SSIS:MatchUp:Result Codes|Result Codes]]
|-
|[[Returned Result Codes:Enterprise Solutions#Data Quality Components for SSIS|Returned Result Codes]]
|-
|[[Result Codes]]
|}
|}

{{CustomTOC}}

The MatchUp Editor can use the following matching algorithms:

===Exact Matching===
Determines whether two strings are identical.

===Jaro===
Gathers common characters (in order) between the two strings, then counts transpositions between the two common strings.

===Jaro-Winkler===
A variation to the Jaro algorithm. Strings that have matching characters at the beginning will be accounted for and are given additional weight to similarity.

===N-Gram===
Counts the number of common sub-strings (grams) of a specified length between the two strings.

===Dice's Coefficient===
A variation of the N-Gram algorithm. Dice's Coefficient counts matching n-Grams but does not count extra duplicate n-Grams.

===Jaccard Similarity===
A variation of the N-Gram algorithm. The Jaccard Similarity is identical to the N-Gram algorithm but uses a different formula for similarity computation.

===Overlap Coefficient===
A variation of the N-Gram algorithm. The Overlap Coefficient is identical to the N-Gram algorithm but uses a different formula for similarity computation.

===Levenshtein===
The Levenshtein algorithm computes for the similarity of two strings by taking into account the amount of character mistakes. Mistakes are based off the number of incorrect characters, inserted characters, and deleted characters.

===Needleman-Wunsch===
A variation of the Levenshtein algorithm. Levenshtein and Needleman-Wunsch are identical except that character mistakes are given different weights depending on how far two characters are on a standard keyboard layout. For example: A to S is given a mistake weight of 0.4, while A to D is a 0.6 and A to P is a 1.0.

===Smith-Waterman-Gotoh===
A variation of the Needleman-Wunsch algorithm. Needleman-Wunsch and Smith-Waterman-Gotoh are identical except that character deletions are given a different weight.
This effectively adds the "understanding" that the keyboarder may have tried to abbreviate one of the words.

===MDKeyboard===
A variation of the Smith-Waterman-Gotoh algorithm. Smith-Waterman-Gotoh and MDKeyboard are identical except that character transpositions are given a different weight.
This effectively adds the "understanding" that the keyboarder may have typed in one character before another.

===Longest Common Substring (LCS)===
The LCS algorithm counts for the longest common set of adjacent characters between 2 strings.

===Containment===
The Containment algorithm will return 100% if one string is a subset of another. A 0% is returned otherwise.

===Frequency===
The Frequency algorithm will match the characters of one string to the characters of another without any regard to the sequence. For example "abcdef" would be considered a 100% match to "badcfe."

===SoundEx===
SoundEx is a string transformation and comparison-based algorithm. For example, JOHNSON would be transformed to "J525" and JHNSN would also be transformed to "J525" which would then be considered a SoundExing match after evaluation.
If the original strings are identical, SoundEx will return 100%. If the SoundEx'd strings are equal, the algorithm returns 99%. Otherwise, SoundEx will return 0%.

===PhonetEx===
A variation of the SoundEx Algorithm. PhonetEx takes into account letter combinations that sound alike, particularly at the start of the word (such as 'PN' = 'N', 'PH' = 'F').

===Double Metaphone===
A variation of the PhonetEx Algorithm. Double Metaphone performs 2 different PhonetEx-style transformations. It creates two PhonetEx-like strings (primary and alternate) for both strings.
The logic used for Double Metaphone Similarity works as follows:
*If primary1 = primary2 and alternate1 = alternate 2, then we have a very good match (99%).
*If either primary1 = alternate2 or alternate1 = primary2, and alternate1=alternate2, then we have a good match (85%).
*If alternate 1 = alternate2, we have an acceptable match (75%).

[[Category:SSIS]]
[[Category:SSIS MatchUp]]