Matchcode Optimization:Smith-Waterman-Gotoh: Difference between revisions
Jump to navigation
Jump to search
Created page with "{{MatchcodeOptimizationNav |AlgorithmsCollapse= }} ==Soundex== ===Specifics=== *https://en.wikipedia.org/wiki/Smith%E2%80%93Waterman_algorithm ===Summary=== A typographical ..." |
No edit summary |
||
| Line 5: | Line 5: | ||
==Soundex== | ==Soundex== | ||
===Specifics=== | ===Specifics=== | ||
*https://en.wikipedia.org/wiki/Smith%E2%80%93Waterman_algorithm | :*https://en.wikipedia.org/wiki/Smith%E2%80%93Waterman_algorithm | ||
===Summary=== | ===Summary=== | ||
A typographical matching algorithm. A variation of the Needleman-Wunch, where it gives a non-linear penalty for inserts and deletes, but less of a penalty for linear inserts and deletes. For example, JNSON has 0.8+0.8 = 1.6 penalty in Needleman-Wunch, but only a 0.8+0.6 = 1.4 penalty in Smith-Waterman-Gotoh. This effectively adds the ‘understanding’ that the keyboarder may have tried to abbreviate one of the words. Because the algorithm creates a 2D array to determine the distance between two strings, results will be more accurate than Fast Near at expense of throughput. | :A typographical matching algorithm. A variation of the Needleman-Wunch, where it gives a non-linear penalty for inserts and deletes, but less of a penalty for linear inserts and deletes. For example, JNSON has 0.8+0.8 = 1.6 penalty in Needleman-Wunch, but only a 0.8+0.6 = 1.4 penalty in Smith-Waterman-Gotoh. This effectively adds the ‘understanding’ that the keyboarder may have tried to abbreviate one of the words. Because the algorithm creates a 2D array to determine the distance between two strings, results will be more accurate than Fast Near at expense of throughput. | ||
===Returns=== | ===Returns=== | ||
Boolean ‘match’ if the normalized distance between two strings is less than the configured scale, where distance is defined as the count of the number of incorrect characters, insertions and deletions. | :Boolean ‘match’ if the normalized distance between two strings is less than the configured scale, where distance is defined as the count of the number of incorrect characters, insertions and deletions. | ||
===Example Matchcode Component=== | ===Example Matchcode Component=== | ||
| Line 19: | Line 19: | ||
{{ExampleDataTableV1|STRING1|STRING2|RESULT | {{ExampleDataTableV1|STRING1|STRING2|RESULT | ||
|AdditionalRows= | |AdditionalRows= | ||
{{EDTRow| | {{EDTRow|Green|Johnson|Jhnsn|Match Found}} | ||
{{EDTRow| | {{EDTRow|Green|Mild Hatr Wks|Mild Hatter Works|Match Found}} | ||
{{EDTRow| | {{EDTRow|White|Deanardo|Dinardio|Unique}} | ||
{{EDTRow|Green|Apco Quik Lube|Apco Quik Oil N Lube|Match Found}} | {{EDTRow|Green|Apco Quik Lube|Apco Quik Oil N Lube|Match Found}} | ||
}} | }} | ||
| Line 34: | Line 34: | ||
===Recommended Usage=== | ===Recommended Usage=== | ||
Hybrid or real-time deduping. This works best in matching where the data entry has a proprietary abbreviation rule set. | :Hybrid or real-time deduping. This works best in matching where the data entry has a proprietary abbreviation rule set. | ||
===Not Recommended For=== | ===Not Recommended For=== | ||
Enterprise size processes. | :Enterprise size processes. | ||
Gather/scatter, survivorship, or record consolidation of sensitive data. | :Gather/scatter, survivorship, or record consolidation of sensitive data. | ||
Quantifiable data or records with proprietary keywords not associated in our knowledgebase tables. | :Quantifiable data or records with proprietary keywords not associated in our knowledgebase tables. | ||
===Do Not Use With=== | ===Do Not Use With=== | ||
UTF-8 data. This algorithm was ported to MatchUp with the assumption that a character equals one byte, and therefore results may not be accurate if the data contains multi-byte characters. | :UTF-8 data. This algorithm was ported to MatchUp with the assumption that a character equals one byte, and therefore results may not be accurate if the data contains multi-byte characters. | ||
[[Category:MatchUp Hub]] | [[Category:MatchUp Hub]] | ||
[[Category:Matchcode Optimization]] | [[Category:Matchcode Optimization]] | ||
Latest revision as of 14:30, 27 September 2018
Soundex
Specifics
Summary
- A typographical matching algorithm. A variation of the Needleman-Wunch, where it gives a non-linear penalty for inserts and deletes, but less of a penalty for linear inserts and deletes. For example, JNSON has 0.8+0.8 = 1.6 penalty in Needleman-Wunch, but only a 0.8+0.6 = 1.4 penalty in Smith-Waterman-Gotoh. This effectively adds the ‘understanding’ that the keyboarder may have tried to abbreviate one of the words. Because the algorithm creates a 2D array to determine the distance between two strings, results will be more accurate than Fast Near at expense of throughput.
Returns
- Boolean ‘match’ if the normalized distance between two strings is less than the configured scale, where distance is defined as the count of the number of incorrect characters, insertions and deletions.
Example Matchcode Component
Example Data
STRING1 STRING2 RESULT Johnson Jhnsn Match Found Mild Hatr Wks Mild Hatter Works Match Found Deanardo Dinardio Unique Apco Quik Lube Apco Quik Oil N Lube Match Found
| Performance | |||||
|---|---|---|---|---|---|
| Slower | Faster | ||||
| Matches | |||||
| More Matches | Greater Accuracy | ||||
Recommended Usage
- Hybrid or real-time deduping. This works best in matching where the data entry has a proprietary abbreviation rule set.
Not Recommended For
- Enterprise size processes.
- Gather/scatter, survivorship, or record consolidation of sensitive data.
- Quantifiable data or records with proprietary keywords not associated in our knowledgebase tables.
Do Not Use With
- UTF-8 data. This algorithm was ported to MatchUp with the assumption that a character equals one byte, and therefore results may not be accurate if the data contains multi-byte characters.