FAQ:SSIS:Service Speed
Revision as of 00:00, 25 March 2022 by Admin (talk | contribs) (Created page with "← SSIS:Data Quality Components {{CustomTOC}} Melissa’s SSIS Enterprise level components can process hundreds of thousands of records p...")
← SSIS:Data Quality Components
Melissa’s SSIS Enterprise level components can process hundreds of thousands of records per hour. SSIS is available in 2 processing modes, Cloud and On Premise, which will affect throughput speeds.
Major Factors
Major factors impacting SSIS processing speed include:
- Number of threads or processes
- SSIS Cloud Processing components can handle multiple requests, increasing throughput speeds.
- We recommend up to 10 threads for our web services; if you require significantly more speed, contact your sales representative to discuss available options.
- We recommend creating multiple instances of our on-premise components to increase speed, utilizing a Balanced Data Distributor component to send data to each instance.
- Type of input method provided
- Using Melissa Address Key (MAK) as input will process faster since MAK is a globally unique identifier
- Using parsed data elements (such as AddressLine1, City, and State) instead of unstructured data such as Free Form inputs will result in faster processing speeds.
- Quality of input data
- Data of poor quality that requires additional input, corrections and standardization will reduce processing speed
- Number of output columns
- Requesting additional output columns will reduce processing speeds as more data is requested from our services
- Premium Mode Processing
- Phone and Email services offer Premium mode processing, which will reduce processing speeds due to performing live requests that provide higher accuracy data
- Type of SSIS input source utilized
- SSIS can use a variety of input sources available via SSIS Toolbox. We recommend OLE DB or Flat file source for faster processing
- Data Flow Buffer Size - DefaultBufferMaxRows and DefaultBufferSize Properties
- The buffer size controls the amount of rows transferred by the data flow. Specifications for buffer size vary due to environment and input source speed. Typically, a slow source input speed will require a smaller buffer size
- Adding Options and Actions
- Including additional options or actions from our services will reduce processing speeds as additional data is requested or additional logic is utilized
- Utilizing SQL Server Agent/Command Prompt to run projects
- Processing speed can increase by utilizing SQL Server Agent or by running the project via Command Prompt. Process quicker without the overhead of Visual Studio
On Premise Components
Speeds will vary due to hardware differences; higher speeds may be achieved with multiple instances of on-premise components.
| SSIS Component | Throughput (records / hour) |
|---|---|
| Contact Verify – Name, Address, Phone, Geo, Email (Express) | 500,000 – 600,000 |
| Generalized Cleanser | 9,000,000 – 12,000,000 |
| MatchUp (Basic Match Code) | 3,000,000 – 5,000,000 |
| Profiler | 800,000 – 1,250,000 |
Cloud Processing Components
| SSIS Component | Throughput (records / hour) |
|---|---|
| Contact Verify – Name, Address, Phone, Geo, Email (Express) | 90,000 – 160,000 |
| Contact Verify – Name, Address, Phone, Geo, Email (Premium) | 50,000 – 70,000 |
| Personator (Check, Verify) | 200,000 – 220,000 |
| Personator (Check, Verify, Move) | 170,000 – 190,000 |
| Personator (Check, Verify, Move, Append – Always) | 140,000 – 160,000 |
| SmartMover NCOA | 1,500,000 – 2,000,000 |
| SmartMover CCOA | 450,000 – 490,000 |
| BusinessCoder | 220,000 – 270,000 |
| Property | 200,000 – 260,000 |
| Global Verify – Name, Address, Phone (Express), Email (Express) | 110,000 – 150,000 |
| Global Verify – Name, Address, Phone (Premium), Email (Premium) | 60,000 – 85,000 |
| IPLocator | 5,000,000 – 6,000,000 |
| Personator World – Check – Name, Address, Phone (Express), Email (Express) | 80,000 – 100,000 |