SSIS:Best Practices: Difference between revisions
No edit summary |
No edit summary |
||
| Line 18: | Line 18: | ||
BP_MUXX_002 <br> | BP_MUXX_002 <br> | ||
Applying certain optimizations, or configuring the matchcode settings, can greatly reduce processing time without sacrificing accuracy. For a complete discussion see the Matchcodes > Optimization section of the MatchUp SSIS WebHelp or <br> | Applying certain optimizations, or configuring the matchcode settings, can greatly reduce processing time without sacrificing accuracy. For a complete discussion see the Matchcodes > Optimization section of the MatchUp SSIS WebHelp or <br> | ||
[[Best Practices | [[MatchUp Object:Best Practices#Optimizing Speed: Matchcodes|Optimizing MatchUp Matchcodes]] | ||
| Line 24: | Line 24: | ||
BP_MUXX_004 <br> | BP_MUXX_004 <br> | ||
Although the component's interface lets you change the location of the MatchUp work files, you should only override this setting after thorough testing for operating systems and speed. | Although the component's interface lets you change the location of the MatchUp work files, you should only override this setting after thorough testing for operating systems and speed. | ||
[[Best Practices | [[MatchUp Object:Best Practices#Keep Work File Location Local|general info here]] | ||
==MatchUp: Back up your Matchcode database== | ==MatchUp: Back up your Matchcode database== | ||
BP_MUXX_005 <br> | BP_MUXX_005 <br> | ||
[[Best Practices | [[MatchUp Object:Best Practices#Back up your Matchcode database|general info here]] | ||
Revision as of 16:44, 23 May 2014
Running SSIS packages from the Command Line
BP_XXSS_001
For more efficient memory usage, run your saved SSIS package from the command line
instead of running directly from visual studio. Example:
dtexec.exe /F "c:\fullpath\Package.dtsx" /Rep EWPD > "OptionalCreateLog.log"
tests have shown memory usage to be up to 66% less. Running from visual studio
will start devenv and DtsDebugHost processes in your Task Manager, whereas
command line processing will start DTEXEC.
Update to the latest version.
BP_XXSS_002
We always recommend users use the most current build (version), which may contain new or improved features, code fixes, or updated functionality. Melissa data provides many ways to ensure you keep your components to the latest version.
Update details here
MatchUp: Optimize Your Matchcode
BP_MUXX_002
Applying certain optimizations, or configuring the matchcode settings, can greatly reduce processing time without sacrificing accuracy. For a complete discussion see the Matchcodes > Optimization section of the MatchUp SSIS WebHelp or
Optimizing MatchUp Matchcodes
MatchUp: Keep Work File Location Local
BP_MUXX_004
Although the component's interface lets you change the location of the MatchUp work files, you should only override this setting after thorough testing for operating systems and speed.
general info here
MatchUp: Back up your Matchcode database
BP_MUXX_005
general info here
Leverage SSIS
BP_XXSS_003
One of the nice things about SSIS is that you can use a canned component (like Conditional split) to separate source records by zipcode into different streams and run the jobs separately. If you have the hardware, this may allow you to take advantage of multi-threading of the processor and multi-instance of the components.
Limit Pass Through Fields
BP_XXSS_004
The greater the number of Pass-through fields required will slow down the process.
You should pass a unique record identifier as a passthru, then using the source record identifier, join the results with the source fields not used outside the component.
Run on a local instance
BP_XXSS_005
VERY IMPORTANT - We are not convinced that VM memory management can be replicated as well as a physical machine - at least not without advanced configuration on your part and high performance hardware hosting the VM - which is beyond the scope of our supporting the performance of the component. Below are some links to not unrelated concerns with VM performance...
http://www.sqlsolutions.com/articles/articles/SQL_Server_and_VMware-A_Potentially_Fatal_Combination.htm
http://stackoverflow.com/questions/149318/virtualized-sql-server-why-not
http://virtualization.info/en/news/2008/12/running-sql-server-in-virtual-machine.html
http://www.hanselman.com/blog/VirtualMachineCPUPerformance.aspx
http://stackoverflow.com/questions/2678119/is-there-a-reason-why-ssis-significantly-slows-down-after-a-few-minutes
These are not meant as a deterrent, only to point out the potential issue.
Minimum Hardware Requirements
BP_XXSS_006
Although our components are tested and function on older hardware configurations, we "officially" support a recommended hardware configuration and will "unofficially" support configurations in which the components will run. Hardware configurations below the minimum requirements are not supported, in which we will advise hardware upgrades.