SSIS:Best Practices

From Melissa Data Wiki
Revision as of 23:19, 18 November 2019 by Admin (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

← SSIS:Data Quality Components

Expired License and Community Edition Mode

When a license string is expired, SSIS will enable Community Edition mode unless a valid license is entered

SSIS BP CommunityEdition.png

The Community Edition of SSIS has limited functionality. Providing a few basic abilites for you to sample our products. Some options may be greyed out. If you continue to run the project with a community license, the component will attempt to remove mappings not supported by the license in order to run. Please make sure to backup your configuration in order to avoid data mapping loss.

For more information on the Community Edition features, see the SSIS Subscribe wiki page.

Adding a Valid License

To add a valid license, follow the steps below.
1. Click the Apply License button.
SSIS will now prompt the Advanced Configuration screen
SSIS BP CommunityEdition AdvConfig1.png
2. Select the Processing Mode from the drop down menu - On-Premise, Melissa Data Cloud or Dedicated Cloud.
SSIS BP CommunityEdition AdvConfig2.png
3. Once the Processing Mode has been selected, insert the valid license into the License Key box.
4. Click Test Configuration.
SSIS BP CommunityEdition AdvConfig3.png
5. Click Close, then OK. The valid license string is now loaded into SSIS

Saving Configuration File

Field Mappings

We recommend backing up your configuration files for each project made in SSIS. The configuration file holds all the field mapping information made in the project.
The following steps will show how to save the configuration file.
1. Open the project that has all the field mappings set
2. Double click the component to access the interface
3. Click File, then choose Save as .cfg file.
SSIS BP SaveConfig Save.png
4. Choose a name and directory to save the configuration file.
SSIS BP SaveConfig SaveMessage.png

Loading Configuration File

The following instructions will display how to load a saved configuration file.
1. Open project and double click the component to access the interface
2. Click File, then choose Open .cfg file.
SSIS BP SaveConfig Open.png
3. Locate the previously saved configuration file and click Open.
SSIS BP SaveConfig Reload.png
4. A message will appear stating the configuration was successfully loaded.
SSIS BP SaveConfig Load.png
SSIS will load the field mappings contained within the configuration file into your project.

Running SSIS packages from the Command Line

For more efficient memory usage, run your saved SSIS package from the command line instead of running directly from visual studio. Example: dtexec.exe /F "c:\fullpath\Package.dtsx" /Rep EWPD > "OptionalCreateLog.log" tests have shown memory usage to be up to 66% less. Running from visual studio will start devenv and DtsDebugHost processes in your Task Manager, whereas command line processing will start DTEXEC.

Update to the latest version.

We always recommend users use the most current build (version), which may contain new or improved features, code fixes, or updated functionality. Melissa data provides many ways to ensure you keep your components to the latest version.

MatchUp: Optimize Your Matchcode

Applying certain optimizations, or configuring the matchcode settings, can greatly reduce processing time without sacrificing accuracy. For a complete discussion see the Matchcodes > Optimization section of the MatchUp SSIS WebHelp or

MatchUp: Keep Work File Location Local

Although the component's interface lets you change the location of the MatchUp work files, you should only override this setting after thorough testing for operating systems and speed.

MatchUp: Back up your Matchcode database


Leverage SSIS

One of the nice things about SSIS is that you can use a canned component (like Conditional split) to separate source records by zipcode into different streams and run the jobs separately. If you have the hardware, this may allow you to take advantage of multi-threading of the processor and multi-instance of the components.

Limit Pass Through Fields

The greater the number of Pass-through fields required will slow down the process. You should pass a unique record identifier as a passthru, then using the source record identifier, join the results with the source fields not used outside the component.

Run on a local instance

VERY IMPORTANT - We are not convinced that VM memory management can be replicated as well as a physical machine - at least not without advanced configuration on your part and high performance hardware hosting the VM - which is beyond the scope of our supporting the performance of the component. Below are some links to related concerns with VM performance...

These are not meant as a deterrent, only to point out the potential issue.

Minimum Hardware Requirements

Although our components are tested and function on older hardware configurations, we "officially" support a recommended hardware configuration and will "unofficially" support configurations in which the components will run. Hardware configurations below the minimum requirements are not supported, in which we will advise hardware upgrades.

SSIS System Requirements