Cloud Service:Best Practices: Difference between revisions

From Melissa Data Wiki
Jump to navigation Jump to search
Created page with "==Accessing Melissa Data’s web services and staying up to date== Melissa Data Web Services are accessed using a CustomerID or license string. This ID is provided to you by y..."
 
No edit summary
 
(14 intermediate revisions by the same user not shown)
Line 1: Line 1:
==Accessing Melissa Data’s web services and staying up to date==
==Accessing Melissa Data® cloud services and staying up to date==
Melissa Data Web Services are accessed using a CustomerID or license string. This ID is provided to you by your sales representative or through an automatically generate demo email. The nature of our business and our commitment to improving our services require periodic changes. We strongly recommend you provide the technical contact emails responsible for using and maintaining our service to your sales representative. In the case we need to notify you of upcoming changes, we want all stakeholders to receive that notice. We also recommend setting up a distribution email as one of your technical contacts so you can adjust the distribution list to any changes in personnel. Please add melissadata.com to your whitelist so our emails are not blocked.  
Melissa Data Cloud Services are authenticated using a License Key. The License Key is provided to you by your customer service representative or through an automatically generated email after signing up for a promotional demo.  
 
 
Due to the nature of our business and our commitment to improving our services, sometimes there will be changes. We strongly recommend you create an email distribution group address for several technical users in your organization, that will be responsible for consuming our services, and convey that email address to your Melissa Data customer service representative. This email address will be used in case we need to notify you of upcoming change either to the service, the data or the network infrastructure. Also kindly add melissadata.com to your domain whitelist so this vital communication is not blocked.




==Single Record vs Batch Processing==
==Single Record vs Batch Processing==
Melissa Data web services are capable of both single record real-time processing and batch processing. The difference is simply in the number of records sent in each request. Melissa Data web services take an array of records. This array can contain a single record or 100 records. For a real time process like a web form entry or a call center application, send in a request with one record. For batch processing scenario like a database, send requests of up to 100 records until all the records are processed.  
Melissa Data cloud services are capable of both single record real-time processing and batch processing. The difference is simply in the number of records sent in each request. Melissa Data cloud services take an array of records. This array can contain a single record or 100 records. For a real time process like a web form entry or a call center application, send in a request with one record. For batch processing scenario like a database, send requests of up to 100 records until all the records are processed.  




Line 32: Line 35:
==Network Layer Considerations==
==Network Layer Considerations==
===IP Addresses===
===IP Addresses===
If your company uses a firewall for security, you will need to add our IP address ranges to your list of allowable IP addresses in order to communicate with our web services. Please see this page:
If your company uses a firewall for security, you will need to add our IP address ranges to your list of allowable IP addresses in order to communicate with our cloud services. Please see this page:


<pre>
<pre>
Line 38: Line 41:
</pre>
</pre>


We give out best effort to plan ahead and allocated IP ranges for the future, however, it is possible that we will need to add IP ranges as we expand and upgrade or infrastructure. We will send out multiple notifications of any impending update. This is a reason to make sure you have your technical contacts registered with Melissa Data  
We give our best effort to plan ahead and allocated IP ranges for the future, however, it is possible that we will need to add IP ranges as we expand and upgrade our infrastructure. We will send out multiple notifications of any impending update. This is a reason to make sure you have your technical contacts registered with Melissa Data  
(See [[#Accessing Melissa Data’s web services and staying up to date|Accessing Melissa Data’s Web Services]]).
(See [[#Accessing Melissa Data’s cloud services and staying up to date|Accessing Melissa Data’s Cloud Services]]).
 


===Ports===
===Ports===
Melissa Data uses the default ports of 80 for http and 443 for https.  
Melissa Data uses the default ports of 80 for http and 443 for https.  


===Using specific IP’s===
===Using specific IP’s===
Melissa Data does not allow you to access our services using direct IP addresses. We have multiple server clusters and use a load balancing system to distribute incoming traffic. To update our servers, we have to periodically need to take them down and redistribute traffic to available servers.  
Melissa Data does not allow you to access our services using direct IP addresses. We have multiple server clusters and use a load balancing system to distribute incoming traffic. To update our servers, we have to periodically need to take them down and redistribute traffic to available servers.  


===TTL===
===TTL===
To update a server, we take it off of our load balancer and let it drain of traffic. Then, we stop the web service and update the server. People accessing the service using our domain name will automatically be directed to available servers only. However, if your DNS caches the resolved IP address for a significant length of time, it is possible the cached server will be down when you try to hit it. For this reason, if you have access to your DNS TTL (time to live) option, please set it to a value of 10 minutes or less.  
To update a server, we take it off of our load balancer and let it drain of traffic. Then, we stop the cloud service and update the server. People accessing the service using our domain name will automatically be directed to available servers only. However, if your DNS caches the resolved IP address for a significant length of time, it is possible the cached server will be down when you try to hit it. For this reason, if you have access to your DNS TTL (time to live) option, please set it to a value of 10 minutes or less.  




==Speed Considerations==
==Speed Considerations==
If speed is a top concern for you, there are a few strategies that can be employed to increase the normal speed of using the service. They are ranked in terms of least effective to most effective.
If speed is a top concern for you, there are a few strategies that can be employed to increase the normal speed of using the service. They are ranked in terms of least effective to most effective.


===Use batch instead of single===
===Use batch instead of single===
Processing 100 records at a time will always have a faster per record speed than a single record.
Processing <span style="color:red;">**</span>100 records at a time will always have a faster per record speed than a single record.


<span style="color:red;"><nowiki>**</nowiki></span>The exception to this rule is Global Email. If you are using Global Email in Premium Mode, we recommend a max of 10 records. If you attempt to do 100 records at a time with Global Email in Premium mode, it will break apart the records and in effect slow down the process.


===Order by Zip code===
===Order by Zip code===
Ordering the records by zip code within each request can have a small increase the speed of the request.
Ordering the records by zip code within each request can have a small increase the speed of the request.


===Compression===
===Compression===
For our services that allow GZip compression, turning it on can have up to a 10% increase in speed. GZip is usually built into a programming language and turned on simply by a flag.
For our services that allow GZip compression, turning it on can have up to a 10% increase in speed. GZip is usually built into a programming language and turned on simply by a flag.


===Multi-Threading===
===Multi-Threading===
Increasing the number of threads is an effective way to quickly increase the speed of using our services. We recommend having between 3-5 threads for batch processing.
Increasing the number of threads is an effective way to quickly increase the speed of using our services. We recommend having between 3-5 threads for batch processing.


===Contact Verification Server===
===Contact Verification Server===
For our enterprise customers with speed and/or privacy concerns, we can make available a local server hosting our web services. This server is dedicated to you so that you are not sharing server resources with anyone else. Additionally, since the server is located within your organization, the data does not have to travel over long distances. Also, authentication is turned off to further increase speed. Contact your sales representative if you are interested in this solution.
For our enterprise customers with speed and/or privacy concerns, we can make available a local server hosting our cloud services. This server is dedicated to you so that you are not sharing server resources with anyone else. Additionally, since the server is located within your organization, the data does not have to travel over long distances. Also, authentication is turned off to further increase speed. Contact your sales representative if you are interested in this solution.




==Understanding the different protocols SOAP vs XML vs REST vs JSON==
==Understanding the different protocols SOAP vs XML vs REST vs JSON==
Melissa Data web services are available in multiple protocols in order to provide our customers a wide range of technologies to fit into their current architecture. The terminologies used above (SOAP, XML, REST, JSON) are not strictly technologically accurate, but used to differentiate the different options available. In the end, all the protocols return the same data, so the choice is really one of programming preference.
Melissa Data cloud services are available in multiple protocols in order to provide our customers a wide range of technologies to fit into their current architecture. The terminologies used above (SOAP, XML, REST, JSON) are not strictly technologically accurate, but used to differentiate the different options available. In the end, all the protocols return the same data, so the choice is really one of programming preference.
 


===SOAP===
===SOAP===
Ideal for languages with SOAP development toolkits like Visual Studio .NET. This makes consuming and using the service very easy. The SOAP protocol can be used for both single record processing and batch.
Ideal for languages with SOAP development toolkits like Visual Studio .NET. This makes consuming and using the service very easy. The SOAP protocol can be used for both single record processing and batch.


A side note for Java developers planning on using SOAP with our cloud services: please consider using XML instead. Using SOAP entails generating code from the WSDL to invoke the service's methods which is not always a sure process as changes or upgrades to the cloud services in the future could break client code. Using XML instead is less problematic based on our experiences working with clients on such cases. Both XML and SOAP have similar capabilities in that both protocols allows single record and batch processing.


===XML===
===XML===
XML is a HTTP POST call where you send in a request in xml format and receive a response in xml format. The format of the request must follow the structure defined in our documentation for each specific service.  
XML is a HTTP POST call where you send in a request in xml format and receive a response in xml format. The format of the request must follow the structure defined in our documentation for each specific service.  


===JSON===
===JSON===
JSON is also a HTTP POST call just like XML. The only difference is that the request and response uses the JSON format instead of XML format.
JSON is also a HTTP POST call just like XML. The only difference is that the request and response uses the JSON format instead of XML format.


===REST===
===REST===
Line 111: Line 106:
The XML and REST protocols require special character handling when creating your request and interpreting the response. This is because certain special characters are reserved within the XML language.  
The XML and REST protocols require special character handling when creating your request and interpreting the response. This is because certain special characters are reserved within the XML language.  


===XML Encoding===
The following XML encoding should be implemented to correctly handle reserved special characters:
The following XML encoding should be implemented to correctly handle reserved special characters:
{|class="alternate01" cellspacing="0"
{|class="alternate01" cellspacing="0"
Line 133: Line 129:




===URL Encoding===
The following URL encoding should be implemented when using REST to correctly handle reserved special characters:
The following URL encoding should be implemented when using REST to correctly handle reserved special characters:
{|class="alternate01" cellspacing="0"
{|class="alternate01" cellspacing="0"
Line 138: Line 135:
!style="border-right:0px;"|Encoding
!style="border-right:0px;"|Encoding
|-
|-
|Space
|[Space]
|%20 or +
|%20 or +
|-
|-
Line 155: Line 152:
|$
|$
|%28
|%28
|-
|`
|%60
|-
|:
|%3A
|-
|<
|%3C
|-
|>
|%3E
|-
|[
|%5B
|-
|]
|%5D
|-
|{
|%7B
|-
|<nowiki>}</nowiki>
|%7D
|-
|“
|%22
|-
|<nowiki>+</nowiki>
|%2B
|-
|(
|%28
|-
|)
|%29
|-
|.
|%2E
|-
|!
|%21
|-
|@
|%40
|-
|/
|%2F
|-
|;
|%3B
|-
|=
|%3D
|-
|?
|%3F
|-
|\
|%5C
|-
|^
|%5E
|-
|<nowiki>|</nowiki>
|%7C
|-
|~
|%7E
|-
|‘
|%27
|-
|,
|%2C
|-
|<nowiki>-</nowiki>
|%2D
|-
|_
|%5F
|}
|}




==Reporting Errors==
==Reporting Errors==
If you are experiencing errors trying to connect to our web services or interruptions to the service, you can:
If you are experiencing errors trying to connect to our cloud services or interruptions to the service, you can:


#Submit a support ticket at support.melissadata.com.  
#Submit a support ticket at support.melissadata.com.  
Line 166: Line 244:


If you are experiencing a sudden loss of service, please be ready to provide us with as much of the following pieces of information so that we can assist you promptly and investigate thoroughly.
If you are experiencing a sudden loss of service, please be ready to provide us with as much of the following pieces of information so that we can assist you promptly and investigate thoroughly.
*Your Customer ID
*Your License Key
*The location of your server accessing the web service
*The location of your server accessing the cloud service
*The URL of the web service you are using
*The URL of the cloud service you are using
*The type of error you are experiencing in detail
*The type of error you are experiencing in detail
*Any error messages or logs you may have
*Any error messages or logs you may have
Line 178: Line 256:




[[Category:Web Services]]
==Output Considerations==
Melissa Data Cloud Services are constantly undergoing improvements and additions. As the products evolve, the format of the output may change as features are added to each of the services. This does not mean that the current elements or element names will be subject to sudden change, as this is considered bad practice, but that additional possible elements may be added to the products. Melissa Data reserves the right to add these additional features in order to improve and advance its products to meet customer needs.
 
In terms of cloud services, many of our services tend to offer an option to return back all available outputs, usually denoted with something like a “GrpAll” option. As a result of this option, certain problems in handling the output of may arise. When a new field might be added to the service, the response would contain this new field when all the elements are called to return. This could conflict with the often static schema in which outputs are generally parsed - for all formats, be it JSON, XML, or SOAP. Our services require that the parsing of the response can be dynamic in its allocation of the output, in order to avoid this error. In order to have true static output and avoid the possibility of this error, we recommend that each of the desired output columns is listed in the request individually.
 
 
[[Category:Cloud Services]]
[[Category:Best Practices]]
[[Category:Best Practices]]

Latest revision as of 18:19, 18 November 2019

Accessing Melissa Data® cloud services and staying up to date

Melissa Data Cloud Services are authenticated using a License Key. The License Key is provided to you by your customer service representative or through an automatically generated email after signing up for a promotional demo.


Due to the nature of our business and our commitment to improving our services, sometimes there will be changes. We strongly recommend you create an email distribution group address for several technical users in your organization, that will be responsible for consuming our services, and convey that email address to your Melissa Data customer service representative. This email address will be used in case we need to notify you of upcoming change either to the service, the data or the network infrastructure. Also kindly add melissadata.com to your domain whitelist so this vital communication is not blocked.


Single Record vs Batch Processing

Melissa Data cloud services are capable of both single record real-time processing and batch processing. The difference is simply in the number of records sent in each request. Melissa Data cloud services take an array of records. This array can contain a single record or 100 records. For a real time process like a web form entry or a call center application, send in a request with one record. For batch processing scenario like a database, send requests of up to 100 records until all the records are processed.


Error Handling

Since Melissa Data uses multiple clustered servers, in the unlikely event that one server is temporarily overloaded or down, it is very likely that another server is up and running. For this reason, it is best to catch any errors returned by the service and retry. We recommend retrying up to 5 times.

int Retry = 0;
Boolean ReqRet = false;
do
{
  try
  {
    // Perform Phone Lookup and store results to the Response
    ResPhone = PhoneClient.doPhoneCheck(ReqPhone);
    ReqRet = true;
  }
  catch (Exception ex)
  {
    MessageBox.Show(ex.ToString());
    Retry++;
  }
} while ((ReqRet == false) && (Retry < 5));2


Network Layer Considerations

IP Addresses

If your company uses a firewall for security, you will need to add our IP address ranges to your list of allowable IP addresses in order to communicate with our cloud services. Please see this page:

http://www.melissadata.com/tech/webservices-ip-information.htm

We give our best effort to plan ahead and allocated IP ranges for the future, however, it is possible that we will need to add IP ranges as we expand and upgrade our infrastructure. We will send out multiple notifications of any impending update. This is a reason to make sure you have your technical contacts registered with Melissa Data (See Accessing Melissa Data’s Cloud Services).

Ports

Melissa Data uses the default ports of 80 for http and 443 for https.

Using specific IP’s

Melissa Data does not allow you to access our services using direct IP addresses. We have multiple server clusters and use a load balancing system to distribute incoming traffic. To update our servers, we have to periodically need to take them down and redistribute traffic to available servers.

TTL

To update a server, we take it off of our load balancer and let it drain of traffic. Then, we stop the cloud service and update the server. People accessing the service using our domain name will automatically be directed to available servers only. However, if your DNS caches the resolved IP address for a significant length of time, it is possible the cached server will be down when you try to hit it. For this reason, if you have access to your DNS TTL (time to live) option, please set it to a value of 10 minutes or less.


Speed Considerations

If speed is a top concern for you, there are a few strategies that can be employed to increase the normal speed of using the service. They are ranked in terms of least effective to most effective.

Use batch instead of single

Processing **100 records at a time will always have a faster per record speed than a single record.

**The exception to this rule is Global Email. If you are using Global Email in Premium Mode, we recommend a max of 10 records. If you attempt to do 100 records at a time with Global Email in Premium mode, it will break apart the records and in effect slow down the process.

Order by Zip code

Ordering the records by zip code within each request can have a small increase the speed of the request.

Compression

For our services that allow GZip compression, turning it on can have up to a 10% increase in speed. GZip is usually built into a programming language and turned on simply by a flag.

Multi-Threading

Increasing the number of threads is an effective way to quickly increase the speed of using our services. We recommend having between 3-5 threads for batch processing.

Contact Verification Server

For our enterprise customers with speed and/or privacy concerns, we can make available a local server hosting our cloud services. This server is dedicated to you so that you are not sharing server resources with anyone else. Additionally, since the server is located within your organization, the data does not have to travel over long distances. Also, authentication is turned off to further increase speed. Contact your sales representative if you are interested in this solution.


Understanding the different protocols SOAP vs XML vs REST vs JSON

Melissa Data cloud services are available in multiple protocols in order to provide our customers a wide range of technologies to fit into their current architecture. The terminologies used above (SOAP, XML, REST, JSON) are not strictly technologically accurate, but used to differentiate the different options available. In the end, all the protocols return the same data, so the choice is really one of programming preference.

SOAP

Ideal for languages with SOAP development toolkits like Visual Studio .NET. This makes consuming and using the service very easy. The SOAP protocol can be used for both single record processing and batch.

A side note for Java developers planning on using SOAP with our cloud services: please consider using XML instead. Using SOAP entails generating code from the WSDL to invoke the service's methods which is not always a sure process as changes or upgrades to the cloud services in the future could break client code. Using XML instead is less problematic based on our experiences working with clients on such cases. Both XML and SOAP have similar capabilities in that both protocols allows single record and batch processing.

XML

XML is a HTTP POST call where you send in a request in xml format and receive a response in xml format. The format of the request must follow the structure defined in our documentation for each specific service.

JSON

JSON is also a HTTP POST call just like XML. The only difference is that the request and response uses the JSON format instead of XML format.

REST

REST is a HTTP GET call. This allows you to quickly formulate a call within the URL and submit it through a browser. An example of a REST call:

https://personator.melissadata.net/v3/WEB/ContactVerify/
doContactVerify?&id=123456&act=Check&a1=22382%20avenida
%20empresa&postal=92688

The returned response is in XML and is identical to what is returned by the XML HTTP POST call.

Only one record can use sent using a REST call so it is ideal for real time single record processing.


Special Character Handling

The XML and REST protocols require special character handling when creating your request and interpreting the response. This is because certain special characters are reserved within the XML language.

XML Encoding

The following XML encoding should be implemented to correctly handle reserved special characters:

Character Encoding
& & (ampersand)
" (left/right quotes should be replaced with straight quotes)
' (apostrophe)
< < (less-than)
> > (greater-than)


URL Encoding

The following URL encoding should be implemented when using REST to correctly handle reserved special characters:

Character Encoding
[Space] %20 or +
* %2A
# %23
& %26
% %25
$ %28
` %60
: %3A
< %3C
> %3E
[ %5B
] %5D
{ %7B
} %7D
%22
+ %2B
( %28
) %29
. %2E
! %21
@ %40
/ %2F
; %3B
= %3D
? %3F
\ %5C
^ %5E
| %7C
~ %7E
%27
, %2C
- %2D
_ %5F


Reporting Errors

If you are experiencing errors trying to connect to our cloud services or interruptions to the service, you can:

  1. Submit a support ticket at support.melissadata.com.
  2. Call Tech Support at 1-800-800-6245 ext 4.


If you are experiencing a sudden loss of service, please be ready to provide us with as much of the following pieces of information so that we can assist you promptly and investigate thoroughly.

  • Your License Key
  • The location of your server accessing the cloud service
  • The URL of the cloud service you are using
  • The type of error you are experiencing in detail
  • Any error messages or logs you may have
  • Date and time of occurrences
  • Behavior of the incidents (is it all the time or periodic. If peridoc, what percentage of requests are affect).
  • What is your IP Address?
  • Are you using a proxy or firewall?
  • Any other information you think may be relevant


Output Considerations

Melissa Data Cloud Services are constantly undergoing improvements and additions. As the products evolve, the format of the output may change as features are added to each of the services. This does not mean that the current elements or element names will be subject to sudden change, as this is considered bad practice, but that additional possible elements may be added to the products. Melissa Data reserves the right to add these additional features in order to improve and advance its products to meet customer needs.

In terms of cloud services, many of our services tend to offer an option to return back all available outputs, usually denoted with something like a “GrpAll” option. As a result of this option, certain problems in handling the output of may arise. When a new field might be added to the service, the response would contain this new field when all the elements are called to return. This could conflict with the often static schema in which outputs are generally parsed - for all formats, be it JSON, XML, or SOAP. Our services require that the parsing of the response can be dynamic in its allocation of the output, in order to avoid this error. In order to have true static output and avoid the possibility of this error, we recommend that each of the desired output columns is listed in the request individually.