Cloud Service:Best Practices: Difference between revisions
No edit summary |
No edit summary |
||
(7 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
==Accessing Melissa Data® cloud services and staying up to date== | ==Accessing Melissa Data® cloud services and staying up to date== | ||
Melissa Data Cloud Services are authenticated using | Melissa Data Cloud Services are authenticated using a License Key. The License Key is provided to you by your customer service representative or through an automatically generated email after signing up for a promotional demo. | ||
Line 41: | Line 41: | ||
</pre> | </pre> | ||
We give | We give our best effort to plan ahead and allocated IP ranges for the future, however, it is possible that we will need to add IP ranges as we expand and upgrade our infrastructure. We will send out multiple notifications of any impending update. This is a reason to make sure you have your technical contacts registered with Melissa Data | ||
(See [[#Accessing Melissa Data’s cloud services and staying up to date|Accessing Melissa Data’s Cloud Services]]). | (See [[#Accessing Melissa Data’s cloud services and staying up to date|Accessing Melissa Data’s Cloud Services]]). | ||
Line 58: | Line 58: | ||
===Use batch instead of single=== | ===Use batch instead of single=== | ||
Processing 100 records at a time will always have a faster per record speed than a single record. | Processing <span style="color:red;">**</span>100 records at a time will always have a faster per record speed than a single record. | ||
<span style="color:red;"><nowiki>**</nowiki></span>The exception to this rule is Global Email. If you are using Global Email in Premium Mode, we recommend a max of 10 records. If you attempt to do 100 records at a time with Global Email in Premium mode, it will break apart the records and in effect slow down the process. | |||
===Order by Zip code=== | ===Order by Zip code=== | ||
Line 104: | Line 106: | ||
The XML and REST protocols require special character handling when creating your request and interpreting the response. This is because certain special characters are reserved within the XML language. | The XML and REST protocols require special character handling when creating your request and interpreting the response. This is because certain special characters are reserved within the XML language. | ||
===XML Encoding=== | |||
The following XML encoding should be implemented to correctly handle reserved special characters: | The following XML encoding should be implemented to correctly handle reserved special characters: | ||
{|class="alternate01" cellspacing="0" | {|class="alternate01" cellspacing="0" | ||
Line 126: | Line 129: | ||
===URL Encoding=== | |||
The following URL encoding should be implemented when using REST to correctly handle reserved special characters: | The following URL encoding should be implemented when using REST to correctly handle reserved special characters: | ||
{|class="alternate01" cellspacing="0" | {|class="alternate01" cellspacing="0" | ||
Line 131: | Line 135: | ||
!style="border-right:0px;"|Encoding | !style="border-right:0px;"|Encoding | ||
|- | |- | ||
|Space | |[Space] | ||
|%20 or + | |%20 or + | ||
|- | |- | ||
Line 148: | Line 152: | ||
|$ | |$ | ||
|%28 | |%28 | ||
|- | |||
|` | |||
|%60 | |||
|- | |||
|: | |||
|%3A | |||
|- | |||
|< | |||
|%3C | |||
|- | |||
|> | |||
|%3E | |||
|- | |||
|[ | |||
|%5B | |||
|- | |||
|] | |||
|%5D | |||
|- | |||
|{ | |||
|%7B | |||
|- | |||
|<nowiki>}</nowiki> | |||
|%7D | |||
|- | |||
|“ | |||
|%22 | |||
|- | |||
|<nowiki>+</nowiki> | |||
|%2B | |||
|- | |||
|( | |||
|%28 | |||
|- | |||
|) | |||
|%29 | |||
|- | |||
|. | |||
|%2E | |||
|- | |||
|! | |||
|%21 | |||
|- | |||
|@ | |||
|%40 | |||
|- | |||
|/ | |||
|%2F | |||
|- | |||
|; | |||
|%3B | |||
|- | |||
|= | |||
|%3D | |||
|- | |||
|? | |||
|%3F | |||
|- | |||
|\ | |||
|%5C | |||
|- | |||
|^ | |||
|%5E | |||
|- | |||
|<nowiki>|</nowiki> | |||
|%7C | |||
|- | |||
|~ | |||
|%7E | |||
|- | |||
|‘ | |||
|%27 | |||
|- | |||
|, | |||
|%2C | |||
|- | |||
|<nowiki>-</nowiki> | |||
|%2D | |||
|- | |||
|_ | |||
|%5F | |||
|} | |} | ||
Line 159: | Line 244: | ||
If you are experiencing a sudden loss of service, please be ready to provide us with as much of the following pieces of information so that we can assist you promptly and investigate thoroughly. | If you are experiencing a sudden loss of service, please be ready to provide us with as much of the following pieces of information so that we can assist you promptly and investigate thoroughly. | ||
*Your | *Your License Key | ||
*The location of your server accessing the cloud service | *The location of your server accessing the cloud service | ||
*The URL of the cloud service you are using | *The URL of the cloud service you are using | ||
Line 169: | Line 254: | ||
*Are you using a proxy or firewall? | *Are you using a proxy or firewall? | ||
*Any other information you think may be relevant | *Any other information you think may be relevant | ||
==Output Considerations== | |||
Melissa Data Cloud Services are constantly undergoing improvements and additions. As the products evolve, the format of the output may change as features are added to each of the services. This does not mean that the current elements or element names will be subject to sudden change, as this is considered bad practice, but that additional possible elements may be added to the products. Melissa Data reserves the right to add these additional features in order to improve and advance its products to meet customer needs. | |||
In terms of cloud services, many of our services tend to offer an option to return back all available outputs, usually denoted with something like a “GrpAll” option. As a result of this option, certain problems in handling the output of may arise. When a new field might be added to the service, the response would contain this new field when all the elements are called to return. This could conflict with the often static schema in which outputs are generally parsed - for all formats, be it JSON, XML, or SOAP. Our services require that the parsing of the response can be dynamic in its allocation of the output, in order to avoid this error. In order to have true static output and avoid the possibility of this error, we recommend that each of the desired output columns is listed in the request individually. | |||
[[Category:Cloud Services]] | [[Category:Cloud Services]] | ||
[[Category:Best Practices]] | [[Category:Best Practices]] |
Latest revision as of 18:19, 18 November 2019
Accessing Melissa Data® cloud services and staying up to date
Melissa Data Cloud Services are authenticated using a License Key. The License Key is provided to you by your customer service representative or through an automatically generated email after signing up for a promotional demo.
Due to the nature of our business and our commitment to improving our services, sometimes there will be changes. We strongly recommend you create an email distribution group address for several technical users in your organization, that will be responsible for consuming our services, and convey that email address to your Melissa Data customer service representative. This email address will be used in case we need to notify you of upcoming change either to the service, the data or the network infrastructure. Also kindly add melissadata.com to your domain whitelist so this vital communication is not blocked.
Single Record vs Batch Processing
Melissa Data cloud services are capable of both single record real-time processing and batch processing. The difference is simply in the number of records sent in each request. Melissa Data cloud services take an array of records. This array can contain a single record or 100 records. For a real time process like a web form entry or a call center application, send in a request with one record. For batch processing scenario like a database, send requests of up to 100 records until all the records are processed.
Error Handling
Since Melissa Data uses multiple clustered servers, in the unlikely event that one server is temporarily overloaded or down, it is very likely that another server is up and running. For this reason, it is best to catch any errors returned by the service and retry. We recommend retrying up to 5 times.
int Retry = 0; Boolean ReqRet = false; do { try { // Perform Phone Lookup and store results to the Response ResPhone = PhoneClient.doPhoneCheck(ReqPhone); ReqRet = true; } catch (Exception ex) { MessageBox.Show(ex.ToString()); Retry++; } } while ((ReqRet == false) && (Retry < 5));2
Network Layer Considerations
IP Addresses
If your company uses a firewall for security, you will need to add our IP address ranges to your list of allowable IP addresses in order to communicate with our cloud services. Please see this page:
http://www.melissadata.com/tech/webservices-ip-information.htm
We give our best effort to plan ahead and allocated IP ranges for the future, however, it is possible that we will need to add IP ranges as we expand and upgrade our infrastructure. We will send out multiple notifications of any impending update. This is a reason to make sure you have your technical contacts registered with Melissa Data (See Accessing Melissa Data’s Cloud Services).
Ports
Melissa Data uses the default ports of 80 for http and 443 for https.
Using specific IP’s
Melissa Data does not allow you to access our services using direct IP addresses. We have multiple server clusters and use a load balancing system to distribute incoming traffic. To update our servers, we have to periodically need to take them down and redistribute traffic to available servers.
TTL
To update a server, we take it off of our load balancer and let it drain of traffic. Then, we stop the cloud service and update the server. People accessing the service using our domain name will automatically be directed to available servers only. However, if your DNS caches the resolved IP address for a significant length of time, it is possible the cached server will be down when you try to hit it. For this reason, if you have access to your DNS TTL (time to live) option, please set it to a value of 10 minutes or less.
Speed Considerations
If speed is a top concern for you, there are a few strategies that can be employed to increase the normal speed of using the service. They are ranked in terms of least effective to most effective.
Use batch instead of single
Processing **100 records at a time will always have a faster per record speed than a single record.
**The exception to this rule is Global Email. If you are using Global Email in Premium Mode, we recommend a max of 10 records. If you attempt to do 100 records at a time with Global Email in Premium mode, it will break apart the records and in effect slow down the process.
Order by Zip code
Ordering the records by zip code within each request can have a small increase the speed of the request.
Compression
For our services that allow GZip compression, turning it on can have up to a 10% increase in speed. GZip is usually built into a programming language and turned on simply by a flag.
Multi-Threading
Increasing the number of threads is an effective way to quickly increase the speed of using our services. We recommend having between 3-5 threads for batch processing.
Contact Verification Server
For our enterprise customers with speed and/or privacy concerns, we can make available a local server hosting our cloud services. This server is dedicated to you so that you are not sharing server resources with anyone else. Additionally, since the server is located within your organization, the data does not have to travel over long distances. Also, authentication is turned off to further increase speed. Contact your sales representative if you are interested in this solution.
Understanding the different protocols SOAP vs XML vs REST vs JSON
Melissa Data cloud services are available in multiple protocols in order to provide our customers a wide range of technologies to fit into their current architecture. The terminologies used above (SOAP, XML, REST, JSON) are not strictly technologically accurate, but used to differentiate the different options available. In the end, all the protocols return the same data, so the choice is really one of programming preference.
SOAP
Ideal for languages with SOAP development toolkits like Visual Studio .NET. This makes consuming and using the service very easy. The SOAP protocol can be used for both single record processing and batch.
A side note for Java developers planning on using SOAP with our cloud services: please consider using XML instead. Using SOAP entails generating code from the WSDL to invoke the service's methods which is not always a sure process as changes or upgrades to the cloud services in the future could break client code. Using XML instead is less problematic based on our experiences working with clients on such cases. Both XML and SOAP have similar capabilities in that both protocols allows single record and batch processing.
XML
XML is a HTTP POST call where you send in a request in xml format and receive a response in xml format. The format of the request must follow the structure defined in our documentation for each specific service.
JSON
JSON is also a HTTP POST call just like XML. The only difference is that the request and response uses the JSON format instead of XML format.
REST
REST is a HTTP GET call. This allows you to quickly formulate a call within the URL and submit it through a browser. An example of a REST call:
https://personator.melissadata.net/v3/WEB/ContactVerify/ doContactVerify?&id=123456&act=Check&a1=22382%20avenida %20empresa&postal=92688
The returned response is in XML and is identical to what is returned by the XML HTTP POST call.
Only one record can use sent using a REST call so it is ideal for real time single record processing.
Special Character Handling
The XML and REST protocols require special character handling when creating your request and interpreting the response. This is because certain special characters are reserved within the XML language.
XML Encoding
The following XML encoding should be implemented to correctly handle reserved special characters:
Character | Encoding |
---|---|
& | & (ampersand) |
“ | " (left/right quotes should be replaced with straight quotes) |
‘ | ' (apostrophe) |
< | < (less-than) |
> | > (greater-than) |
URL Encoding
The following URL encoding should be implemented when using REST to correctly handle reserved special characters:
Character | Encoding |
---|---|
[Space] | %20 or + |
* | %2A |
# | %23 |
& | %26 |
% | %25 |
$ | %28 |
` | %60 |
: | %3A |
< | %3C |
> | %3E |
[ | %5B |
] | %5D |
{ | %7B |
} | %7D |
“ | %22 |
+ | %2B |
( | %28 |
) | %29 |
. | %2E |
! | %21 |
@ | %40 |
/ | %2F |
; | %3B |
= | %3D |
? | %3F |
\ | %5C |
^ | %5E |
| | %7C |
~ | %7E |
‘ | %27 |
, | %2C |
- | %2D |
_ | %5F |
Reporting Errors
If you are experiencing errors trying to connect to our cloud services or interruptions to the service, you can:
- Submit a support ticket at support.melissadata.com.
- Call Tech Support at 1-800-800-6245 ext 4.
If you are experiencing a sudden loss of service, please be ready to provide us with as much of the following pieces of information so that we can assist you promptly and investigate thoroughly.
- Your License Key
- The location of your server accessing the cloud service
- The URL of the cloud service you are using
- The type of error you are experiencing in detail
- Any error messages or logs you may have
- Date and time of occurrences
- Behavior of the incidents (is it all the time or periodic. If peridoc, what percentage of requests are affect).
- What is your IP Address?
- Are you using a proxy or firewall?
- Any other information you think may be relevant
Output Considerations
Melissa Data Cloud Services are constantly undergoing improvements and additions. As the products evolve, the format of the output may change as features are added to each of the services. This does not mean that the current elements or element names will be subject to sudden change, as this is considered bad practice, but that additional possible elements may be added to the products. Melissa Data reserves the right to add these additional features in order to improve and advance its products to meet customer needs.
In terms of cloud services, many of our services tend to offer an option to return back all available outputs, usually denoted with something like a “GrpAll” option. As a result of this option, certain problems in handling the output of may arise. When a new field might be added to the service, the response would contain this new field when all the elements are called to return. This could conflict with the often static schema in which outputs are generally parsed - for all formats, be it JSON, XML, or SOAP. Our services require that the parsing of the response can be dynamic in its allocation of the output, in order to avoid this error. In order to have true static output and avoid the possibility of this error, we recommend that each of the desired output columns is listed in the request individually.