,

API Series: Indentation for Humans

APIs are built for machines, and when it comes to working with data inputs and outputs, machines are very good at parsing and interpreting data very quickly. With all responses in the Znode API as JSON strings, it can sometimes be hard to read responses with the human eye, especially for large responses. While this doesn’t matter to machines, it can be useful at times to open an endpoint in a browser to see the results. For this, you can use the indent parameter for better readability.

Take for example the following endpoint to get the list of payment gateways:

GET /paymentgateways

If you open this endpoint in a browser, you’ll see something like this on screen:

{"PaymentGateways":[{"Name":"Authorize.Net","PaymentGatewayId":1,"Url":"http://www.authorize.net"},{"Name":"Verisign PayFlow Pro","PaymentGatewayId":2,"Url":"http://www.verisign.com"},{"Name":"Paymentech Orbital","PaymentGatewayId":3,"Url":"http://www.paymentech.com"},{"Name":"PayPal Direct Payment","PaymentGatewayId":4,"Url":"http://www.paypal.com"},{"Name":"WorldPay","PaymentGatewayId":5,"Url":"http://www.worldpay.com"},{"Name":"CyberSource","PaymentGatewayId":6,"Url":"http://www.cybersource.com"},{"Name":"Custom","PaymentGatewayId":7,"Url":"NA"}],"PageIndex":null,"PageSize":null,"TotalPages":null,"TotalResults":null,"ErrorCode":null,"ErrorMessage":null,"HasError":false}

Granted, it’s not a massive amount of data, but you can still make it easier for your own human eyes to comprehend that response by using the indent parameter and setting it to true, as such:

GET /paymentgateways?indent=true

Now when you open that endpoint in a browser, you get back the exact same response, but formatted for human readability:

{
  "PaymentGateways": [
    {
      "Name": "Authorize.Net",
      "PaymentGatewayId": 1,
      "Url": "http://www.authorize.net"
    },
    {
      "Name": "Verisign PayFlow Pro",
      "PaymentGatewayId": 2,
      "Url": "http://www.verisign.com"
    },
    {
      "Name": "Paymentech Orbital",
      "PaymentGatewayId": 3,
      "Url": "http://www.paymentech.com"
    },
    {
      "Name": "PayPal Direct Payment",
      "PaymentGatewayId": 4,
      "Url": "http://www.paypal.com"
    },
    {
      "Name": "WorldPay",
      "PaymentGatewayId": 5,
      "Url": "http://www.worldpay.com"
    },
    {
      "Name": "CyberSource",
      "PaymentGatewayId": 6,
      "Url": "http://www.cybersource.com"
    },
    {
      "Name": "Custom",
      "PaymentGatewayId": 7,
      "Url": "NA"
    }
  ],
  "PageIndex": null,
  "PageSize": null,
  "TotalPages": null,
  "TotalResults": null,
  "ErrorCode": null,
  "ErrorMessage": null,
  "HasError": false
}

So while not a mission critical piece of the Znode API, the indent parameter is very useful for those times when you just need to see an endpoint response in a browser.

,

API Series: Sorting and Paging

Last we left the API series, we were talking about expands and filters and the flexibility they provide when querying data. So today we’re going to discuss paging and sorting, which is extremely important for limiting the amount of data you need at any given time, and how it should be sorted when it gets back to you.

Sorting

When retrieving a list of items, the Znode API supports using the sort parameter to sort the results. The sort parameter is a key-value pair separated by the tilde (~) character, which is configurable. There are only two acceptable values, asc for ascending order and desc for descending order:

key~asc
key~desc

For example, to retrieve all products sorted by name in ascending order, you would use the following endpoint:

GET /products?sort=name~asc

And of course you can combine sorting with expands and filters. For example, to get a list of products and their SKUs, where product name contains “pea”, sorted by wholesale price in descending order, you would make the following API call:

GET /products?expand=skus&filter=name~cn~pea&sort=wholesaleprice~desc

Paging

In many cases, retrieving lists of items needs to be done a page at a time, which the Znode API handles by providing the page parameter. For example, to retrieve page 4 of a list of products with a page size of 25, you will will use the following endpoint:

GET /products?page=index~3,size~25

The page parameter consists of two key-value pairs: index and size. Both key-value pairs are separated by the tilde (~) character and should always be used together, separated by a comma.

NOTE: The index key-value is 0-based.

For endpoints in the Znode API that support paging, the following properties will be returned in the response:

Property Description
PageIndex The current page index that was requested.
PageSize The current page size that was requested.
TotalPages The total number of pages for the request.
TotalResults The total number of results for the request.

And here is an example of what these properties look like as part of the JSON response:

{
    "PageIndex": 3,
    "PageSize": 25,
    "TotalPages": 5,
    "TotalResults": 108,
}
,

API Series: Expands and Filters

Any API is really all about one thing: data. An API needs to accept data, do something with it, then give data back. For the consumer of an API, what happens in that middle part doesn’t matter. All they care about is that they sent the API some data to work with and the API returned data to them in a format they expect. And a big part of data for an API is being able to support rich querying capabilities so that the client gets back exactly what they need. For the Znode API, this is handled by expands and filters.

Expands

The concept of expands is nothing new in API design, as many APIs support them, and even our own implementation is loosely based on the OData specification for expands. In short, expands is a way of asking for related data during the same request.

For example, when retrieving a product, you might also want to get its customer reviews and any categories it belongs to. Without expands, you might have to do this in three separate calls, perhaps something like this:

GET /products/123
GET /products/123/reviews
GET /products/123/categories

But with expands, you can consolidate those three calls into one, like so:

GET /products/123?expand=reviews,categories

You can chain as many expands together as you need, simply use a comma to separate them (a configurable option we’ll cover in a later post). And that’s the big benefit of expands – to eliminate extraneous API calls that you can otherwise handle with one call.

Filters

Similar to expands, filters were also roughly based on their OData specification. Using filters allow you to retrieve only the data that satisfies an expression; think of filters as the WHERE clause in a SQL statement.

For example, you might want to retrieve a list of products that have a retail price greater than $5.00. With filters, the URL request would look like this:

GET /products?filter=retailprice~gt~5.00

Also like expands, you can chain filters together. For example, you want to retrieve a list of products that have a retail price greater than $5.00, but only for products whose name starts with the letter A. The request for that would look like this:

GET /products?filter=retailprice~gt~5.00,name~sw~A

Filter expressions are basically tuples that consist of the following format:

key~operator~value

The tilde character (~) is used as the default separator between key/operator/value, while the comma is used to separate multiple filter expressions (both of which are configurable).

Below is the list of currently supported filter operators:

Operator Description Example
eq Equal to GET /products?filter=name~eq~apple
ne Not equal to GET /products?filter=name~ne~apple
cn Contains GET /products?filter=name~cn~pea
gt Greater than GET /products?filter=retailprice~gt~4.99
ge Greater than or equal to GET /products?filter=retailprice~ge~5.00
lt Less than GET /products?filter=retailprice~lt~5.00
le Less than or equal to GET /products?filter=retailprice~le~4.99
sw Starts with GET /products?filter=name~sw~app
ew Ends with GET /products?filter=name~ew~pple

Filters in the Znode API can also be used to query for nullable items by using a “null” value for the eq (equal to) and ne (not equal to) operators, as such:

GET /products?filter=externalid~eq~null
GET /products?filter=externalid~ne~null

Powerful and Flexible

When it comes to querying data from an API, expands and filters are important because it gives clients great power and flexibility for how they want to retrieve data, and the Znode API has very robust expand and filter capabilities.

,

API Series: Caching

Here at Znode, performance is always at the front of our brains. We sell an ecommerce platform, so the faster it performs, the better the bottom line is for people who build sites on top of it. Performance is basically Feature #1, as literally every single potential customer asks about it when evaluating our software. And when it came to creating the Znode API, this was no exception.

Different Customers, Different Uses

We knew early on in the API design process that we were going to implement caching as part of the API, we just weren’t sure exactly what it was going to look like. But what we did know was that it needed to be a completely configurable solution, as every customer will have their own unique requirements for interacting with their ecommerce data.

Some customers will want to use sliding cache expirations, while others will require absolute expiration times. Some customers will want to cache their product data for 60 seconds, yet others will cache their products for an hour. And some customers won’t want to cache anything at all. They should all be given the opportunity to handle any of these scenarios, and do so without modifying, recompiling, and redeploying code.

Dial Up, Dial Down

Every customer has their own performance requirements, and giving them configurable cache options allows them to tweak and test until they are satisfied they’ve got the right settings. This flexibility lets customers fiddle with the dials, if you will, and they can do this using a simple configuration file that can be updated on-the-fly.

Cache.config

The solution we came up with is the cache.config file, located in the root of the Znode API. This file provides cache options for every GET endpoint in the API and is automatically tied to the AppDomain, so changes to this file are the same as making changes to the standard web.config file. It’s important to remember that API caching occurs for GET requests only; caching for POST, PUT, and DELETE are not supported (and of course, don’t make much sense).

Here’s a snippet of what the cache.config looks like:

<znodeApiCache>
    <cache enabled="true">
        <routes>
            <!-- Catalog routes -->
            <route template="catalogs/{catalogId}" enabled="true" sliding="true" duration="60" />
            <route template="catalogs" enabled="true" sliding="true" duration="60" />

            <!-- Category routes -->
            <route template="categories/{categoryId}" enabled="true" sliding="true" duration="60" />
            <route template="categories" enabled="true" sliding="true" duration="60" />
            <route template="categories/catalog/{catalogId}" enabled="true" sliding="true" duration="60" />

            <!-- Product routes -->
            <route template="products/{productId}" enabled="true" sliding="true" duration="60" />
            <route template="products/{productId}/{skuId}" enabled="true" sliding="true" duration="60" />
            <route template="products" enabled="true" sliding="true" duration="60" />
            <route template="products/{productIds}" enabled="true" sliding="true" duration="60" />
            <route template="products/catalog/{catalogId}" enabled="true" sliding="true" duration="60" />
            <route template="products/category/{categoryId}" enabled="true" sliding="true" duration="60" />
        </routes>
    </cache>
</znodeApiCache>

The root <cache> node allows you to enable or disable the entire API caching mechanism. If set to false, then no caching will occur at all for any routes.

The <routes> node contains a list of <route> nodes, each with the following attributes:

Attribute Description
template The route template as defined in the \App_Start\WebApiConfig.cs file.
enabled True/false value that determines if caching is enabled for the route (default = true).
sliding True/false value that determines if a sliding cache is used for the route; if false, absolute caching is used (default = true).
duration The number of seconds that the response data for the route should be kept in cache (default = 60 seconds).

Cache Keys And JSON Strings

There are two elements in the cache design that make all this possible:

1. The cache keys for storing data in the cache are the raw URLs of the request, including the entire querystring. A URL maps to a route, thus allowing us to cache data for that route and any parameters it may have, i.e. the querystring.

2. The data stored for each cache key is the raw JSON string returned for the request. Storing the JSON string itself, instead of an object that contains the JSON string, saves a serialization/deserialization trip, which increases performance even further.

Huge Performance Boost

Like any other cache solution, the caching built into the Znode API can provide massive performance gains for your websites and applications and should be utilized as much as possible.

,

API Series: HTTP Methods and Status Codes

As previously discussed, we based our API on REST to fully embrace and utilize the power of HTTP. However, with this comes certain responsibilities; after all, HTTP dictates specifications and guidelines that should be adhered to in order to properly work with the protocol. Two such items are HTTP methods and status codes.

The HTTP Specification

To ensure we understood exactly how and when to use certain methods and status codes, we spent quite a bit of time going through the actual HTTP specification, in particular those two areas. We kind of obsessed about it to be honest, but then again, we obsessed about all aspects of our API design. Many people say they have a RESTful API, yet under closer examination only take it about 80% of the way there. We wanted to be 100% RESTful, and the only way we could accomplish that was to truly understand those two sections of the HTTP specification.

HTTP Methods

Simply put, HTTP methods are actions to be taken by a web site and its underlying web server, and for the Znode API, we standardized on GET, POST, PUT, and DELETE to handle any request to all of our endpoints, as described below:

Method Description
GET Used for two scenarios – 1) retrieving a single object/resource or 2) retrieving a list of objects/resources.
Example: getting a product with ID 302 or getting a list of products for a particular category.
POST Used when creating a new object/resource.
Example: adding a new product to the store catalog.
PUT Used when updating an existing object/resource.
Example: updating the description and attributes for product with ID 302.
DELETE Used for deleting an existing object/resource.
Example: deleting product with ID 302.

HTTP Status Codes

Status codes, sometimes referred to as response codes, represent the result of a particular HTTP method. Status codes tell you whether or not the API call was successful, a key design principle in RESTful APIs. In general, codes in the 2xx range indicate success, codes in the 4xx range indicate an error that resulted from the provided information, and codes in the 5xx range indicate a server error.

Below are the status codes used by the Znode API and when they are used:

Code Description Method Usage
200 OK The request was successful. Returned when GET and PUT are successful.
201 Created The item sent in the request was successfully created. Returned when POST is successful.
204 No Content The request was successful, but there was nothing to return. Returned when DELETE is successful.
403 Forbidden API key was invalid or not supplied. Could happen on any request.
404 Not Found The requested item doesn’t exist. Returned when GET fails.
500 Internal Server Error Something went wrong processing the request. Returned when GET, POST, PUT, DELETE fails.

100% RESTful

We mentioned earlier that some people claim to be RESTful yet only make it about 80% of the way there. So what did we mean by that? Let’s discuss this with a couple examples.

Example 1. Creating a New Object

Let’s say an API has an endpoint that allows you to create a new object, in this case a product. And to create a new product, you must use the POST method, sending in the data used to create the product in the request body. In this case the endpoint is POST /products.

The problem is that most people get the response wrong when this call is successful. What they do is return a 200 OK status code along with the newly created object. This is incorrect. What they should do is actually return a status code of 201 Created and also populate the Location response header with the value of the URL of the newly created object.

Example 2. Deleting an Object

Let’s say an API has an endpoint that supports deleting an object, in this case a product. And to delete a product from this endpoint, you must use the DELETE method and put the product ID in the URL, as such: DELETE /products/302. So far, so good.

However, what many people do when this DELETE call is successful is return a 200 OK status. This is also incorrect. If the DELETE was successful, the resource at that endpoint no longer exists, thus, the correct status code should be 204 No Content, which says the call was successful but there is nothing to return.

Our API Gets It Right

The above two examples are what we mean by having an API that is 100% RESTful, and our Znode API handles those actual scenarios the way they should be handled.

Consistent and Predictable

What all of this ultimately comes down to is having an API that is consistent, because a consistent API is a predictable API, and predictability is one of the guiding tenets of our API design. You can be assured that when you create a new object in our API, you’ll get back a 201 Created status code with a Location header, and that when you delete an object you’ll get back a 204 No Content status code.