Web APIs are often the best way to go when different parts of your application need to communicate, or when people outside of your organization need to communicate with your application. Not only are web apis based upon a widely understood and implemented protocol, but they are also widely accepted across industry. Further, if your API is structured properly, using HTTP enables developers using a wide variety of platforms and languages to easily interact with it.
However, there is always a price to wide acceptance and easy use. Usually, that price is that there are expectations to how your system will behave, expectations both obvious and surprising. This is the case with building a REST API. While many concepts are fairly obvious and easily understood, their implications can occasionally be surprising.
Before we get into the episode a little review of REST. Representational State Transfer, which is an architectural style for communications between systems on the web. REST systems are characterized by statelessness and separation of the concerns of the client and the server. Defined as request/response between the client and the server.
Many apps have one or more components that are exposed through a RESTful API. If you do this well and make it easy to work with your API, people will find it easier to start connecting to your system. This is one of the best ways to get and keep larger clients, as most companies are far more hesitant to disconnect from automated systems than they might from manual ones. Building your API using REST best practices will reduce the friction for these clients.
Improper Use of Response Codes
There are known response codes for various states of the server. Use them. Another common mistake is to create redirect loops accidentally. It’s very common to see incorrect response codes (for instance, returning a 200 along with an error message).
- Informational responses (100-199) – Notables => 101: Switching protocol
- Success responses (200-299) – 200: Ok, 204: No content
- Redirects (300-399) – 301: Moved Permanently
- Client Errors (400-499) – 401: Unauthorized, 404: Not found, 403: Forbidden
- Server Errors (500-599) – 500 -Internal Server error
Improper Use of HTTP Verbs
Typical mistakes in this area include using the wrong verb for the intent of your action. This would be things like using a GET to make changes to the system or POST to retrieve something.
- Get – Gets things from the server.
- Head – Identical to a get, but doesn’t get the body.
- Post – Submits an entity to the resource.
- Put – Replaces a resource.
- Delete – Deletes a resource.
- Patch – Applies partial modifications to a resource.
Incorrect use MIME types
The client is responsible for specifying what type of content they want and they negotiate that with the server. A common failure is when this doesn’t happen. Another common mistake is to completely ignore the content type requested, or to send content back that doesn’t have the mime type set (sending json back as text, for instance).
- text/plain is exactly what it sounds like, raw text.
- text/html is also exactly what it sounds like, html for use in a browser.
- application/x-www-form-urlencoded sends the body of an HTTP message as one large string with name/value pairs separated by the ampersand.
- multipart/form-data sends name/value pairs as separate parts of the message separated by a string boundary.
Incorrect use of Caching
The purpose of caching is to reduce the use of bandwidth, reduce latency, reduce the load on servers, and hide issues with connections and servers. Caching is heavily dependent on http verbs. GET requests are cached by default, for instance. POST requests are not cached by default, but can be cached using the expire and cache-control headers. Responses to PUT and DELETE requests are not cacheable at all. The Expire header is used to indicate when the result of a request is to be considered stale and needs to be re-validated. ETags are string tokens that a server associates with a resource to uniquely identify the state of the resource over time. When the resource is changed, the eTag changes. Cache-control headers are composed of a set of directives indicating whether a response is cacheable, who it can be cached by, and for how long. Common mistakes in this area include failing to cache, caching for an incorrect amount of time, or caching when you shouldn’t.
Improper Request Structuring
It’s also common to see people incorrectly structure things based on the type of request or response. For instance, you should include any required parameters inside the path on a GET request, not in the body of the request. Similarly, cross-cutting concerns, such as authentication tokens, should generally be placed in headers, instead of the body. Data being placed into the system via a PUT, POST, or PATCH command should typically be in the request body. The response structure is under similar constraints, based upon what HTTP verbs were supplied. Improperly structuring requests tends to be confusing to consumers of your REST API, and is best avoided if you want your system to be easy to work with.
Poor handling of errors
Error codes should accurately indicate what went wrong, along with who originated the problem. Server-side errors start with 500 and indicate that the problem occurred on the server. Client-side errors start with 400 and indicate that the client did something wrong. It’s very common to see these get mixed up. For instance, let’s say that the client requests a resource that doesn’t exist. If this causes a null reference exception, it would generate a 500, rather than the 404 that it should. Similarly, you also need to use the correct codes within each section to indicate what is going on. This aids in debugging. For instance, a 404 means that the resource isn’t found, whereas a 405 is used to indicate that you can’t use a particular http verb on a resource.
Weak or incorrect documentation can cause a lot of issues for anyone working with your API. While written documentation is useful, a lot of your documentation should be built around the notion of keeping documentation close to the thing being documented. You should probably look into something like the OpenAPI specification or Swagger for making it easy to work with and test out your API without having to write custom code. Any sort of API testbed should allow clients to work with their own data, or should let them work with a reasonable sample dataset. Such tools can also be used to generate client SDKs for your API in the client’s target language. You should also consider having a test environment so that clients can try things out without the risk of damaging their own data. Such environments should also have ways of generating common errors based on input data so that clients can test their response to common issues. Most API documentation of this sort is produced using some form of annotation on your code, and you’ll want to prioritize keeping this up to date during the development lifecycle.
Poor consideration of scaling
As an API gets used more frequently, scaling will become an issue that impacts more and more of your clients. Caching will help you put off these issues, but isn’t enough once you reach enough scale. Early on in your design, you should consider replacing synchronous workflows with asynchronous ones wherever it makes sense. Typically, this is done with webhooks. Prioritize replacing any polling loops with webhooks in general. “Don’t call us, we’ll call you” will protect your servers from wasting time on useless requests. You might also want to consider pre-building response payloads that are frequently requested and infrequently changed. Such things could potentially be stored cheaply in a CDN or in something like blob storage. Also make sure and rate limit client requests while separating test and production systems. These strategies will keep client mistakes from damaging your system. Unlike many of the other API antipatterns here, this one will be a constantly morphing, ongoing concern.
Your app will evolve over time. It’s unlikely that all of your clients will evolve at the same speed, even if they are all internal. Therefore a versioning strategy is critical to success. If your application is not new, you probably will need to support at least three versions of your API in production. You’ll need to have a beta version of your next api version so that clients can work out bugs. You’ll need the existing (or stable) version of your system for ongoing work. You’ll probably also need to keep the previous version online for a while so that clients can migrate their systems. You’ll also have multiple environments. At the minimum, you’ll probably have environments for production, testing, and sales demos. You’ll have to keep these separate from each other, and they probably have to work across versions.
John C. Maxwell
This week we are looking at the sixth quality of a team player: dependable. “Teams go to go-to players.” Maxwell starts off telling about Christopher Reeves, the man who played Superman in the original movies, and how he was thrown from a horse and became a quadriplegic. He went from being in peak physical condition to needing a team of people just to function daily. Reeves had to depend on these people for his very life. Maxwell then talks about what he sees as the essence of dependability. First he says that a dependable team member will have pure motives, not putting their agenda before the team. Next is responsibility, team members desire to do the things they are capable of doing. Third is good judgement which must be combined with responsibility to make a dependable teammate. Finally he states that dependable teammates are consistent. Closing out the chapter he gives three ways to improve your own dependability: Look at your motives, Find out what your word is worth, and get someone to be an accountability partner.
Tricks of the Trade
It’s all political. Consider the political implications of what you do or you will suffer from them.