Professional Documents
Culture Documents
Pragmatic Recommendations For API Design
Pragmatic Recommendations For API Design
1. Introduction
1.1. HTTP/2
2. REST Background
2.1. What is REST?
2.2. REST Architecture
2.3. REST Constraints
2.3.1. ClientServer
2.3.2. Statelessness
2.3.3. Caching
2.3.4. Uniform Interface
2.3.5. Layered System
2.3.6. Code on Demand (Optional)
3. REST API Maturity Levels (Richardson Maturity Model)
3.1. Level 0
3.2. Level 1
3.3. Level 2
3.4. Level 3
4. Resources
4.1. Resource types
4.1.1. Document Resources
4.1.2. Store Resources
4.1.3. Collection Resources
4.1.4. Controller Resources
4.2. Enterprise resources
4.2.1. Entities
4.2.2. Operations
4.2.3. Entity updates with farreaching effects
4.3. URI structure
4.4. Filtering
4.4.1. Searches
4.4.2. Resource content filtering
4.4.3. Filtering and request types
4.5. Sorting
4.6. Paging
4.6.1. Paging links
4.7. ToDo’s for Resource Identifiers
4.8. ToDos for Resource representation
5. HTTP Methods
5.1. Note on content negotiation
5.2. CRUD Methods
5.3. NonCRUD methods
5.4. Idempotence
5.5. Safety
5.6. Nonidempotent, nonsafe
5.7. RESTful use of methods
5.7.1. GET
5.7.1.1. Collection Resources
5.7.1.2. Document Resources
5.7.2. POST
5.7.2.1. POST to a Collection Resource
5.7.2.2. POST to a Controller Resource on an Operation resource
5.7.2.3. POST to the execute Controller Resource on an Operation Collection
1
5.7.3. PUT
5.7.4. PATCH
5.7.5. DELETE
5.7.6. OPTIONS
5.7.7. The “ContentLocation Response Pattern”
5.8. Example interactions
6. Caching
6.1. The HTTPS Caveat
6.2. The Authentication Caveat
6.3. Server Responsibility
6.4. Other Effects on Caching
7. Status Codes
200 OK
201 Created
202 Accepted
204 No Content
304 Not Modified
400 Bad Request
401 Unauthorized
403 Forbidden
404 Not Found
405 Method Not Allowed
406 Not Acceptable
(408 Request Timeout)
409 Conflict
410 Gone
412 Precondition Failed
500 Internal Server Error
501 Not Implemented
503 Service Unavailable
8. Hypermedia
8.1. Representation Proposal
8.2. Relation Types
8.2.1. Use of Structural Relation Types
8.2.2. Use of Business Relation Types
8.3. The OPTIONS Response format
8.3.1. Example 1: A Collection Resource
8.3.2. Example 2: A Document Resource
8.3.3. Example 3: Collection Resources and Multitype Item Creation
8.3.4. The OPTIONS Media Type
8.4. HATEOAS and Documentation
9. Headers
9.1. Entity Headers
9.1.1. ContentEncoding
9.1.2. ContentLanguage
9.1.3. ContentLength
9.1.4. ContentType
9.2. Request Headers
9.2.1. Accept
9.2.2. AcceptCharset
9.2.3. AcceptEncoding
9.2.4. AcceptLanguage
9.2.5. CacheControl
9.2.6. IfMatch
9.2.7. IfNoneMatch
9.2.8. IfModifiedSince
9.2.9. IfUnmodifiedSince
2
9.3. Response Headers
9.3.1. Allow
9.3.2. CacheControl, Expires, and Vary
9.3.3. ContentLocation
9.3.4. ETag
9.3.4.1. What is an Entity Tag?
9.3.4.2. Generating an Entity Tag
9.3.4.3. Comparing Entity Tags
9.3.4.4. Usage Considerations
9.3.5. LastModified
9.3.6. Link header
9.3.7. Location
10. API Versioning
11. REST API Security
11.1. Authentication
11.2. Authorization
11.3. Other Considerations
12. Do’s and Don’ts
13. Samples and Recommendations
13.1. GitHub API analysis
13.2. Twitter API analysis
13.3. What we can learn
3
1. Introduction
This document is intended as a guide for designing a robust REST API. A well designed API must meet
certain objectives to be useful:
● Lightweight and easy to use
● Require no special tooling (even if tooling to facilitate its use is available)
● Loose integration contracts
● Selfdocumenting and discoverable, assumes little or no preexisting knowledge by consumers
● Versionable to handle changes to underlying business objects
The rationale and business value of creating APIs for our software products in the first place has been
established in a separate document titled “The Business Value of APIs”.
Although this document will be useful to programmers, it is not a programmer’s guide and will not
recommend specific implementation frameworks. The practicalities related to the implementing and
running the resultant design are also not part of this document, but will be covered elsewhere.
1.1. HTTP/2
There is a new standard in the HTTP world HTTP/2.
This is, however, a bit of a misnomer. The specification isn’t actually
a new version of HTTP as a whole,
it is “simply” a new line protocol that can significantly improve the performance of HTTP interactions
between client and server.
Nevertheless, in order to coherently specify this new line protocol, the HTTP 1.1 specification, embodied
mainly in RFC 2616, had to be revised to more cleanly separate the line protocol and application protocol
parts of the specification.
As part of this revision, the authors took the opportunity to update, clarify and “tweak” the specification of
HTTP 1.1 application features and, in particular, to address some of the uncertainties related to using
HTTP in RESTful applications.
Consequently, where this document references HTTP, it is (unless otherwise stated) referring to the
new
RFCs that define HTTP 1.1 and not to the (now deprecated) RFC 2616 description.
2. REST Background
2.1. What is REST?
REpresentational State Transfer (abbreviated as REST) is a simple way to organize interactions between
independent systems.
Highlights:
● Everything is Represented as a Resource
● Resources have State
● State is Transferred over the wire (HTTP)
The design rationale for REST is based on the fact that the web is a massively scalable distributed
software system that works effectively and reliably. We should therefore be able leverage the success of
its underlying architecture to build integrated systems more easily.
In the
words of Roy Fielding he who first expressed the concept (and with a touch of hyperbole)
:
4
A REST API is just a website for users with a limited vocabulary (machinetomachine) interaction
Due to the relative simplicity of interacting with REST services, they make an excellent candidate
architecture for a product API.
REST is not a Standard such as HTML, CSS, XML or WSDL. The W3C will not ratify REST
REST is not a Protocol such as HTTP or SOAP (although it does get a specific mention in the
updated HTTP 1.1 Specifications created as part of the HTTP/2 standardization process
).
2.3.1. Client-Server
The principle of Separation of Concerns encourages us to isolate the user interface from data storage
and business processing concerns.
The result is:
● Improved portability of UIs across OS and device platforms
● Improved scalability by simplifying server components
● Client and Server components evolve independently
2.3.2. Statelessness
Following on from the clientserver constraint is the assertion that communication must be stateless. The
server should not be relied upon to maintain
application state (as opposed to
resource state) by storing
session objects for example. Each request from client to server must contain all of the information
necessary to understand the request, and cannot take advantage of any stored application context on the
server. Session state is therefore maintained entirely by the client.
Note that imposing statelessness is a tradeoff in which simplicity and scalability may come at the
expense of increase in network traffic since the server “forgets” application state between requests. This
tradeoff is normally addressed by adjusting the granularity of the exchanges between client and server to
reduce chattiness. In general REST is most efficient for largergrained resources, and by exploiting...
2.3.3. Caching
As with the web, the state of HTTP accessible resources can be saved in a clientside cache so we don’t
have to hit the server each time the resource representation is required. The Cache constraint requires
that resources be identified as cacheable if appropriate. If a response to a request is cacheable, then a
client can choose to cache and reuse that response for subsequent requests for a configurable period of
time.
The four parts of the interface that must remain uniform are:
5
1. Resource Identification
Resources are uniquely identifiable using URIs such as:
http://api.csc.com/user/ecartman
2. Resource Representation
A
representation
of the state of a resource is produced by the server as a response when it is
requested using its resource identifier.
For example, the following request
GET http://api.csc.com/employee/ecartman
may return the following state representation:
{
"empid": "123456"
"name": "Eric Cartman",
"role": "Deputy Sheriff"
}
3. SelfDescriptive Messages
Each client request and server response is a standard HTTP message consisting of headers and
an optional body. The body+headers should contain all the information necessary to complete
the requested task on both ends of the exchange. For example, response messages should
explicitly indicate their cacheability.
This type of message exchange is also referred to as stateless and contextfree.
4. Hypermedia as the engine of application state (A.K.A. HATEOAS)
Clients make state transitions only through actions that are dynamically identified by the server
within the current state of the application (e.g., by hyperlinks within a resource representation).
Except for simple fixed entry points to the application, an ideal client does not assume that any
particular action is available for any particular resource beyond those described in representations
previously received from the server.
6
3. REST API Maturity Levels (Richardson Maturity Model)
It is common while designing REST APIs to get drawn into academic discussions about what is “truly”
REST. You may find that as soon as you share a document or code describing or implementing your
API, someone will claim that it is not “really” REST.
A constructive and pragmatic way of approaching this is to refer to the REST Maturity Model developed
by Leonard Richardson. This model describes four levels of maturity for REST API implementation.
Note that in this section, and all following sections, we refer to GET, POST, and PUT et al as HTTP
methods because that is the term used by the HTTP specifications and the IANA HTTP Method
Registry
. However these concepts are often also referred to as HTTP verbs
in other nonnormative
contexts.
3.1. Level 0
Use
HTTP
to tunnel RPC calls usually by POSTing plain old XML (POX) to a single URI endpoint.
At level 0 of the model, HTTP is used for transporting data from client to server without taking advantage
of the web’s features and capabilities. Requests and responses are usually tunnelled through a special
protocol without considering application state. This usually involves the use of a single entry point (such
as http://ws.csc.com/blackhole ) and the POST method.
SOAP and XMLRPC are considered to be REST level 0 APIs.
3.2. Level 1
Use meaningfully named
Resources
.
A problem with SOAP and XMLRPC is that remote resources cannot be uniquely identified without
inspecting the contents of the request payload.
When an API can distinguish between different resources, it is probably at level 1. This is done by using
multiple URIs, where every URI refers to a specific resource.
We start referring to resources as:
or
http://api.csc.com/employees
http://api.csc.com/products
This level still only uses a single HTTP method (usually POST).
3.3. Level 2
Use appropriate HTTP
methods
for different actions.
This level indicates that your API is beginning to exploit the properties of the HTTP protocol in order to
deal with scalability and failures; the POST method is no longer being used for all interactions; GET is
being used to request resources, and DELETE for deleting resources.
The list of methods used
should
be as limited as possible, constrained as far as possible to those that
are defined by the HTTP specifications themselves.
While not specifically mentioned, level 2 also implies correct normative use of
HTTP status codes
as
well; Use 200 OK code only for success and when the response message contains an entity.
3.4. Level 3
Use
Hypermedia
controls to describe capabilities of Resources.
7
This is where REST really takes off. Level 3 indicates that the resources of the API expose URI links to
other resources within the application, both other “entity” resources and “action” resources. Resource
representations that include such data are known as hypermedia representations.
This is the start of using HTTP as the state machine for your application what the original thesis
describing REST calls Hypermedia as the engine of application state HATEOAS which basically
means that by following the URIs contained in your current
resource (or your context
resource), you can
make all the legal state transitions from your current state. By implication, the context resource should
not
advertise URIs for illegal transitions from its current state e.g. a nondeletable resource should not
advertise a DELETE link (although see below for more on the difference between the semantics of the
API and how the server implements those semantics).
Level 3 isn’t really an arrival point, it is more a departure point. One can be minimally at Level 3 when the
application resources simply contain discoverable links to other resources. However, also making
discoverable the invocation semantics of the links, and the effects of the transitions those links represent
is another level of HATEOAS, and part of a process of refinement that can continue almost ad infinitum,
towards a completely discoverable API which allows consumer components to be built with fewer and
fewer preconceptions (or “out of band” knowledge) of the application behind the API.
8
4. Resources
In REST each resource can be considered to have it's own API for example it is no longer a case of
"the insurance API" it is a case of "the policies collection API", then the "policy API", then the linked
"coverages collection API", then the "coverage API" etc. etc.
It is helpful, at least at this level, to treat each resource in a similar fashion to an OO object (in fact Roy
Fielding originally referred to REST as the “ HTTP Object Model ”):
● A resource encapsulates a single concept to which we want to provide an API
● A resource has state, part of which comprises links to other resources c.f. an OO instance
containing references to other instances
● A resource’s API is “discoverable” c.f. introspection, or reflection, in OO languages
● A resource is responsible
solely for exposing and handing it’s own state and API
Applying these concepts to resource API design ensures that the overall application API is as flexible,
loosely coupled, and dynamic as possible.
It can be instructive to think of any system fronted by a REST API as simply a web site a network of
web pages (resources) that are logically and/or conveniently linked with each other via well links
(URIs).
In REST literature we usually find that resources are categorised into one of four archetypes.
Document resources may have child resources, or collections of child resources, as the application’s
resource model requires.
An example might be a resource that represents a bookmark list. The client adds references to resources
to this list, removes them when they are no longer interesting, and references them as desired.
In that respect they can also be considered to be directories of subresources managed by the server,
and, in many cases, also “factories” of the resources they contain.
9
The client never
directly
adds a resource to a collection. Instead it usually “proposes” to the collection
resource that a new resource be created and inserted into the collection. The client may however directly
address the resource after it has been created and may even be permitted to directly delete the resource
(depending upon the business need of course).
An example here might be the collection of claims that have been made against an insurance policy.
They can be attached to any of the above three resource types one may think of those attached to store
resources, or, even more so, collection resources, as class methods, and those available on document
resources as instance methods but the OO analogy shouldn’t be pushed too far.
4.2.1. Entities
Entities represent "what is in the portfolio" over a period of time. For Insurance, one can name contracts,
coverages, risk insureds, other parties,...
Entities typically have identity and lifetime/lifecycle.
Entities have attributes, they can be created and then queried (e.g. 'retrieve all contracts created today').
They can also hold references to other entities to which they are related, and can be organized in
hierarchical relationships.
So it is fairly straightforward to see that entities map very cleanly to
Document resources
.
4.2.2. Operations
Operations represent "what actions can be, or have been, taken" against their
associated
Entities. For
an insurance contract, for example, these could include
surrender,
issuePremium ,
validateClaimFolder ,...
With this initial view of operations, they could seem to fit naturally into the
Controller
resource archetype
.
However, in many ways, many (even most) operations can be considered very much
like
Entities:
● they have attributes (e.g. their execution arguments)
● they can be created and then queried:
retrieve all validateClaimFolder where user == "SMITH"
● they can be updated/deleted (consider the gradual population of the arguments of an operation
over a period of time)
● they are ALSO persistent their state can be retrieved at any time after creation
● and they have identity
By taking
this
more sophisticated view of operations, we are reifying
the implicit "action" model of
operations, and making them "first class resources" in their own right.
10
And if this view of operations is modelled, then they become
Document resources
, organized as child
resources of the entity resource to which they apply.
The difference between Entities and Operations is that Operations can be executed
to update their
associated Entity. In this way they are analogous to methods available on the "object" encapsulated by
their associated entity, in the same way that controller resources can be soconsidered.
Note, however, that "execution" of an Operation modeled as a Document resource does NOT entail
deletion/destruction of that resource. Although, once executed, the Operation's state will, in all likelihood,
be changed to disable reexecution (at least until after an "undo"type action has been performed), that
state is queryable into the future, a feature that can be exploited for logging, audit, undo/redo...
purposes!
To model the
executable aspect of operations, our reified Operation resources themselves have
assigned actions that can be performed on the Operation resource, such as execute , ,
undo etc.
redo
These actions
are
modelled as
Controller resources
organized as child resources of the Operation
resource to which they apply.
An analogy from the Java world that might help clarify this approach is the ability to introspect an
application instance and obtain a
java.lang.reflect.Method instance that reifies the otherwise
implicit concept of a method defined on the application instance’s class. One can execute methods on
that Method instance, including calling its method to actually run the target method.
invoke
There are a number of reasons for modelling Operations in this way:
● Operations need to be logged and a history of operations executed on an Entity is certainly an
audit requirement as well as a customer service requirement
● Operations MAY be reversible
● The arguments required for an Operation can be numerous, where it may be more convenient to
build up the argument list of an Operation iteratively.
● A client application may need to manipulate several operations at one time. It is therefore useful
to ask the server for data initialization and validation before the complete argument data is
"committed" and the Operation is executed.
Executing an Operation (at a fine grained level) is achieved as follows:
● Create an Operation resource (from the ones available on an Entity, filtered based on user
authorization etc.),
● Populate the Operation's required attributes i.e. the operation arguments (e.g. validation date
and signature for a validateClaimFolder Operation)
● and then perform an allowable action on the Operation, represented by a Controller resource
attached to the Operation resource
However, it is probably desirable to implement some protocol on the server side to aggregate these three
steps in order to make operation invocations by a client appear more "action" like, and so less verbose
and more intuitive.
Such changes may need to be protected from "accidental" execution, logged, audited, and may well
require "undo" functionality. It may be preferable, then, for such changes to be
also
modeled as
Operations available on an Entity, rather than as simple attribute updates, even though
modeling as an
Operation is more complex and, possibly, less intuitive.
In many instances, even "simple" attribute updates (say, change of a person’s preferred communication
channel) will probably need to be logged. In these cases we MODEL the change as a "simple attribute
11
update" (e.g. modifying the status attribute of a communication channel resource in a resource update
dialog), but IMPLEMENT it as an Operation, the state of which can be retrieved as with any other
Operation. For this latter purpose, such "simple" updates can be implemented via a generic Operation,
available on all applicable Entities, named
update .
The important point is that the
client
side of the API the semantics of the API as seen by client should
be a natural expression of the client’s desired interactions, while the
implementation can be whatever is
necessary to assure the business effects of those interactions, and should not bleed into the API itself
In the REST model for Insurance a basic URI looks like this :
/csc/insurance/{component}/{mainEntityType}/{mainEntityID}/{detailType}/{detailID}
where
The segments identify a specific “instance” of the preceding DM entity type.
{...ID}
The {detailType}/{detailID} segments can, of course, appear multiple times to describe paths to
child resources of detail entities.
In addition, any path segment can instead specify the resource type "
operations " which then allows
the operation resources applicable to the parent resource to be accessed. Operation URIs have the form:
<path to parent resource>
/operations/{operationType}/{operationInstanceID}
Operation URIs can end with an "action" path segment, such as or
execute .
undo
URIs that end with ,
{mainEntityID} , and
{detailID} are modelled as
{operationInstanceID}
Document resources .
The "action" segment that can be applied to an operation resource maps directly to the
Controller
resource archetype
.
Note, again, that operations are being modelled as
Document resources, not as, what at first sight,
might appear the more natural classification of
Controller
resources. The reasoning behind this is
described above.
Finally, for retrieval requests, the use of predefined search or extract inquiry facilities (as described
below) can be specified in the URI.
12
● For "collection" and “store” resources, an "inquiry" ID can be specified in the query portion of the
URI in order to specify search and sort
criteria to apply to the list of URIs that the retrieval will
return. E.g:
.../contracts?inquiryID=contractInquiry1234&
parameters for contractInquiry1234
< >&
>
other list citeria e.g. paging
<
● For "document" resources, an "extract" ID can be specified in the query portion of the URI in order
to tailor the content of the resource representation that the retrieval will return. E.g:
.../contracts/123/investment_funds/4545?extractID=fundExtract2345&
<parameters for fundExtract2345>
A set of query parameters and syntax norms that allow a more "adhoc" approach to response content
tailoring should be considered.
Note that, although standard naming of the various components of the URI might be desirable, it is not
absolutely essential, especially in an API that conforms to
Level 3 of the Richardson Maturity model
where URIs to functionality (such as search functionality) are discovered dynamically.
4.4. Filtering
Filtering has two aspects to it:
● Reducing the number of results returned by a GET of some kind of list resource (such as a
Collection resource)
● Tailoring the
content
of the representation returned from a GET of a Document resource, OR
(more controversially) the attribute values included in each item returned for a list resource.
4.4.1. Searches
The first aspect corresponds to searching for a particular subset of items from a list resource and is a
vital capability to standardise. It is also a well established pattern on the web think Google.
The query parameters segment of a URI (those that follow a ?
placed at the end of the path segments of
the URI) is designed specifically for this type of functionality.
Some common vocabulary for expressing search criteria will be established and documented.
Additionally, the use of applicationspecific prepackaged searches (such as AIA’s Inquiries capability)
will be catered for, probably using a specific parameter key for the package ID in the query parameters,
and passing the rest of the parameters to the identified package as the arguments for the search.
● reducing the size of response data from the server,
● adding to that data information that might not normally be returned.
The AIA administration REST API has the following functionality that is a candidate approach for this type
of filtering:
● Each resource has a default set of attributes that it will return in a representation. This may or
may not be the full set of attributes that the resource holds.
● The URI query parameters may contain a special parameter named _attributes whose value is a
list of attribute names:
○ Attribute names that begin with a are removed from the set of attributes that are returned
13
○ Attribute names that appear normally are added to the set of attributes that are returned
○ The _none attribute removes all default attributes from the set of attributes that are
returned
○ The _all
attribute adds all optional attributes to the set of attributes that are returned
○ Consequently, combinations of _all/_none and other attribute names allow for a
completely customizable representation to be returned.
Again, prepackaged “views” of resources (such as AIA’s Extracts capability) will also be catered for,
probably using a specific parameter key in the query parameters for the package ID.
However for this aspect, it should be noted that the client may have to be aware of “out of band”
information (information not available or deducible from normal interactions with the resources
themselves) and, therefore, necessarily ties the client to the server. Consequently, provision of,
and reliance on, this type of filtering should be done sparingly if at all.
This is because GET is the only method (outside of the special methods HEAD and OPTIONS) that is
safe, idempotent, and cacheable, which means that the result of a GET of a particular URI even one
with query parameters attached to it can be safely cached and reused (within the confines of standard
cache management headers), thus freeing the HTTP protocol to do what it does best serve resources
at web scale!
4.5. Sorting
Sorting applies to lists of resources returned when GETting a collection or store resource. Since it is
closely related to filtering the result set and affects the content representation returned by the GET, the
logical place to specify sort criteria is, again, in the query parameters of the URI.
The default sort order of the resource
should
be logical to the resource type of the contained items, but is
technically undetermined.
Again, using the AIA administration REST API as an example, sort criteria can be specified using a
special query parameter key (such as _sort
) whose value is a list attribute names, in the order they
should be applied, with a prefix of if the value of a specific attribute should be used to sort “in reverse”.
For performance reasons it may be necessary to enforce the constraint that, to be eligible as a sort
attribute, the value of the attribute should be part of the “summary state” of the resource returned in the
list (which can be customized using filtering parameters as specified above).
As with resource content filtering, sorting on the server may involve the client being aware of “out of
band” information, although it may be possible to dynamically convey the legal sort fields for a resource
in the associated OPTIONS interaction data , thus avoiding binding the client too closely to the server.
4.6. Paging
Paging applies to results returned by a GET of some kind of list resource (such as a collection resource),
is usually only of concern to a UIcentric client, and is a “
necessary evil
when you have too many items to
easily show them all on one screen ”.
The usual approach for standard web pages is to have a default page size of a reasonable number of
items (perhaps device specific), and to allow the user to specify a different size in an application
preference store of some kind. These are, however, client concerns.
14
The server simply needs to know in a stateless manner the first item to return and the number of items to
return. Taking the classic example of Google search, these items are passed as query parameters, start
indicating the 0based index of the first item to return, and num the number of items to return. The
defaults are 0 and 10 respectively.
This is the approach that the AIA administration REST API takes (although it uses
_start
and
_num as
the query parameter names, and the default value of _num is unbounded to cater for unconstrained
requests from headless clients) and is suggested for the general approach.
Each page of items received in the response page will also contain paging links to the
first
,
previous
,
next
and
last
pages of the response set.
In large systems, where a request could potentially match millions of resources, the cost of simply
counting that many resources in order to calculate a URI for the last page may very well be so great as to
seriously degrade the responsiveness of the system. To circumvent this problem (again, from the AIA
administration REST API) a third pagingrelated query parameter will be allowed, _count, the value of
which is the number of items after which the server should merely indicate "there are more than this
number of items" (by returning a link to the next page but not to a last page).
URIs in links soreturned should be constructed according to the context of the current request. For
example, requests for the contents of a
persons resource that contains 200 items might appear as
follows:
GET .../persons?_num=20 :
first .../persons?_num=20
(the first page)
next: .../persons?_start=20&_num=20
(not present)
prev:
last: .../persons?_start=180&_num=20
GET .../persons?_start=20&_num=30 first:
.../persons?_num=30
(second page but new value)
_num
next: .../persons?_start=50&_num=30
prev: .../persons?_num=30
last: .../persons?_start=170&_num=30
GET .../persons?_start=37&_num=20 first:.../persons?_num=20
(arbitrary page start)
next: .../persons?_start=57&_num=20
prev: .../persons?_start=17&_num=20
last: .../persons?_start=197&_num=20
Note the following:
● The response for the first page should not contain a link
prev
15
● The response for the last page should not contain a next link
● The link to the first page should
not contain a
_start parameter
● The link to the last page should contain a _num parameter and that parameter should hold the
same value as was passed in the request, not the actual number of items that will appear on the
last page. This is in order to preserve the paging preferences previously set by the caller, and
communicated on the current request, if the caller subsequently decides to follow that link.
● If the
_count parameter is present in the request and, as a consequence, the server does not
calculate the exact size of the full response set, the response should not contain a last link.
Paging links should returned as part of the
hypermedia representation body
and
in the response
Link
header.
2. Different representations should not result in different URIs
3. Use media types
4. JSON primarily
5. XML as secondary
6. Possibly HTML for testing
16
5. HTTP Methods
HTTP interactions are governed by a standard but extensible set of methods that have very specific
semantics. As with most things HTTP these are laid down in the various HTTP “RFCs”. As of HTTP 2,
they are also registered in the
IANA HTTP Method Registry
.
The primary methods used in RESTful interactions are:
● GET obtain a representation of a resource
● HEAD return only the HTTP headers that a GET request would return
● PUT store a resource under the uri to which the request is sent
● POST a bit complicated see below
● PATCH (not an HTTP 1.1 standard method, but registered all the same) update certain
properties of a resource
● DELETE delete the addressed resource (from the client’s
perspective)
● OPTIONS obtain metadata pertaining to the addressed resource (broadly)
The use of each method is described below, but the methods fall into two main groups and are endowed
with two constraints that allow the HTTP protocol to be fault tolerant and highly responsive.
This is a fundamental concept of HTTP which allows HTTP clients (e.g. browsers) and HTTP servers to
agree on the format of the data returned by the server in response to a request.
In REST this negotiation is essential for separating the resource, represented by a formatneutral URI,
from the representation that is passed between client and server, which can be any format that both
support for the resource.
So, for example, a server may support returning the representation of a resource in JSON, XML, or
HTML. The client can express a preference for particular formats (even providing a preference rating) in
the request headers (specifically the Accept* headers). The server will return the resource
representation in the highest priority format that it supports, and specify the chosen formats in the
response headers (specifically the Content* headers). If the server cannot find a format that it supports
in the list, it can choose to either return a default format or to reject the request with a
406 Not
Acceptable status code.
But, whatever the format of the returned representation, the URI of the base resource is the same.
GET and DELETE are obvious they fulfil the R and D roles.
PUT and POST are multifaceted. Both can fulfil the C and U role, depending on the circumstances.
17
used to obtain just the HTTP headers that a GET of a resource would return, in order to discover more
about the resource without actually fetching the full representation (useful if generating that
representation is expensive, or if the representation is very large).
5.4. Idempotence
An idempotent HTTP method is one where the result of the action should be the same no matter how
many times it is executed on a specific URI. These methods allow the client to retry requests when the
status of a previous request isn’t discernible, for example (e.g. because of loss of connection).
The idempotent methods are OPTIONS, GET, HEAD, PUT and DELETE.
5.5. Safety
A safe HTTP method is one that does not affect the state of the addressed resource. It MAY have
serverside repercussions (e.g. logging), but the state of the addressed resource itself is not changed.
The use of safe methods allows for caching mechanisms to be deployed in the client and between the
client and the final server.
The safe methods are GET, HEAD, and OPTIONS.
Note that the safety constraint on the GET method precludes using this method to tunnel modification
requests through HTTP!
The former point simply means that after a POST or PATCH (or any other nonsafe method) any cached
copy of a resource representation must be invalidated.
The latter point, however, has some potentially serious consequences for the otherwise faulttolerant
nature of the HTTP protocol. There are techniques to address these issues (e.g. using conditional
requests and resource version tags, known as ETags).
Interestingly, a response to a POST request
is
specified as response is technically described as
cacheable, although a vast majority of caches do not support this functionality.
5.7.1. GET
GET is use to obtain (the content negotiated representation of) the resource identified by the URI,
optionally tailored by constraints expressed in URI query parameters and/or matrix URI formats.
Note that GET requests must not include a request ENTITY! Such use was expressly prohibited by
the original HTTP 1.1 specification and is simply not supported by most browsers and other HTTP client
software. The HTTP/2 rewrite of the HTTP 1.1 semantics allows for a request entity on GET (and
DELETE) solely
in order to remove
impediments of implementing the standard HTTP method parsing
algorithm introduced by HTTP/2 the semantics of such a body are defined as “having no meaning”!
18
5.7.1.1. Collection Resources
A GET method executed on a "collection" resource will return a list of the URIs of all the resources
considered to be “content” of that collection, where the list may be tailored (e.g. for paging purposes) by
query parameters appended to the request URI.
The absence of content resources will never result in a error, but, instead, result in an
404 Not Found
empty list.
Such a GET can result in a if the identified resource does not exist.
404 Not Found
The query portion of the URI can be used to further refine the request and tailor the response.
5.7.2. POST
The HTTP 1.1 specification defines the POST method as follows
The POST method requests that the target resource process the representation enclosed in the
request according to the resource's own specific semantics
and goes on to say that it is designed to be used for the following types of interaction:
● Providing a block of data, such as the fields entered into an HTML form, to a datahandling
process;
● Posting a message to a bulletin board, newsgroup, mailing list, blog, or similar group of articles;
● Creating a new resource that has yet to be identified by the origin server; and
● Appending data to a resource's existing representation(s).
Following this usage definition, POST will be used in a number of scenarios.
So, if
/csc/insurance/claims/claimEventFolder is modeled as the collection for all Claim Event
Folder resources, then a POST to that URI will create a new Claim Event Folder resource which will then
be a member of the collection “held” by the
.../claimEventFolder resource.
We choose POST as our standard approach to resource creation because with enterprise applications in
general the client cannot specify a unique identifier for the new resource that is under the control of of
the server (e.g. Claim Event Folder IDs are not managed by the client but by the admin system).
If the client were in control of unique identifier allocation, then PUT would be a more appropriate "create"
method (and the "collection" resource would then, in fact, be a "store" resource).
The status code of a successful creation
must
be .
201 Created
The URI of the newly created resource will be returned as the value of the header in the
Location
HTTP response.
This use of POST applies to fine grained control of Operation resources as well as more "standard"
Entity resources.
The response to this type of POST request
is
permitted to use the “
contentlocation response pattern
”.
19
5.7.2.2. POST to a Controller Resource on an Operation resource
When a POST request is sent to a “controller resource” it will have the effect of “executing” that
controller. Controller resources will, primarily, be attached to Operation resources.
The target Operation resource must have been previously created (obviously) and in a state where
executing the desired controller is possible. This could be assessed via standard REST "discovery"
mechanisms e.g. by invoking the OPTIONS method on the Operation resource's URI prior to invoking
POST on the resource's controller URI.
The response to this type of POST request
is not
permitted to use the “
contentlocation response
pattern
”.
Here a POST to a URI of the form
.../funds/4545/operations/surrender/execute
accompanied by an HTTP entity containing the necessary operation arguments, will create, populate,
and then execute a new instance of the
surrender Operation.
There is a slight semantic difference between this way of executing an Operation and the more fine
grained way, in that if creation, population, or execution fails, then the Operation document resource is
NOT persisted (or, in practice, it is deleted). The reason for this is that HTTP requests must either
completely succeed or completely fail. In the case of failure, in this scenario, there is no architecturally
sound way to inform the client "your request failed but you have a new resource hanging around".
The response to this type of POST request
is not
permitted to use the “
contentlocation response
pattern
”.
5.7.3. PUT
PUT is used to add new resources to a “store” resource, or to update the addressed resource. The
semantics of PUT require the entity passed in the request to be a “full representation” of the addressed
resource. That is, all attributes must be present, modified or not.
For that reason it will be rarely used for update purposes, if at all.
If it IS used:
● The intent to create or update can be made clear using, respectively, the
IfNoneMatch
request header (with a value of “*”) and the
IfMatch request header.
● The status code of a successful creation must be
201 Created .
● The URI of a newly created resource will be returned as the value of the
Location header in the
HTTP response.
The response to a PUT request
is
permitted to use the “
contentlocation response pattern
”.
5.7.4. PATCH
The PATCH method is use to perform a "delta update" (i.e. update some attributes) of the resource
referenced by the URI, using the contents of the passed HTTP Entity.
This method is not in the HTTP 1.1 specification, but is gaining more and more traction as the preferred
RESTful "update" method, is specified in another IETF RFC and is registered with the
IANA HTTP
Methods Registry.
As
noted above
, many updates to enterprise system entities can actually have farreaching effects e.g.
a change of marital status could invalidate a certain insurance policy marketed to only single people.
20
Consequently there is some debate as to whether entity modifications should be modeled as "simple"
PATCH HTTP requests, or as operations that are available on an entity. The Entities, Operations and
Inquiries page explains these issues in more detail.
In light of this, the approach we are taking is that SOME attributes of a document resource will be marked
as "writable" in the insurance entity model. These WILL be modifiable using a PATCH method invocation,
but the implementation of that method may, in fact, be analogous to a POST to the execute controller of
the entity's (builtin) update operation. E.g.
PATCH .../contracts/123/funds/4545
may actually be implemented as
POST .../contracts/123/funds/4545/operations/update/execute
This facilitates a fully natural REST semantic in the client while allowing the server to meet the various
administrative requirements of entity updates, in whatever form they are invoked, in a consistent manner.
PATCH method invocations on Operation resources will simply update the specified attributes of the
resource Operation resources do not themselves have Operation subresources!
The response to a PATCH request
is
permitted to use the “
contentlocation response pattern
”.
5.7.5. DELETE
Unsurprisingly, the DELETE method is use to delete the addressed resource.
A hard, absolute delete function is hardly ever implemented in enterprise systems (though it might exist
to delete “workinprogress” resources that need to be abandoned, such as notyetexecuted operation
resources).
However, the semantics of the DELETE method are described more from the point of view of the client’s
intent
the resource should be
“... either physically deleted
or moved to an inaccessible location
”.
In other words, upon successful completion, the resource should no longer be accessible via the URI to
which the DELETE request was sent.
This interpretation allows the serverside implementation to retain the resource, which is often desirable
in an enterprise application, while disabling accessibility via the specified URI. One way to do this is to
mark the resource as deleted and, when subsequent interactions are attempted, on finding that the
deleted “flag” in the resource is set, return a
404 Not Found , or a
410 Gone status (see below). Note
that the resource’s URI should not appear in any lists of resources returned by the server either.
The idempotence of DELETE entails a slightly ambivalent situation with status codes. DELETE should
always return a 200 or
204 status if successful. Consequently a reattempt
should
also return one of these
stati. But attempting any interaction with a URI that does not reference a resource normally returns a
404
Not Found !. To complicate things even further, the status
401 Gone is designed for situations when the
server internally knows that the resource used to exist but no longer does and has no “forwarding
address”.
The solution would appear to be in the client code a “successful” DELETE should be implied from any
2xx code (well, maybe not
202 Accepted the “I’ll do it later” code),
and
a 404 or 410.
5.7.6. OPTIONS
The OPTIONS request is used to obtain metadata relating to the resource referenced by the URI.
The OPTIONS method is the adopted manner that we will implement "HATEOAS discovery" i.e. the
dynamic discovery of metadata and available "next steps" relating to the current contextual resource.
The approach is covered in more detail
below
.
21
5.7.7. The “Content-Location Response Pattern”
Strict, simple REST semantics demand that when a resource is created or updated, the client
should
subsequently make a GET request to obtain the new current representation of that resource.
However, HTTP 1.1 (as revised by the HTTP/2 project) provides a convenient way to signal that the
response to the create/update request contains the new representation, thus avoiding an “extra” GET
request.
This is done using the
ContentLocation
response header,
the description of which
explains the
semantics of the pattern.
Not that only certain forms of creation or update request are eligible for use of this pattern, and that it is
only
applicable to responses that have a 200 OK
or
201 Created status code.
Many of the examples use these concepts with Operation resources to further demonstrate the concept
of modelling operations as document resources.
Where the URI would fold due to page width restrictions, the first couple of segments of the URI have
been replaced with an ellipsis.
GET /csc/insurance/claims/claimEventFolder?customerId=56789
returns the list of claim event folder URIs for the specified customer ID
GET /csc/insurance/claims/claimEventFolder/12345
returns the value of all attributes of CEF 12345
OPTIONS /csc/insurance/claims/claimEventFolder/12345
returns possible operations, inquiries, and other metadata describing CEF 12345 i.e. facilitates
HATEOAS capabilities
GET .../claims/claimEventFolder/12345/operations/validate_cef/6767
returns all the attributes of the 6767 document of the operation type "
validate_cef"
POST /csc/insurance/claims/claimEventFolder
with an appropriately formatted HTTP request entity containing the necessary CEF creation
attribute values, creates a CEF and returns the new instance's URI in the response
Location
header and the ContentLocation header, along with an appropriate representation of the
new CEF resource in the response body and any other headers that would normally be returned
for a GET request for the new resource.
POST .../claims/claimEventFolder/12345/operations/validate_cef
with an appropriately formatted HTTP request entity containing the necessary validate_cef
operation parameter values, creates a
validate_cef operation resource for the CEF identified
by the URI /csc/insurance/claims/claimEventFolder/12345 , and returns the new
operation instance's URI in the response
Location header and the
ContentLocation
header, along with an appropriate representation of the new operation resource in the response
body and any other headers that would normally be returned for a GET request for the new
resource.
22
POST .../claims/claimEventFolder/12345/operations/validate_cef/execute
with an appropriately formatted HTTP request entity containing the necessary
validate_cef
operation parameter values, creates a
validate_cef operation resource for the CEF identified
by the URI /csc/insurance/claims/claimEventFolder/12345 , and attempts to
execute it.
If the creation, population and execution are all successful, the new operation instance's URI is
returned in the response Location header, otherwise the operation is NOT persisted, and an
appropriate error is returned.
Note that the
ContentLocation header and the operation resource
validate_cef
representation are not returned in this case.
PATCH .../claims/claimEventFolder/12345/operations/validate_cef/9090
with an appropriately formatted HTTP request entity, modifies the modifiable attributes (i.e.
arguments) of the (asyet not executed) operation instance, and returns a representation of the
updated resource in the response body, and the ContentLocation header set to the original
request URI.
OPTIONS .../claims/claimEventFolder/12345/operations/validate_cef/9090
discovers what you can do with the operation instance e.g. is it in a state where the DELETE
method can be executed on it, or which, if any, the operation's controllers (e.g. execute, undo,
redo, ...) can be executed.
DELETE .../claims/claimEventFolder/12345/operations/validate_cef/9090
deletes the operation instance, if it is in a state where the DELETE method can be executed on it
(otherwise a
405 Method Not Allowed error will be returned). Normally an operation will not
be DELETEable after it has been executed, but maybe DELETEable before that (to facilitate
abandoning operations before the decision to execute has been confirmed).
POST .../claims/claimEventFolder/12345/operations/validate_cef/9090/execute
(normally with an empty HTTP request entity) runs the operation’s controller, if it is in a
execute
state where this controller can be run (otherwise a
405 Method Not Allowed error will be
returned).
GET .../claims/claimEventFolder/12345/operations/validate_cef/9090
returns the (content negotiated) representation of the current state of the operation so, for
example, the arguments of the operation, whether it has been executed, whether it can be
"undone" (although the OPTIONS method would be the more RESTful way of determining that),
execution outcome (e.g. log) information etc.
23
6. Caching
Representations of RESTful resources can be treated similarly to pages on the web, and, just as with
web pages, the state of resources
may
be
temporarily retained in and served from caches, the
basics of
which are described in the HTTP 1.1 specification
.
Based on the “Cacheable” Rest Constraint, resources can be identified to the client as being cacheable.
Clients can then choose to cache and reuse that data in response to subsequent requests for a
configurable period of time.
Having caches means that we don’t have to hit the server every time for each resource, which is great for
scalability.
Note that caches are not the exclusive responsibility of a REST Client. Resources marked as being
cacheable can reasonably be cached anywhere along the network route between the REST Client and
Server. This allows clients that may not have caching capability to benefit from intermediate caches such
as are found in Content delivery networks, or proxy servers
It is the responsibility of the
server to explicitly communicate the cacheability of a response. It does this
by using the HTTP CacheControl header. For example:
CacheControl: nocache maxage=0 Identifies the resource as noncacheable
CacheControl: maxage=300 Identifies the resource as cacheable for 300
seconds
CacheControl: maxage=300 public The public directive further indicates that the
response MAY be cached by any cache including
intermediate caches
CacheControl: maxage=300 private The
private directive indicates that the
response may only be cached in a userspecific
nonshared cache e.g the browser cache
24
Basically no shared cache in the communication path between an HTTPS client and the server
(technically, the SSL endpoint the serverside point where the HTTPS communication is managed) can
cache an HTTPS response if only because the communication is (normally) encrypted using a privately
negotiated algorithm between that endpoint and the client, rendering it illegible to any other client.
Private caches (those in the browser, for example) will normally be usable, depending on the cache
control header.
Caches behind
the SSL endpoint may be usable, depending on the communication protocol between the
endpoint and the destination server(s), as well as the cache control header. Such caches (often
implemented within “reverse proxy” servers) are easier to control than public caches because they are
part of the same secure environment within which the application servers run. Consequently they are
frequently deployed in order to achieve at least some of the benefits of caching, even in the presence of
HTTPS communications.
The HTTP specification indicates that, when a request contains an
Authorization header, a shared
cache will not serve the response to any other request, except under certain circumstances:
● Where the revalidation directives in the cache control headers associated with the original
response cause the cache to expressly revalidate that response using the current request
authorization data, or
● Were the cached response was explicitly marked public.
Revalidation is a fairly complex mechanism, and, when coupled with the fact that different users may be
served different data from the same resource based upon their authorization, the mechanism should
probably not be relied upon to secure such data.
To avoid confusion or “accidents” a server should do this explicitly for every response it generates! Using
the
private directive will prevent all shared caches from caching the data while still allowing “private”
caches to hold that data, whereas the public directive will render the response cacheable everywhere
outside of an HTTPS environment. Other directives can be added as necessary.
Data that is sensitive to authorization constraints (e.g. business data that comprises the current state of a
resource) should be marked at least private , if not
nocache, whereas nonsensitive data (schemas,
HTML templates, style sheets, scripts, etc.) may be marked public to enable caching in a trusted
shared cache.
Using HTTPS (properly encrypted) between the client and a trusted SSL endpoint in the server’s secure
environment will prevent data from being cached in untrusted public shared caches along the
communication path no matter what the cache control header is set to.
25
Briefly, a server can (in fact should) add two headers to every response which enable conditional
requests:
● ETag: a unique "state ID" for a resource representation, set by the server when returning a
resource representation.
● LastModified : a datetime stamp set by the server indicating when the resource
representation was last modified.
Clients and caches can (and should) use conditional GET or HEAD requests, using the IfNoneMatch
or
IfModifiedSince headers, to determine if the copy of a resource representation they have is still
valid, thus avoiding full regeneration and transmission of a representation in the case that the local copy
is valid.
The server can also use the
Expires header to inform caches of a fixed time after which the cached
response should be considered stale. However, this header is ignored if the
CacheControl header is
present.
Finally, the server
should
add the
Vary header to a response to inform caches which request
headers
had an effect on choosing the format of the current response. So, for example, if the server used the
Accept header to choose the media type of the response, and the
AcceptLanguage header to
choose the language, but used a default charset (e.g. UTF8), then the header should be set as follows:
Vary: Accept, AcceptLanguage
This “suggests” to the cache that the response only applies to requests that have the same values for
these headers as did the request that elicited this response.
26
7. Status Codes
The HTTP protocol specifications contain a large number of status codes that a server can return in
response to a request. This set is extensible, and has its
own public registry
for those deemed to be
globally applicable.
A RESTful API exploits these codes fully: that is, these status codes are the
primary indication of the
status of the request, with any response body representing the secondary indication.
This is different from what has normally been the case with nonRESTful service architectures, such as
SOAP, where the status codes were normally limited to 200 OK and
500 Server Error , and the
content of the response body was used to determine if the request really was successful or not at an
application level.
Some authors on REST prescribe limiting the set of status codes used by an application to a small
subset of those in the HTTP specification. However, we take the view that the set of codes is already
constrained by that specification, with each code having a clear and precise use, and, therefore, it is
unnecessary to encumber the API with artificially constrained subsets.
The only admonition is that the specified HTTP status codes be used only for situations they precisely
describe.
That said, there
is
a subset that is more frequently used than others and it is worthwhile covering their
semantics here rather than always pointing the reader to the HTTP specification itself:
200 OK The request (
any request with the exception of a resource creation request) was
successful
and the response contains a body entity , the format of which is
conveyed by the media type given in the ContentType header.
The URI of the primary created resource must be returned in the Location
header. So, for example, if the associated request was for the creation of a quote
resource, but as part of that some subordinate and linked resources were also
created, it is the URI of the new quote resource that must be returned, not those
of the subordinate and linked resources (which should be available from the new
quote resource).
Additionally, the response body
may contain a representation of the new primary
resource (the quote resource in the above example). If this is the case the
ContentLocation header
must be set to the same value as the Location
header, and any other headers that would normally be returned for a GET request
for the new primary resource
must also be set.
202 Accepted The request has been accepted and will be processed “later”.
204 No Content The request (
any
request) was successful
and the response does not contain a
27
body entity
.
All successful requests resulting in an empty body should return a 204 status,
with the exception of a resource creation request (see 201 above), and a
conditional GET for which the client already has the latest representation (see
304 below).
304 Not Modified A conditional GET discovered that the resource representation has not changed
since last retrieved by the client.
This response is, by definition,
without
an entity the object being to avoid the
overhead associated with generating, transmitting, and parsing a new
representation when the client already has an uptodate representation.
400 Bad Request The request could not be understood by the server.
Use this to indicate a badlyformed request but for reasons
other
than those for
which more precise 4xx status codes are provided.
The response body should indicate more precisely what the problem is.
401 Unauthorized The current user credentials (if any) do not permit the request to be fulfilled.
The response body should not be too precise about this error for the same
reason systems rarely inform a user of a failed login which piece of information is
incorrect!
403 Forbidden The request will
never
be fulfilled modified user credentials will not make a
difference.
Disabled directory listing in Apache is one example where this code is returned.
However, the specification indicates that if the server does not ‘wish’ to indicate
why
the request is forbidden (with a response body) it should, instead, use 404
Not Found
404 Not Found Technically “The server has not found anything matching the RequestURI”.
This is the standard “resource not found” status noting the slight semantic
difference of 410 Gone (see below).
405 Method Not The resource does not support the request HTTP method.
Allowed
The response must contain a list of the allowable methods in the
Allow header
(the same header used by the OPTIONS method to convey the same information).
406 Not The request could have been fulfilled, but the client sent one or more
Accept*
Acceptable headers that specified response characteristics that the resource does not
support (e.g. asking for a PDF representation of a movie resource, perhaps).
Note, however, that servers are not required
to use this code they are permitted
to return a default representation type instead.
Note, also, that if this response
is
used then the server should return the list of
“acceptable” characteristics in the response entity
. I.e. there is no specified HTTP
28
header to convey this information.
(408 Request This
isn’t
common, but it is included to stress the point that this is
NOT
a timeout
Timeout) in the server! This indicates simply that a multipacket request did not completely
arrive at the server within the server’s configured timeout window a highly
technical problem (or DDOS avoidance).
Again : Do NOT use this to indicate a situation where the server is processing the
request but it is taking longer than some configured or requested time to complete
that is one of the possible use cases of the 202 Accepted response.
409 Conflict A nonsafe request was attempted on a resource, but a conflict between the
request and the current state of that resource exists.
The user is expected to be able to fix this in some way and resubmit a modified
request.
There is a subtle difference between this code and 412 Precondition
Failed . On the face of it, this code is appropriate when conflicts are found in
update requests that exploit condition headers ( If* headers ). However, since
the
412 status more precisely identifies that situation, is used where other
409
types of conflict occur. E.g:
● Where no locking or versioning scheme is implemented (probably
indicating that the resource is not expected to be in conflict)
● Where the client does not provide required conditional headers to manage
locking/versioning (400 Bad Request is technically more accurate, but
409 is more helpful, perhaps)
● When a resource creation is attempted but the resource already exists. For
example, where PUT is used to create
and update resources (as in store
resource contents), a PUT without
conditional headers might be interpreted
as a create request. If the resource addressed by such a request already
exists, then 409 is, indeed, the appropriate status code to return.
410 Gone A more precise version of 404 where the server is aware that the resource used
to exist (at the request URI) but no longer does and has no forwarding address.
May
be appropriate in the “virtual delete” functionality described for the DELETE
method, above.
412 Precondition A conditional request (one which specifies
conditional headers
) was made and
Failed the conditions were not met.
The conditional headers provide a flexible system of ensuring that requests are
only fulfilled if the state of the addressed resource meets certain conditions. They
are particularly useful in preventing race conditions, where the client might
attempt to update a resource based upon a stale understanding of the state of
that resource.
412 is not always the appropriate status to a failed condition a GET using the
IfModifiedSince header should not receive a status if the condition
412
fails, but, instead, a status (see above).
304
500 Internal Something out of the client’s control went wrong while processing the request.
Server Error
29
This is normally the appropriate status for any serverrelated problems, with really
only two common exceptions (see the following entries).
In general, stack traces should
not
be returned to the client in the response body.
501 Not The server either did not recognize the request method (as opposed to the
Implemented resource not supporting
the request method 405), or lacks some other
functionality necessary to fulfill the request.
503 Service DDOS attack!
Unavailable
Well, not necessarily, but that’s one example of a situation where this status might
be appropriate: The server is currently unable to handle the request due to
temporary overloading, or server maintenance.
The implication is that this is a temporary condition which will be alleviated after
some delay. If known, the length of the delay can be indicated in a RetryAfter
header.
30
8. Hypermedia
The book
Building Hypermedia APIs with HTML5 and Node.js
begins with a succinct introduction to the
concept of “hypermedia”:
The World Wide Web is driven by hypermedia: the ability of a document to describe its possible
states, and its relationship to other documents. Hypermedia is not just a way of making websites
that average people can use; it’s a new style for distributed computing, powerful and flexible.
APIs that exploit this architectural style are known as Hypermedia APIs, and the
“HATEOAS” constraint
on REST basically requires that REST APIs are Hypermedia APIs.
In order to successfully (and dynamically) navigate a REST API, the client needs to know what it can do
with the current (or context) resource. There are two aspects to this:
● The application
state can be modified (i.e. the client can move from the current resource to a
different resource)
● The context resource state can be interpreted, manipulated, and modified
A Hypermedia application advertises both types of state modification using URIs i.e. both types of
transition boil down to following URIs that are available from the context resource.
How the context resource presents these links is important to the discoverability of the API, and there are
many ways to do this, some enshrined in real standards, others gathering support without a real
standardisation effort.
Some of the better supported proposals are:
● Hypermedia JSON (hmjson)
● Hypermedia Application Language (HAL)
● Siren
● JSONLD
● JSON Hyper Schema
● The Link HTTP header
There is a related issue concerning how a client that (supposedly) has few preconceptions concerning
the resource it is currently dealing with, actually finds out what is
in
that resource in the first place what
“type” of thing it has in front of it.
This issue can be broken down into two (albeit interconnected) aspects:
● What is the basic syntax and structure of the resource representation?
● What is the “thing” that the representation represents!
The first aspect is a question of
representation
of the resource.
● Is it XML or JSON ... or an MPEG4 movie (the client must know how to parse it!)?
● Does it have a specific overall structure to it (e.g. is it a HAL document, or a Siren document)?
The client can negotiate with the server concerning what representation formats it accepts and what
formats the server supports. This is
content negotiation
and is a basic part of all industrial strength HTTP
clients.
The second aspect is more businessoriented does the resource represent a “Person”, a “Contract”, a
“Surrender” operation etc. This is what is normally referred to as the “schema”.
31
In general, the first aspect allows a client to discover what the application state transitions
from
this
resource are, while the second allows the client to interpret, use and modify the state of the resource
itself.
Hypermedia representation is most closely related to how the links presented by a resource can be
discovered. It is important to lay down to representing these hypermedia links so that client’s can indeed
easily discover the functionality exposed by an API.
While the above proposals all differ in some respects, a few general principles are followed by most:
● The links
should be discoverable in a special object associated with the representation. Most
prescribe that the object is in the representation itself (e.g. in the JSON of the GET response
body), while some prescribe that it be part of the business schema (wholly or partly) or even in a
standard response header (the Link header)
● Each link
MUST have a URI (obviously) and SHOULD have an RFC 5988 “rel” property that
describes its relationship to the context resource.
The “rel” property of a link is the key to being able to navigate through the application “state
machine”. By searching the hypermedia for a specific relationship (e.g. the next relationship
when paging through a list of resources), a client need know very little about the semantics of the
application behind the API.
● The interactions available with the URI may be either “described” in the link object or described by
calling OPTIONS method on the link’s URI basically the proposals sit in one camp or the other.
● Use the
HAL structure to present application
state changes, for a number of reasons:
○ It has a good base of support
○ It presents an intuitive structure that a “slightly less generic” client can easily traverse
○ It supports both JSON and XML (practically all the others only describe for JSON)
○ It has registered MIME Types (the ContentType and Accept header values used in
content negotiation) for both JSON and XML.
○ By exposing links in the representation itself, those links can be dynamically tailored by the
server based upon requester context (e.g. security credentials). This is harder to achieve
when links are exposed only
via a schema.
○ By not
exposing interaction information (i.e. resource state change information) in it’s link
structures, it implements the “purest” approach to the principle that a resource should only
be responsible for managing its own API specifically, it is not a given resource’s
responsibility to expose details of the APIs of the resources to which it is linked (see the
section on Resources ).
○ The HAL format includes a “ hypertext cache pattern ” mechanism that might help alleviate
the constraints on resource cacheability imposed by the use of HTTPS and
authenticated
connections .
● All implementations MUST
support the type
application/vnd.hal+json , and that should be
the default
representation (i.e. used when no other requested representation can be supported,
rather than returning an HTTP 406 Not Acceptable error). It should also be the representation
returned for the generic JSON media type application/json .
32
● In addition, if the implementation supports XML resource representations, then it MUST support
the type application/vnd.hal+xml and this should be the representation returned for the
generic XML media types text/xml and
application/xml .
The location of this schema should be advertized in the
Link response header, with a value
rel
of “ ” (as per
type RFC 6903 Additional Link Relation Types
).
However, clients are not obliged to refer to this schema the very fact that a client has navigated
to a particular resource based upon the server’s publication of hypermedia should be enough
assurance for a client that the resource it has is of the type that the API documentation for that
resource indicates.
● The OPTIONS HTTP method will be used to obtain the permissible interactions for each resource
(the
resource
state manipulation links).
That is, once a client has located a URI with which it needs to interact, it should first call the
OPTIONS method for that URI. This will return a standard document that describes the allowable
interactions with a URI, including input arguments.
This is the recommended approach because the legal interactions published to the client can be
dynamically tailored by the server based upon requester context (e.g. security credentials). This is
more difficult to achieve when the interactions are exposed only via a schema.
This leads to the following (highly generic) actions in a client:
● Response from a previous request is received becomes the context resource
● Check that the content type indicates a HAL JSON document ( ContentType response header)
● Find the
_links object, then the desired URI (keyed by relationship so, perhaps, a URI to an
address list named addressList )
● Invoke the OPTIONS method on the URI, requesting the standard options media type
representation (
Accept request header)
● Interrogate the returned document based upon the standard options media type .
● Construct a request to the desired URI based upon the parsed content of the OPTIONS
response.
The more generic the client, the more dynamic this discovery will be for clients that need not be so
generic, for example, an assumption that the OPTIONS invocation returns a specific format may be
embedded.
These links are distinguished by the relationship they have with the current (context) resource. In HAL
these relationships are “named” using the standards laid down by the Web Linking RFC . That RFC
established a registry of standard relation types, the use of which enables clients to dynamically discover
and navigate through the legal states of an application.
Relation types are nothing less than what turns Hypermedia into HATEOAS!
33
Relation types really fall into two groups
● Structural types that describe the navigation of the overall application structure
● Business types that describe business relationships between resources and allow the business
semantics of a system to navigated and exploited.
With these types, in order to ensure coherent navigation of an API, strict usage guidelines are to be
followed. To that end the relation types listed below should be considered as reserved for the
documented purpose across all CSC API implementations. An Implementation is free to exploit others as
befits the semantics of that implementation's particular use case.
self The "canonical" URI of the resource that which is considered to be it's true identifying
URI. Normally this is the request URI (minus any query parameters).
Should be in the
Link header
and
the HAL
_links hypermedia envelope.
up The "previous hierarchy level of the URI of the contextual resource" i.e. “ ”
cd ..
applied to contextual REST resource hierarchy.
Should be in the
Link header
and the HAL _links hypermedia envelope.
typet
The URI to the (JSON) Schema resource that describes the business content of a
resource i.e. the “classic” meaning of “schema”.
Should be in the Link header
and the HAL _links hypermedia envelope.
Note
the URI here acts as a type name as well as a schema locator
content A URI to the (JSON) Schema resource that describes a resource that can legally be an
typetx
item of the current Collection resource.
Only applicable when the current resource is a Collection resource.
Should be in the header
Link and
the HAL hypermedia envelope.
_links
There can be multiple contenttype entries
first Used for paging through the “content” links of a collection or store resource. The use of
last these is detailed in
the section on paging, above
.
next Should be in the Link header and
the HAL
_links hypermedia envelope.
prev
item The links to the content resources of a “container” resource (Collection or Store).
MUST be in ( only) the HAL
_links hypermedia envelope for a container resource.
MUST be a list (array) empty or not named “item ” (singular) not “ ” (plural).*
items
MUST NOT be used in the hypermedia of any other resource archetype.
* with HAL, the relation type name is provided as the property key in the _links object so the
item
array, here, should be read "here is a list of links that all have the same relationship to the context
resource, that of
item ".
t
type and
contenttype URIs act as both canonical type names and schema locators
x
contenttype is not a registered relation type. The final value to use has yet to be determined.
34
8.2.2. Use of Business Relation Types
In general business types are domainspecific and follow the “custom type” standards laid down in the
Web Linking RFC . Some naming norms will be laid down to ensure that a coherent approach is taken
when naming such types.
TO BE COMPLETED
As mentioned before, this metadata is retrieved using the OPTIONS HTTP method. The metadata must
provide all the information necessary for a client to build a correct request to interact with the target
resource of the link. It really is not very important which metadata representation is chosen as long as the
chosen format meets this requirement.
There are a number of hypermediaenabled schema languages that can fulfil this role, two of the most
prominent being JSON Hyperschema and
Hydra. Since JSON Hyperschema is an extension of JSON
Schema , the schema language with which our resource metadata will be published, the chosen format
for the OPTIONS response body is JSON Hyperschema.
There follows three examples of an OPTIONS response body formatted as JSON Hyperschema.
● Retrieval interactions do not identify a method because the default method is GET.
● The use of the sorting and paging query parameters described earlier
.
● The list of recommended “rel” values to use within an OPTIONS response (i.e. the label of each
interaction) has not yet been decided upon, but it will be derived from
the general approach to
relation types adopted by the API program .
● The response should be dynamically generated specifically for the target resource and based
upon the context of the request (e.g. the authorizations associated with the authentication
characteristics of the request, current resource state etc.).
● The presence of a type
property in the create interaction object this is explained in more detail
below .
● The create interaction prescribes only one required field create actions do not necessarily have
create businessready resources, only technically valid resources i.e. resources that are then
directly addressable for further modification.
35
{
"title": "Persons collection Interactions",
"links": [
{
"rel": "contents",
"title": "Fetch persons collection content",
"href": "http://example.api.com/persons"
},{
"rel": "search",
"title": "Search for a person by name pattern",
"href": "http://example.api.com/persons",
"schema": {
"type": "object",
"properties": {
"namePattern": { "type": "string" },
"_count": { "type": "integer", "minimum": 100},
"_start": { "type": "integer", "minimum": 1},
"_num": { "type": "integer", "minimum": 1},
"_sort": { "type": "string"}
},
"required": ["namePattern"]
}
},{
"rel": "create",
"title": "Create a person",
"type": "http://example.api.com/schema/Person",
"href": "http://example.api.com/persons",
"method": "POST",
"schema": {
"type": "object",
"properties": {
"uniqueId": { "type": "string" },
"name": { "type": "string" },
"birthDate": {
"type": "string", "format": "datetime"
},
"birthPlace": { "type": "string" },
"primaryLanguage": { "type": "string" },
"residenceCountry": {
"type": "string", "maxLength": 3
},
"dateOfDeath": {
"type": "string", "format": "datetime"
},
"gender": {
"type": "string", "enum": ["MALE", "FEMALE"]
}
},
"required": ["name", "uniqueId"]
36
}
}
]
}
{
"title": "Person Resource Interactions",
"links": [
{
"rel": "read",
"title": "Fetch the person detail",
"href": "http://example.api.com/persons/12345"
},
{
"rel": "delete",
"title": "Delete the person",
"href": "http://example.api.com/persons/12345",
"method": "DELETE"
},
{
"rel": "update",
"title": "Modify the person",
"href": "http://example.api.com/persons/12345",
"method": "PATCH",
"schema": {
"type": "object",
"properties": {
"name": { "type": "string" },
"birthDate": {
"type": "string", "format": "datetime"
},
"birthPlace": { "type": "string" },
"primaryLanguage": { "type": "string" },
"residenceCountry": {
"type": "string", "maxLength": 3
},
"gender": {
"type": "string", "enum": ["MALE", "FEMALE"]
}
}
}
}
]
}
37
Note the following characteristics:
For example, the collection represented by the URI /csc/insurance/policies/12345/coverages
is unlikely to contain multiple coverages of the
same type! They will be different, with different data
requirements. Thus the ability to create an item resource of a type chosen from among the legal
types for
the Collection (which are identified in the
Link response header as described above) is required.
To facilitate this, the OPTIONS response for a Collection resource will include a separate
create
interaction object for each
possible item creation interaction, in the following format:
...
,{
"rel": "create",
"type": <type URI of new item type>
"title": <helpful title for this creation>,
"href": <URI of collection with optional URI query parameters to
identify the desired type to create>,
"method": "POST",
"schema": {
<JSON Schema of data required in the create request>
}
},
...
The
type property allows the
client
to identify the correct interaction “instructions” for the item it wishes to
create. As already discussed, the OPTIONS response is to be returned in JSON Hyperschema format
and JSON Hyperschema does not
prescribe a type property in its link object. In JSON Schema
semantics, however, properties other than those described in a schema may appear in objects. Thus the
addition of the
type
property here is technically valid and can be construed as a CSC extension to JSON
Hyperschema.
The optional query parameters attached to the collection URI in the href property value allows the
server
to easily identify which type of item it is being asked to create. This extra information passed in the
href
URI is "resource constraint" information pertaining to the Collection resource acting as an item factory,
not information destined for the resource instance that is being created (analogous to passing the name
of the class of a desired new instance to a factory method). Consequently the URI is deemed the correct
place for such information, rather than the body of the request.
As an example, then, consider a coverages collection for a policy resource
/policies/12345/coverages . The collection is empty at the moment, but the product associated
38
with the policy specifies that it can have a
death
coverage (which can be either
natural
or
accidental
)
and a disability
coverage.
An OPTIONS request to http://example.api.com/policies/12345/coverages
might return the following
interaction entries in the response:
39
...
,{
"rel": "create",
"type": "http://example.api.com/schemas/insurance/coverages/Death"
"title": "Create a natural death coverage for this policy",
"href":
"http://example.api.com/policies/12345/coverages
?type=http%3A%2F%2Fexample.
api.com%2Fschemas%2Finsurance%2Fcoverages%2FDeath&subtype=natural
",
"method": "POST",
"schema": {
<JSON Schema of data required to create a "natural death" coverage>
}
},{
"rel": "create",
"type": "http://csc.insurance.com/schemas/insurance/coverages/Death"
"title": "Create an accidental death coverage for this policy",
"href":
"http://example.api.com/policies/12345/coverages
?type=http%3A%2F%2Fexample.
api.com%2Fschemas%2Finsurance%2Fcoverages%2FDeath&subtype=accidental
",
"method": "POST",
"schema": {
<JSON Schema of data required to create an "accidental death"
coverage>
}
},{
"rel": "create",
"type": "http://example.api.com/schemas/insurance/coverages/Disability"
"title": "Create a disability coverage for this policy",
"href":
"http://example.api.com/policies/12345/coverages
?type=http%3A%2F%2Fexample.
api.com%2Fschemas%2Finsurance%2Fcoverages%2FDisability
",
"method": "POST",
"schema": {
<JSON Schema of data required to create a "disability" coverage>
}
},
...
It is important to note that the format of the extra type information passed in the create request URI is not
being prescribed here the href
URI is “dark” to the client and should be followed “blindly” to create the
item of the desired type. Additionally note that the type information has been URL Encoded, as required
by the HTTP standards.
Finally to highlight the dynamic nature of OPTIONS responses, let us now assume that this policy can
have up to two coverages one of the death coverages and/or the disability coverage. Let us also
assume that the user has already chosen to create an accidental death coverage.
Technically
, to be as responsive and generic as possible, the client application
should
refetch the
coverages Collection resource (which will now contain the link to this new accidental death coverage)
and
reexecute an OPTIONS request on the coverages Collection resource.
40
If this is done, the OPTIONS response should now contain only one create action entry (plus other
interaction details as necessary) that pertaining to the disability coverage.
Whatever the outcome of that decision, API implementations should return this format as the default
format (i.e. used when no other requested representation can be supported, rather than returning an
HTTP 406 Not Acceptable error) and in response to the generic JSON media type
application/json .
For now, clients should specify in the
application/json request header and assume a
Accept
JSON Hyperschema response.
● The technical interaction mechanism
● The behavioral contract
HATEOAS only provides for the first of these. There is still a need to properly document an API in such a
way as to make clear the behavioral aspects, constraints and limitations of each aspect of the API. This
even more for webscale APIs where the client base is usually vastly bigger than for proprietary or private
APIs.
There exist tools for building APIs
and
documentation at the same time, two of which are Swagger and
Mulesoft API Designer . These (and others) are being considered as our tool of preference for APi
development.
9. Headers
It is probably fairly obvious by now that a lot of the protocol between client and server relies upon the
presence or absence of HTTP headers that accompany a request of a response. There are many places
in this document that mention specific headers for a specific feature. Here we try to summarize the
headers that should accompany the request or the response in an application that uses these REST
practices.
Note that this is not an exhaustive review of standard HTTP headers. For example we do not cover
authentication/authorization, or “low level” control headers (such as , or
Host , or transportlevel
Agent
negotiation). The former will be covered by a separate document, the latter are usually handled by lower
levels of the HTTP frameworks used by most applications.
41
9.1.1. Content-Encoding
list of encoding names in the order they were applied
ContentEncoding: < >
This header informs the recipient of any content encodings basically the compression (and possibly
encryption) routines that the sender has applied to the entity.
This is “application layer” encoding so the application layer of the recipient is responsible for
decoding
such an encoded entity. “Transport layer” encoding is not in the scope of our applications.
A client can indicate the encodings it supports for a response using the
AcceptEncoding request header
.
In general, in our case, there is probably little reason to encode message entities at this level.
9.1.2. Content-Language
language of the message entity
ContentLanguage: < >
This header indicates the language in which a message entity is presented. The value is known as a
“language tag” and should conform to the
wellestablished standards for these tokens
.
Technically it is possible to indicate that a message is in
multiple
languages by presenting a list of
language tags.
A client can indicate language preferences for a response using the
AcceptLanguage request header
.
It is recommended that
responses
should
always
have this header, whereas
requests
rarely need this
header.
9.1.3. Content-Length
length of message entity in bytes
ContentLength: < >
This header simply tells the recipient the length of the message entity after any content encodings have
been applied to it (but before any transport encodings have been applied). It is the length of the entity as
received by the application layer in the recipient.
Note that it is
possible
to not specify this header recipients have a variety of ways of figuring out the
length of an entity but the strong recommendation is that, where it is possible to provide the length of
the message entity before the entity is serialized to the communication stream, this header should be
provided.
On any request that
could
contain an entity (so PUT, POST and PATCH requests), this header should be
sent, even if it’s value is zero (indicating that no entity is, in fact, present).
On
requests for which an entity makes no sense (so, GET, HEAD, DELETE, OPTIONS, ...) the header
should not be provided.
On responses that contain an entity the header should be provided.
9.1.4. Content-Type
media type of the message entity
ContentType: < >
This header tells the recipient the basic media type of the entity. If the entity is a textual type, it will also
indicate the character set encoding of the entity.
42
So, for example, a server indicates that a response is a UTF8 encoding of a HAL JSON resource
representation, using the following setting.
ContentType: application/vnd.hal+json; charset=utf8
Whereas a client indicates that a request contains the input arguments for a POST request in JSON
format, encoded as ISO88591 using the following setting.
ContentType: application/json; charset=iso88591
Obviously, this header must be present on a message that contains an entity, and should always
specifically indicate the character encoding when using a textual media type. The header should not be
present on a message that does not contain an entity.same here
A client can indicate its format and character encoding preferences for a response using the
Accept
and
AcceptCharset request headers.
9.2.1. Accept
Accept: application/vnd.hal+json,application/json;q=0.5, */*
Accept: application/vnd.hal+xml,application/xml;q=0.5, */*
This is a
Content negotiation header
. This header lists the client’s media type preferences for response
data (including preference weightings).
Technically, the absence of the header means that the client has no preference, and the server is free to
either present responses in a different default format if it supports none of those specified on the header
or
to reject the request with a
406 Not Acceptable status code. Preferences are marked using a
weighting value followed by the order in which the preferences are listed.
However, note (from before) that the server should:
● Return resource representations as
application/vnd.hal+json or
application/vnd.hal+xml
● Return all other responses as
application/json or
application/xml
In general, in response to a (successful) GET request, the server will return a HAL JSON representation
of a resource by default
(i.e, if the header is missing or does not contain a media format that it
Accept
supports).
However, a successful POSTcreate, PUT or PATCH request can return the created/updated resource
representation
or
a “status report”.
In
these
cases the
Accept header is the way the client expresses it’s preference for one or the other.
The JSON example above requests the representation of the created/updated resource, whereas
removing the
application/vnd.hal+json declaration, or adjusting the weightings to prefer
application/json , would request the status report as a preference.
A failed POSTcreate, PUT or PATCH request will
only
return a “status report” (if anything), and the
default format will be .
application/json
The server indicates the chosen format in the response header
ContentType .
43
9.2.2. Accept-Charset
AcceptCharset: utf8
AcceptCharset: *
This is a
Content negotiation header
. This header lists the client’s character set preferences for textual
responses.
As with the
Accept header, multiple weighted preferences could be provided, and the server is free to
impose a default if it does not support any of the specifications, or to reject the request with a
406 Not
Acceptable status code.
However the UTF8 encoding is now universally supported and and can all (known) character sets so the
very strong recommendation is to always only specify utf8, “” or leave the header unset (which
*
default’s to “”), and for the server to
* only
serve (text) response in UTF8 encoding.
The server indicates the chosen character set of a textual entity in the response header
ContentType ,
using the
charset media type parameter.
9.2.3. Accept-Encoding
AcceptEncoding: gzip
AcceptEncoding: *
This is a
Content negotiation header . This lists the client’s supported “encoding” preferences for
responses basically, the compression (and possibly encryption) routines that the server can apply to the
response. This is “application layer” encoding so the application layer of the recipient would be
responsible for
decoding such an encoded message. Transport layer encoding negotiation is not in the
scope of this discussion.
Standard supported values are identity ,
gzip,
deflate and
compress , where means
identity
“do not encode the response” (i.e. leave it in the raw form of its declared media type).
Beware that
not
specifying this technically means that the server is free to apply any encoding or no
encoding.
The server indicates the chosen encoding
s
, in the order that they were applied, in the response
ContentEncoding header
.
In general, in our case, there is little reason to use such applicationlevel encoding.
9.2.4. Accept-Language
AcceptLanguage: da, engb;q=0.8, en;q=0.7
This is a
Content negotiation header . This lists the client’s language preferences for responses, and is
the standard way that a client indicates to the server in what language textual responses should be
presented. The value of each entry is known as a “language tag” and should conform to the
wellestablished standards for these tokens .
As with the header, multiple weighted preferences may be provided, so the above example
Accept
means
”I prefer Danish, but will accept English, in which case I prefer British English”
44
As with the other content negotiation headers, the absence of the header means that the client has no
preference, and the server is free to either present responses in a different default language if it supports
none of the specified languages or to reject the request with a
406 Not Acceptable status code.
The server indicates the chosen language in the response header
ContentLanguage .
This header should most certainly be used in multilingual deployments.
9.2.5. Cache-Control
The CacheControl header in a
request
can be used to finely direct caches in the request path as to
the characteristics of a possiblycached response that they are willing to accept.
The full set of directives that can be used are explained in the
HTTP cache specification
.
While highly sophisticated general clients (such as browsers) may use this request header, in our clients
it is unlikely that it is worth the complexity involved in managing caching at the this level to make the use
of this header worthwhile.
9.2.6. If-Match
IfMatch:
<entity tag from previouslyfetch representation>
This is a “precondition” header that directs the server to process the request only if the
current
entity tag
for the “selected representation” of the target resource for the request matches the entity tag that is the
value of the header. If the request cannot be processed (because the tags do not match) then the server
returns a status of
412 Precondition Failed .
This is most commonly used to ensure that an update request (POST/PUT/PATCH/DELETE) is targeting
the state of a resource that the client expects. It relies upon the serverside entity tag being changed if
another agent has updated the resource between when the requesting client obtained the representation
and when it attempts the update.
The value can be set to “*” which effectively directs the server to process the request only if the “any
representation” of the target resource exists basically, if the resource exists. This can be used to
prevent a PUT request creating a resource when the client is simply expecting it to update an existing
resource.
Entity tags must be treated as completely opaque by the client.
If the original representation was returned with a “strong”
ETag header (that is one whose value does
not
begin with “w/ ”), then any associated update request
should
use the
IfMatch header to manage the
update
See the ETag response header for more information on the construction and use of entity tags by the
server, and for why the client doesn’t really need to worry about the semantics of the phrase “selected
representation”!
9.2.7. If-None-Match
IfNoneMatch:
<LIST of entity tags from previouslyfetched
representations>
This is a “precondition” header that directs the server to process the request only if none of the presented
entity tags matches the current
entity tag for the “selected representation” of the target resource. If the
request will not be processed (because a matching tag is present) then the server returns a status of 304
Not Modified .
45
This is most commonly used on a GET request to cause a client or cached copy of a resource
representation to be refetched only if it is out of date.
However, the value can be set to “*” to direct the server to process the request only if “no representation”
of the target resource currently exists basically, if the resource does not exist. This can be used to
prevent a PUT request inadvertently updating a resource when the client is simply expecting it to
create
the resource. In this case, if the request fails because a matching tag is present, the server returns a
status of
412 Precondition Failed .
Entity tags must be treated as completely opaque by the client.
If the original representation was returned with any ETag header value (“weak” or “strong”), then any
attempt to refetch the resource should use the
IfNoneMatch header to avoid the server needlessly
regenerating the resource representation.
9.2.8. If-Modified-Since
IfModifiedSince:
<HTTP Date>
This is a “precondition” header that directs the server to process the request only if the
LastModified
header value on the “selected representation” of the target resource later than the specified date. In
practical terms this simply means that the request will processed if the state of the target resource has
changed since the specified date.
If the request will not be processed (because the resource state has not changed) then the server returns
a status of
304 Not Modified .
This is most commonly used on a GET request to cause a client or cached copy of a resource
representation to be refetched only if it is out of date.
The value is normally provided by the LastModified header on a previous response to a previous
retrieval of the resource representation, but
can
technically be any date that the client desires .
If the original representation was returned with a
LastModified header then any attempt to refetch
the resource should use the
IfModifiedSince header to avoid the server needlessly regenerating
the resource representation, but only if the
IfNoneMatch header is
not used.
9.2.9. If-Unmodified-Since
IfUnmodifiedSince:
<HTTP Date>
This is a “precondition” header that directs the server to process the request only if the
LastModified
header value on the “selected representation” of the target resource is not later than the specified date.
In practical terms this simply means that the request will processed if the state of the target resource has
not changed since the specified date.
If the request will not be processed (because the resource state has changed) then the server returns a
status of
412 Precondition Failed .
This header has a similar update control purpose to the
IfMatch header. However, the use of
entitytags provides a much more accurate control of such updates and so should be used in preference
to the
IfUnmodifiedSince header if possible.
For many of them, their usage is explained in more detail in other parts of this document. Those that are
will be listed here with links to those more extensive explanations.
9.3.1. Allow
Allow: GET, PATCH, POST
This header is used to indicate which HTTP methods a resource supports. It is used in two
circumstances:
● When the client has attempted to use an unsupported method, resulting in a status code
405
Method Not Allowed
● As part of the response to the
OPTIONS request
The value should reflect the current state of the resource so, for example, if a resource normally
supports the DELETE method but in it’s current state does not, the DELETE method should not be part
of the header value.
These headers should be used by servers in the manner described in order to profit from the caching
mechanisms that may exist between the client and the server.
9.3.3. Content-Location
ContentLocation:
<Location header URI, or original request URI>
The main purpose of this response header is to indicate to the client that the entity included with a
response to a creation of update request is the new/updated representation of a newly created or
updated resource. In the event that this is the case, a subsequent GET request is not required to obtain
this updated state representation.
● If the representation of a new resource is returned in response to a
creation
request (e.g. a POST
to a collection resource, or a PUTcreate to a store resource), the
ContentLocation header
must be set to the same value as the
Location header.
● If the updated representation of resource is returned in response to an update request (e.g. a
PATCH or PUTupdate request) the ContentLocation header must
be set to the same value
as the request URI (without query parameters).
Note that when this pattern is used, to render it as effective as possible, the response should
also
contain
all
the response headers that would normally accompany the response to a GET request for the
resource.
The absence of this header is taken to indicate that the response contains a status report of the
processing of the associated request. In this case the client must perform a subsequent GET request for
the resource to obtain a representation of it’s updated state.
The client can request that this pattern be used via the
Accept request header
. If the client so wishes
then
all
the
content negotiation request headers that would accompany a GET request of the resource
must accompany the creation/update request as well
47
9.3.4. ETag
ETag:
<clientopaque ‘entity tag’ string with optional “strength” prefix>
This header is used in responses that contain resource representations (whether for retrieval or
“contentlocation pattern”
creation/update requests) to provide a “version” tag for the representation.
The tag value can then be used in subsequent conditional requests associated with the representation.
If at all possible, resources should support the generation and use of “strong” entity tags. Note that the
value of this response header, if used,
must always be generated it must
never
simply be copied from a
precondition header that accompanied the associated request.
Entity tags are used for two purposes:
● Cache control
(i.e.
IfNoneMatch conditional requests)
● Update control (or optimistic locking) (i.e.
IfMatch conditional requests)
Entity tags can be weak or strong:
● Strong tags
must be changed by the server if the byte content of the associated representation
changes in any way at all (e.g. even if an extra space is added between a JSON object property
name and it’s associated value).
They can be used for cache control and update control.
● Weak tags are changed by the server only when it considers that the semantic meaning conveyed
by a representation has changed in an applicationsignificant manner.
They are denoted with a prefix of “
w/” and can only be used in caching operations.
However, various implicit constraints and facts concerning resources and the HTTP protocol reduce the
burden considerably:
● The server will normally generate each supported representation identically for a given resource
in a given state (it is highly unlikely to add spurious whitespace in the manner described before).
● There are only three significant response headers that normally
vary between representations of
a resource in a given state ContentEncoding ,
ContentLanguage and
ContentType
the contentnegotiation response headers.
● An entity tag must be treated as completely opaque by the client
● But the tag value can quite legitimately be a highly structured parseable field from the server’s
point of view!
48
Consequently a legitimate strong entity tag can be built according to a strict
meaningful
pattern, such as
concatenating the values of the three significant headers in a response with a some kind of resource
specific version id (such as an update counter or a finegrained time stamp).
The server is free to obfuscate this value in anyway that it sees fit as long as the obfuscation algorithm
is symetrical.
Adding the “wrinkle” that representations
may be different depending on the request authorization context
merely adds the requirement that an extra field denoting this context be added to the value, or, perhaps,
used as the obfuscation “key” of the value.
For example:
ETag:
12345~agent~application/vnd.hal+json;charset=utf8~fr~gzip
is a legitimate strong value for the French HAL JSON representation of a resource whose update counter
currently has the value 12345, using the UTF8 character set, compressed using the gzip algorithm, and
generated for a user with the role of Agent (here shown in cleartext for clarity).
The server is free to obfuscate this in anyway that it sees fit (and that is symmetric and repeatable).
The only “burden” for the server is to implement some kind of resource version ID for each resource the
content negotiation header values take care of refining that version for each specific representation of
that resource.
● If necessary deobfuscate the If* request header value
● Copy the value
● Replace the first token in the copy with the current “version id value” of the addressed resource
● Compare the (altered) copy to the original
● Respond accordingly
Entity tag comparison is preferred in all cases over date comparison. Consequently, it is highly
recommended that servers generate an entity tag for all resource representation responses they send.
However, even using the above scheme, this necessarily involves some way of determining a “version id”
for a resource. Even for weak tags it necessary to generate a unique value per representation
(which
may, in fact, be
harder
to build than the above approach to strong tag construction).
This small requirement
may
be difficult to fulfill when adapting older systems to a RESTful architecture.
Consequently, if it is not possible to implement some kind of version id for a resource without significant
effort, then it is better to not rely on entity tags at all. In this case, date comparison is a better option.
9.3.5. Last-Modified
LastModified:
<HTTP Date>
This header is the date/time that the representation in the response last changed.
49
Note that technically, as with entity tags, this value applies to
representations
, not directly to
resources
.
However, in practice, this distinction is normally academic LastModified date of a given
representation will likely only change when the resource state changes, and it will then change for all
representations.
The value of this header
may
be used by clients in conditional requests for the purposes of:
● Cache control
(i.e.
IfModifiedSince conditional requests)
● Update control (or optimistic locking) (i.e.
IfUnmodifiedSince conditional requests)
However these are weaker controls than those available via
entity tags
because:
● The client can actually present
any
date in a dateoriented conditional request header, and
● HTTP dates have only 1second resolution
If possible the server should always return this header in a representation response (i.e. a successful
GET or a “contentlocation pattern” creation/update request).
This use is closely related to the design of hypermedia representations and, consequently, is addressed
in detail
elsewhere in this document.
Whenever a response contains a resource representation, this header
must
also be returned with the
appropriate values for the context.
9.3.7. Location
Location:
<URI>
This response header is used to refer the client to a specific resource related to the response.
While the description allows for many uses, the two main uses related to this REST approach are:
● When a request directly creates a new resource, the URI of this new resource will be returned in
this header. Examples of this type of request are a POST to a collection resources, a “creation”
PUT to a store resource, and the “createexecute” POST pattern for an operation resource.
In the event that a number of resources were created, only the URI of the “main” resource should
be set as the value of this header.
● When the server returns a 202 Accepted status, the URI of a “monitoring” resource is returned
in this header.
50
10. API Versioning
Rest resources are used to represent data and functionality from live software systems. It’s therefore
quite common for the underlying business objects to change over time. REST API must evolve with
these changes without breaking compatibility and support for existing API clients. Compatibility is
maintained by implementing versioning of the API.
Well, actually the man who first formalized REST (Roy Fielding) has one word to say about versioning
APIs:
DON'T ! And he was specifically referring to the practice of
sticking clientvisible interface numbers inside various names so that the client labels every
see
interaction as belonging to a given version of that API
However, there are two approaches normally taken to implement REST API versioning:
1. Include version identifier in URIs
2. Include version identifiers in HTTP headers
<This section will be updated with a recommendation on the preferred option>
Since REST APIs are architecturally similar to web sites, they are susceptible to many of the same
security concerns faced by web applications. Fortunately, many of the same security safeguards
developed for web applications are also applicable to APIs.
● What’s common
○ TLS
■ Require TLS for all access to the API
■ Reject all nonsecure requests with
403 status code
■ Avoid http to https redirects
● What’s different
○ No login redirects
■ APIs are headless so no expectation of interactive login UI. API consumer may
present its own login UI
51
11.1. Authentication
REST APIs must verify that each request originates from a known consumer. Whereas web applications
can handle this requirement by displaying a login page, APIs must provide an alternative mechanism for
this authentication.
Typical options for API authentication are:
● Basic Auth
● HMAC
● OAUTH
○ OAUTH1
○ OAUTH2
11.2. Authorization
Once requesters are authenticated and known to be who they claim, the API must then apply access
control rules to limit what requests are allowed to do. Unlike a web application with a user interface, APIs
can’t just display an error message. Instead authorization failures must be handled with appropriate
HTTP status codes as described elsewhere in this document.
Authorization failures should respond with
401 Unauthorized
status codes.
https://www.owasp.org/index.php/REST_Security_Cheat_Sheet
List of considerations:
● Shared secrets
● Common exploits
○ XSS
○ CSRF
● Attack vectors
○ Client inspection
○ MiM
○ Server penetration
● Securing integration layers
52
12. Do’s and Don’ts
Dos Don’ts
Do use
meaningful
resource names (RMM level 1) Don’t tunnel everything through a single
service, resource or endpoint
Do place hypermedia (resource links and their Don’t expect API consumers to know or guess
relationship to the current resource) in resource relationships, or locations of the hypermedia
representations to explicitly describe related links within the resource representation
resources (RMM level 3)
Do implement a JSON representation of resources. Don’t exclusively use XML to represent
Other representations are optional resources
Do use
media type headers (the request
Accept* Don’t create different URIs for different
headers) to request different resource representations
representations
Do
pluralize
“set” resource names (those that Don’t try to distinguish between plural and
denote collection or store resources) singular names such as /person and /people in
URIs
Use the HTTP protocol to it’s fullest if the HTTP Don’t “tunnel”
anything
!
protocol provides for a facility, then strongly prefer
the use of that facility over some “proprietary”
implementation. Examples include:
● Proper use of the HTTP methods (RMM level
2)
● Full use of HTTP status codes
● Asynchronous operations (the 202 status
code and Location header)
● Cache control headers
● Optimistic locking ( ETags and conditional
requests )
● Metadata (
Link header ,
OPTIONS method,
“
rel ” attributes ...)
● https://developer.github.com/v3/
● HTTP methods are properly used where applicable
● Provides a Root endpoint URI which lists all supported resources
● Resource URIs don’t include methods in name
● Semantic and intuitive to understand and consume
○ GET /repos/{user id}/{repo id}/notifications
● Supports all common HTTP methods
○ https://developer.github.com/v3/#httpverbs
53
●
● https://dev.twitter.com/rest/public
● Supports only GET and POST HTTP methods
● Therefore ends up including methods in URI names
○ POST /statuses/create
○ POST /statuses/destroy
■ https://dev.twitter.com/rest/reference/post/statuses/destroy/%3Aid
●
●
54