REST API Conventions

The key abstraction of information in ​REST​ is a ​resource.​ Any information that can
be named can be a resource: a document or image, a temporal service (e.g.
"today's weather in Los Angeles"), a collection of other resources, a non-virtual
object (e.g. a person), and so on. In other words, any concept that might be the
target of an author's hypertext reference must fit within the definition of a resource.
A resource is a concept ual mapping to a set of entities, not the entity that
corresponds to the mapping at any particular point in time.”​ - ​Roy Fielding’s
dissertation​.

What to expect?
RESTful design is an architectural pattern and NOT a protocol. Hence there are no specific
thumb-rules which will guide you to create good REST compliant URIs and APIs.

My goal is to talk about core concepts of REST which one should keep in mind before creating
REST URIs and APIs. One of the key point to understand is that there is an infinite URI space
that is available to you, but with great power comes great responsibilities - it is good to avoid
resource proliferation that may add confusion to the API design. As long as there is a genuine
need for the resources with clear consumer “intent” which fits in the overall API design, a
particular URI space vector can be instantiated for consumption.

You need to have basic concepts on REST Architecture and nothing advanced to understand
this article.

Core Concepts
Idempotent methods
An idempotent HTTP method is a HTTP method that can be called many times without different
outcomes. It would not matter if the method is called only once, or ten times over. The result
should be the same.
x = 4;
x++;
The first example is idempotent: no matter how many times we execute this statement, 'x' will
always be 4. The second example is not idempotent. Executing this n times will result in
different outcomes.

So which HTTP methods are idempotent and which ones aren't:

 HTTP Methods Idempotent

OPTIONS YES

GET YES

HEAD YES

PUT YES

POST NO

DELETE YES

PATCH NO

Reification
Dictionary meaning of Reification is:
make (something abstract) more concrete or real
In other words, reification makes something abstract (e.g. a concept) more concrete/real. The
idea is to not focus on the entity itself, but on the ​intent.​ For example, let's take an upcoming
feature of capturing a lineage in Bedrock from internal and external client:
@RequestMapping(value = "/lineage", method = RequestMethod.POST)
public @ResponseBody JSONResponse saveLineage( @RequestBody LineageVO lineageVO) {
do execute security checks
do execute data validity checks

do get of hdfs data type mapping

do get of entity metadata
do get of process details
do get of workflow steps details
do execute business logic like building relationship DAG
do handle exceptions
}

The above pseudo-code is about creating a new lineage detail for an entity and as expected it
goes through multiple functional steps. As you can see, "lineage" is an abstract idea and what it
primarily represents is entities and connections between them.

Note:​ This could have been modeled with entities like ​/baseuri/entity/lineage​ but
lineage operation is an aggregated business process by itself. A good URI will be
/baseuri/lineage

Reification plays a classic role in this as it helps to define a clear ​resource state​ for lineage. If
you had modeled it with "entity" then you are not clear on your "intent", as entity URI should be
operating on entities only and not creating relationships between them. However a reified idea
of "lineage" has a clear "intent" of resource state which is of relationships (DAG - directed
acyclic graph) of entities and its related properties. I hope you are able to see the differentiation
between the TWO "intended" states.

Also, you could have not created an URI ​/lineage​ and let the consumer call appropriate
smaller APIs to accomplish a task. That would have been worse as you are now migrating
business logic to the API consumer/client which is not desired at all. Further, by doing so we
make the application brittle and error prone. For example, let's assume the consumer forgets to
run security or data validity checks - implications can be catastrophic and make the application
unstable. There’s no reason for the client to have to do all this itself. The API provider needs to
pick one service to handle the coordination responsibility. In object oriented methodology you
can call it encapsulation. You will also see below this concept reinforced via ​Coarse Grained
versus Fine Grained Services.

Parts of Speech

There has been argument on what part of speech the REST URIs should be named i.e. whether
they should be nouns, verbs, adjectives and so on. However, we will focus on just TWO of them
which are NOUNs and VERBs. Generally, the resource names should be NOUNs and the
VERBs (action) are portrayed by HTTP verbs like GET, PUT, POST etc. Hence if you are trying
to create an entity, then you would name the URI as ​/base_uri/entity​ (entity being the
noun) with a VERB of HTTP POST.

But there are instances where business processes are better depicted as VERBs. For example,
let's take this Twitter API
https://api.twitter.com/1.1/direct_messages/sent.json​. Here "Sent" is a
VERB but if you see the definition - its "intended" use is on messages - "​Returns the 20 most
recent direct messages sent by the authenticating user.​" So one could argue that the resource
is not "sent", but rather "direct messages sent".

A guideline in this aspect might be:

● If a process needs a ​resource state​ on its own then it can be part of the URI. As in the
above example - process "sent" has a STATE and it's INTENT(ed) use is on
RESOURCE "messages", hence it can be modeled in the URI.
● Also remember a NOUN can be converted to VERB and vice-versa. Let's take an
example of workflow registration. The URI can be ​/baseuri/workflow/register
with HTTP POST - although "register" is a VERB but it has a STATE on RESOURCE
"workflow", OR the URI can be ​/baseuri/workflow/registration​ with HTTP
POST, where "registration" is NOUN form of "register". We would recommend the
second option.

Coarse Grained versus Fine Grained Services

One of the basic decisions we need to make is about the scope and boundary of our services.
We try to balance how fine-grained versus how coarse-grained the services should be. Services
should not be monolithic which makes them tough for reuse and hard to maintain. On the other
hand they shouldn't be too narrow so that business logic starts to spill over on the client side.
Piggybacking on the previous example of "saveLineage", one can define each of the "do" steps
as separate services which leads to clients having to manage all the interaction points. This is
undesirable as mentioned before.

A guideline to follow here might be that a service shouldn't violate its "​intent​(ed) r​ esource​".
Please pay close attention to the word "intent(ed) resource", what I mean is that a service
should abide by its responsibility for the resource it depicts. For example, if a process just needs
to create an entity resource then define an URI for that like ​/baseuri/entity​ with HTTP
POST. But if you want to define a workflow on entities then the need might be create an entity,
do additional steps and then create a workflow, in which case a URI like
/baseuri/entity/workflow​ with HTTP POST will be a bad REST URI as we are diluting
out "intent". A good REST URI will be ​/baseuri/workflow/registration​ as now
"workflow" represents a RESOURCE and everything related to workflow registration goes via
this service.

CRUD Operations
One of the common mistakes we make while designing REST interfaces is thinking in terms of
CRUD (Create, Read, Update, Delete) operations. This concept works very well from a data
access point of view but not from a REST perspective. If you take the above example of lineage,
thinking it from a CRUD perspective of entity domain model would have led to a wrong REST
design as we will either put a lot of lineage domain specific logic into the client or we overload
the entity with a resource state (like the relationship via DAG) it's not intended to represent.
Hence think from a point of service graininess, how low level or aggregated you
need so that you have correct representation of resource state. Applying SOA
(Service Oriented Architecture) principle - service that hosts the resource is the only
agent that can directly change its state.

Intent

Literal meaning of "intent" means "intention or purpose". Intent helps us to identify the
underlying resource and it's state representation. As per Roy Fielding's definition - "​Any
information that can be named can be a resource: a document or image, a temporal service
(e.g. "today's weather in Los Angeles"), a collection of other resources, a non-virtual object (e.g.
a person), and so on."​ - a resource can be any abstract concept or concrete thing. Hence while
defining a REST API you need to be clear on intent of resource representation. The boundary
should be very crisp and clear with the service operating on a well defined state of a resource.
Each service (URI) then just POSTs their 'intents' or publishes the resource states they
themselves own.
Use HTTP Verbs to Mean Something
API consumers are capable of sending GET, POST, PUT, and DELETE verbs, and these verbs
greatly enhance the clarity of what a given request does. Generally, the four primary HTTP
verbs are used as follows:

GET
Read a representation of a specific resource (by an identifier) or a collection of
resources.
PUT
Update a representation of a specific resource (by an identifier) or a collection of
resources.
Use PUT to create new resources only when clients can decide URIs of resources. For
example, in Ingestion API, if the client can control the URI identifier of the fileSystem
type then use a PUT to create it else use POST.
DELETE
Remove/delete a specific resource by an identifier.
POST
Create a new resource or make various other changes to a resource. Also a catch-all
verb for operations that don't fit into the other categories. In POST the server decides the
URIs of the resource.

Use HTTP Response Codes to Indicate Status

Response status codes are part of the HTTP specification. There are quite a number of them to
address the most common situations. In the spirit of having our RESTful services embrace the
HTTP specification, our Web APIs should return relevant HTTP status codes. For example,
when a resource is successfully created (e.g. from a POST request), the API should return
HTTP status code 201. A list of valid HTTP status codes is available here which lists detailed
descriptions of each.

Suggested usages for the "Top 10" HTTP Response Status Codes are as follows:

200 OK
General success status code. This is the most common code. Used to indicate success.
201 CREATED
Successful creation occurred (via either POST or PUT). Set the Location header to
contain a link to the newly-created resource (on POST). Response body content may or
may not be present.
204 NO CONTENT
Indicates success but nothing is in the response body, often used for DELETE and PUT
operations.
 Bedrock Handled Exception Types - ContentNotFoundException

400 BAD REQUEST

General error for when fulfilling the request would cause an invalid state. You can return
this error when your server cannot decipher client requests because of syntactical errors.

Bedrock Handled Exception Types - IllegalArgumentException,

ValidationException

401 UNAUTHORIZED
Return this when the client is not authenticated to access the resource. If your server will
not let the client access the resource even after authentication, then return ​403​ instead.
403 FORBIDDEN
Use this when your server will not let the client gain access to the resource and
authentication will not help. For instance, you can return this when the user is already
authenticated but is not allowed to request a resource, for reasons such as not having
the right access privileges.

Bedrock Handled Exception Types - OperationNotAllowedException,

AccessDeniedException

404 NOT FOUND

Return this when the resource is not found. If possible, specify a reason in the message
body.

Bedrock Handled Exception Types - ResourceNotFoundException

405 METHOD NOT ALLOWED

Used to indicate that the requested URL exists, but the requested HTTP method is not
applicable. For example, POST ​/users/12345​ where the API doesn't support creation of
resources this way (with a provided ID). The Allow HTTP header must be set when
returning a 405 to indicate the HTTP methods that are supported. In the previous case,
the header would look like "Allow: GET, PUT, DELETE"
409 CONFLICT
Whenever a resource conflict would be caused by fulfilling the request. Duplicate entries,
such as trying to create two customers with the same information, and deleting root
objects when cascade-delete is not supported are a couple of examples.

Bedrock Handled Exception Types - NotUniqueException

500 INTERNAL SERVER ERROR

Never return this intentionally. The general catch-all error when the server-side throws
an exception. Use this only for errors that the consumer cannot address from their end.
 Bedrock Handled Exception Types - BedrockException, Exception

In addition, the response message shouldn't have any HTTP Error Codes in them. Error codes
should be sent back part of the HTTP message body as per its specification. Please take a look
@ GitHub section on the same - ​https://developer.github.com/v3/#client-errors​. The application
response message should contain error messages and its details.

Example (GitHub)
There are several examples of REST APIs out there and I have quoted some from Twitter
above. However I think the best one I have seen so far are from public ​GitHub API​ - it is very
well defined with an appropriate resource granularity.

You will see that all the above concepts are being applied to its resource naming. For example
GitHub has reified concepts of "Forks" and "Merges", which are both abstract concepts but
made concrete via ​merges​ and ​forks​ sub-collection. At the same time it also has very fine
grained resources like "​Tag​" as an API. Hence as you can see in practical implementation of
APIs, you will need both coarse grained aggregate "business processes" plus low level “nouns”
resources.

Summarizing
As you can see there is no single approach that will lead to a compliant REST URI. A fact I
mentioned before REST is an architecture and not a protocol. However if you follow the above
guiding principles, you will come up with a pretty well defined API. Here is a summary of what
we have seen so far:

● Idempotent Methods
○ The very first thing to keep in mind is which methods are idempotent and which
ones aren't. This will help in selecting the right HTTP Verb for the REST API.
○ There is another concept of "safe methods" which I haven't touched as I don't
think it plays a big role. However please feel free to read on that @
https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol#Safe_methods
● Reification
○ You will come across scenarios where you are trying to model a business
process or an aggregate process which cannot be represented using low level
nouns. In those scenarios think from an elevated angle and try to structure an
API which has a crisp "​intent​" and ​distinct​ ​representation​ of ​resource​ ​state​.
● Noun versus Verb
○ Structure the URI's as ​NOUNs​ and if a process needs to be structured as VERB
then ensure it ​has​ a ​resource​ ​state​ on it's own.
 ○ HTTP Verb ​should​ complement as the action for the URI.
● Granularity of services
○ REST URI definitions should also be driven by the granularity of services i.e.
whether low level nouns suffice or needs to be an aggregated service.
○ Ensure business logic doesn't spill into client and all coordination is encapsulated
into a service
○ Don't think from a CRUD perspective, whereas always think from a ​resource
point of view.
○ URI needs to have a clear and crisp "intent" of the underlying ​resource​ ​state​ it is
representing​.
○ Apply SOA design principles as almost most of the times they help in devising a
good REST URI.
● HTTP Verbs
○ Abide by the ​core definition​ of each of the HTTP verbs. They are part of the
REST URI and together they should indicate the "​intent​" of the URI.
● HTTP Response Codes
○ API responses ​should NOT​ contain HTTP error codes in their message body
whereas they should be in HTTP body. However API responses ​can​ contain
application specific error codes.
○ API responses ​should​ contain error details which helps the client display
meaningful messages to the user.

Cookbook
Feature Flag Operation

● Intent:​ Need to indicate to the user that this API is unavailable because the feature Flag
is turned off
● What is the action​ - respond with http 501 error code. Optionally respond with a
"Feature "FLAG_NAME" unavailable." message

Date Time

● Intent:​ Need to indicate a date or time in the REST API

● What is the action​ - W3C strongly ​recommends​ that you use the ISO format such as
1997-07-16 or 1997-07-16T19:20:30+01:00, etc. These formats support time zones and
can be parsed easily by javascript for a human readable, locale-specific format.

URI Design Guidelines

● Use the forward-slash separator (/) in the path portion of the URI to indicate a
hierarchical relationship between resources. For example, all the following URIs convey
a hierarchical association between path segments:
 ○ /ingestion/filesystems/{fileSystemId}/connections/{fileSys
temConnectionId}
○ /ingestion/filesystems/{fileSystemId}/connections
● Use the comma (,) and semicolon (;) to indicate nonhierarchical elements in the path
portion of the URI. The semicolon convention is used to identify matrix parameters:
○ I am not able to find a good example in Bedrock but what I mean is this -
/coordinates;w=39.001409,z=-84.578201
○ Note:​ not all code libraries recognize the comma and semicolon as separators
but Spring MVC does in the form of ​MatrixVariable.​
● Use the ampersand (&) to separate parameters in the query portion of the URI. For
example:
○ /ingestion/filesystems/connections/search?sort=ascending

Save/Create Operation

● Intent:​ Need an API for Ingestion to ​save​ a "Connection" for a ​given​ "Filesystem"
● What is the action​ - SAVE which indicates a new connection object will be created and
identifier generated by the service, hence "​Idempotent​" property is ​NOT​ needed. ​POST
HTTP Verb is what we need .
● What is/are the resource(s)​ - "filesystem" and "connection" and hence each will have
an association with respect to each other.
● "Plural" OR "Singular" -​ Plural, as the resources are collections. Filesystem will have
an identifier in the URI as we are creating a connection for a filesystem and the
connection identifier will be generated by API.
● What is the URI​ - ​/ingestion/filesystems/{fileSystemId}/connections

Get/Read Operation

● Intent:​ Need API for Ingestion to ​get​ a "Connection" from a list of connections in a
filesystem.
● What is the action​ - GET which indicates an existing connection object will be retrieved
by the service, hence "​Idempotent​" property is needed. ​GET​ HTTP Verb is what we
need .
● What is/are the resource(s)​ - "filesystem" and "connection" .
● "Plural" OR "Singular" -​ Plural, as the resources are collections. As we can directly get
a connection from a list of connections, we are going to get our resource via a
connection identifier.
● What is the URI​ -
/ingestion/filesystems/{fileSystemId}/connections/{fileSystemCo
nnectionId}

Delete/Delete Operation
 ● Intent:​ Need API for Ingestion to ​delete​ a "Connection" from a list of connections in a
filesystem.
● What is the action​ - DELETE which indicates an existing connection object will be
deleted by the service, hence "​Idempotent​" property is needed. ​DELETE​ HTTP Verb is
what we need.
● What is/are the resource(s)​ - "filesystem" and "connection" .
● "Plural" OR "Singular" -​ Plural, as the resources are collections. As we can directly
delete a connection from a list of connections, we are going to delete our resource via a
connection identifier.
● What is the URI​ -
/ingestion/filesystems/{fileSystemId}/connections/{fileSystemCo
nnectionId}
● Note:​ ​As you can see the REST URI for GET and DELETE is identical but the difference
is in the intent/behavior which is depicted by the HTTP VERB (GET vs DELETE)

Search/Post Operation

● Intent:​ Need API for Ingestion to ​search​ from a list of connections in a filesystem.
● What is the action​ - This is a classic use case of '​reification​' which means search is an
abstract concept and hence we need to make it concrete in REST resource parlance. As
it's a complex request object, hence we use ​POST​ HTTP Verb.
● What is/are the resource(s)​ - "filesystem" and "connection" .
● "Plural" OR "Singular" -​ It doesn't apply over here.
● What is the URI​ - ​/ingestion/filesystems/connections/search
● Note:​ It cannot be a GET operation as the API is not retrieving a resource on a single
resource property but a combination of resource properties.

Put/Create Operation

● Intent:​ Need API for Ingestion to ​create​ a "Filesystem".

● What is the action​ - PUT which indicates that the client determines the URI of the
resource as it has "​Idempotent​" property. ​PUT​ HTTP Verb is what we need.
● What is/are the resource(s)​ - "filesystem"
● "Plural" OR "Singular" -​ Plural, as the resource is collections.
● What is the URI​ - ​/ingestion/filesystems/{fileSystemId}
● Note:​ ​As the operation is idempotent the service should not create new resources for
multiple requests and hence should ensure it creates the first time and updates the same
resource for​ subsequent ​requests.

Put/Update Operation

● Intent:​ Need API for Ingestion to ​update​ a "Filesystem".

 ● What is the action​ - PUT which indicates that the client determines the URI of the
resource as it has "​Idempotent​" property. ​PUT​ HTTP Verb is what we need.
● What is/are the resource(s)​ - "filesystem"
● "Plural" OR "Singular" -​ Plural, as the resource is collections.
● What is the URI​ - ​/ingestion/filesystems/{fileSystemId}
● Note:​ ​As the operation is idempotent the service should not create new resources for
multiple requests and hence should ensure it creates the first time and updates the same
resource for​ subsequent ​requests.

Batch Operations

● Intent:​ Need API to delete workflows in bulk.

● What is the action​ - DELETE which indicates that the client had to determine an URI of
the resource as it has "​Idempotent​" property. ​DELETE​ HTTP Verb is what we need.
● What is/are the resource(s)​ - "workflow"
● "Plural" OR "Singular" -​ Plural, as the resource is collections.
● What is the URI​ - ​/workflows/bulk
● Note:​ The ids to delete will be sent in the request body
● Response​: The API should return 4xx or 5xx error codes if none of the ID's can be
operated on (depending on the specific issue that occurred). However, if some of the IDs
were successfully deleted, the API should still return a 200OK response code with a list
of IDs that were deleted (we return the ids operated on, instead of the ones not operated
on, because some bulk operations may be unbounded). Note that the validity of one id
should not affect the operations on the other. In other words, if the user supplied 5 ids to
be deleted and 1 of them is invalid, the delete operation should still continue on the other
4. In this cases, the response message may indicated that the operation succeeded with
warnings.

Asynchronous Operation

● Intent:​ Need API to handle asynchronous set of operations on a resource.

● On Receive​(ing) - a POST/PUT/DELETE request, creates a new type of task resource,
and returns status code ​202​ (accepted) with a representation of the new resource. The
purpose of this resource is to let a client track the status of the asynchronous task.
● In Progress​ - Return response code ​200​ (OK)and a representation of the task resource
with the current status.
● On Success​ - Return response code ​303​ and a location link containing the resource
URI that shows the outcome of the task.
● On Failure​ - Return response code ​500​ with a response informing that the resource
creation has failed with details on failure.

Message body for errors

Include a body in the error response for all errors. In the body, include some or all of the
following:

● A brief message describing the error condition

● A longer description with information on how to fix the error condition, if applicable
● An identifier for the error
● A link to learn more about the error condition, with tips on how to resolve it

Request Body in HTTP DELETE Verb

Question​ - Recently I got an issue of using HTTP method DELETE. Any API having DELETE
as request method can not return any value if invoked using ​java.net.HttpURLConnection​.
(java.net.ProtocolException: HTTP method DELETE doesn't support output). This is a bug
solved in jdk-8. (​http://bugs.java.com/view_bug.do?bug_id=7157360​). Is that something we
should take care while making an API as our clients might not be having jdk-8?

Answer -​ Although Oracle portal mentions it is a bug fix but I don’t think it’s a bug and just an
enhancement to support non-defined HTTP RFC Requirement. If you look at this -
https://tools.ietf.org/html/rfc7231#page-29​ there is a paragraph "A payload within a DELETE
request message has no defined semantics; sending a payload body on a DELETE request
might cause some existing implementations to reject the request.” Hence it’s not required
technically to send back a body, however it doesn’t hurt and is a good to have feature. Anyways
most of the popular AJAX APIs support so it’s not a problem there. But what we should do is not
to have a body for DELETE and let the client react on the HTTP status code like 200 etc.
However you might say that our REST responses have a JSON property to depict the status
code and message and for that I would say that’s something we don’t do correctly and should
be amended. HTTP error codes are self descriptive and it should be part of HTTP response
message not part of application response message. The below java portal link mentions what
doesn’t work.

However here is the code that does work and it is a perfect code in my opinion. There is no
response body but there is response code of HTTP which describes the state of the resource
operation perfectly.
URL url;
try {
url = new
URL("http://javafx-jira.kenai.com/rest/api/1.0/issues/47181/watchers"
);
HttpURLConnection connection = (HttpURLConnection)
url.openConnection();
connection.setRequestMethod("DELETE");
int responseCode = connection.getResponseCode();
 System.out.println("Delete Method Response Code: " +
responseCode);
} catch (MalformedURLException e) {
// TODO handle Exception
} catch (IOException e) {
// TODO handle Exception
}

References
● Roy Fielding's REST dissertation (Section 4, 5 and 6 primarily) -
https://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm
● Reification - ​https://en.wikipedia.org/wiki/Reification_(computer_science)
● Twitter REST APIs - ​https://dev.twitter.com/rest/public
● HTTP Status Codes - ​http://www.restapitutorial.com/httpstatuscodes.html
● GitHub AP - ​https://developer.github.com/v3/
● Richardson Maturity Model of REST services -
https://martinfowler.com/articles/richardsonMaturityModel.html
● Different kinds of REST API
Authentication.​https://docs.google.com/spreadsheets/d/1tAX5ZJzluilhoYKjra-uHbMCZra
aQkqIHl3RIQ8mVkM/edit#gid=0