API Developers Never REST

Disclaimer: despite the controversial title, this article is not trying to show that RPC is a superior approach to REST, or GraphQL is superior to RPC. Instead, the goal of the article is to give you an overview of the approaches, their strengths and weaknesses. The final choice anyway will be a trade-off.

Even though HTTP is an application layer (i.e. L7), protocol, when it comes to API development, HTTP de facto plays the role of a lower-level transport mechanism.

There are multiple conceptually different approaches on how to implement an API on top of HTTP:

  • REST
  • RPC
  • GraphQL

...but the actual list of things an average API developer needs to be aware of is not limited by these three dudes. There are also JSON, gRPC, protobuf, and many other terms in the realm. Let's try to sort them out, once and for all!

Representational state transfer (REST)

First and foremost, REST is just a software architectural style. It's a set of design constraints, not a concrete protocol. REST relies on the idea of a resource. I.e. a REST API is comprised of a set of resources (nouns) and a fairly limited number of actions (verbs) to interact with these resources (fetch, create, update, delete, etc). It's ideologically very close to the original HTTP design heavily based on the concepts of resources (URLs) and methods (GET, POST, PUT, DELETE). Thus, from the implementation standpoint mapping of the REST model to the HTTP protocol is relatively straightforward:

# Create new book
POST http://myapi.com/books/ (author="R. Feynman", year=1975) -> book_id

# Get book with ID = 1
GET http://myapi.com/books/1 () -> (id=1, author="R. Feynman", year=1975)

# Update book with ID = 1
PUT http://myapi.com/books/1 (id=1, author="Richard Feynman", year=1975) -> (...)

# Delete book with ID = 1
DELETE http://myapi.com/books/1 () -> null

Probably every modern web framework provides all the needed facilities to build a RESTful web service out of the box. And from the client-standpoint, calling a REST API is pretty trivial - it just needs to be able to send the specified HTTP methods to the set of predefined URLs.

However, from the API designer standpoint, it may be not so obvious how to express a real-world domain model in terms of resources (i.e. nouns) keeping the number of allowed actions (i.e. verbs) limited. Surprisingly, inspite of a super-widespread (ab)use of the word REST nowadays, not so many contemporary APIs can be actually considered truly RESTful. Trying to play a purist while designing a RESTful web service can lead you to a really clunky API design.

Remote procedure call (RPC)

Similarly, RPC is also an API design technique. RPC focuses on the idea of actions (verbs) and that often makes the involved resources (nouns) fairly primitive and ad hoc. For every procedure or transaction in the business model an API designer simply adds an RPC endpoint:

# Create new book
createBook(author, year) -> book_id

# Get book by ID
getBook(book_id) -> book

# Change book's author
setBookAuthor(book_id, author) -> null

# Delete book by ID
deleteBook(book_id) -> null

Under the hood, there should be another layer mapping such procedure calls to the HTTP requests. E.g. setBookAuthor(1, "Richard Feynman") can be mapped to something like:

POST http://myapi.com/set-book-author/ (book_id=1, author="Richard Feynman") -> null

Now, let's compare the tasks of changing the author's name in the REST and RPC worlds. While the implementation seems trivial in the case of RPC, there is a bunch of controversial questions that need to be answered in case of the RESTful implementation. If the author's name is an attribute of the book, should we provide the full book object with the modified author field while sending PUT /books/1? What if there is no full book object around on the client-side? Should we fetch the book first or can we just send only the book ID and the new author name? But what's about the rest of the attributes on the server-side? Should they be zeroed out if an almost empty PUT request arrives (that would be disastrous but fully aligned with the REST principles decision)? Or maybe we should give up on the HTTP PUT method and start using PATCH (have you ever heard of it)?

Luckily, if we follow the RPC approach, we can simply omit most of the questions from above. But of course, there is always a downside - a typical RPC API usually consists of a pretty high number of custom procedures. Obviously, it makes the mapping of the RPC model to the HTTP layer a non-trivial task. The good thing is that API designers rarely need to think of this part at all. There are a lot of libraries implementing the RPC layer on top of HTTP and some of them are truly prominent (yes, I'm talking about Google's gRPC and Facebook's Thrift). But there is another, probably much more complicated problem with the RPC approach...

A dozen rich REST resources combined with 3-5 HTTP methods usually can cover a hundred use cases. REST API deliberately brings a regular structure to the domain model making its growth and evolution much more controllable. On contrary, RPC APIs grow organically. Introducing a new use case often require adding a few more API endpoints to the already bloated list. Since there is no entity-oriented structure enforced by the API design, over a relatively short period of time the number of RPC calls may exceed the maximum level of complexity a team can handle.

GraphQL

By this time, we already learned that neither RPC nor REST approach is ideal. REST suffers from over and under fetching problem and may lead to a significant mental overhead on the design phase. But if we try to avoid the entity-oriented design by replacing it with a hundred ad hoc RPC endpoints, over time the maintenance of the overgrown RPC API may become a nightmare.

GraphQL tries to address the weaknesses of both techniques. Assuming the entity-oriented domain model contributes to the peace of mind of developers in the long term, the GraphQL approach starts from defining a schema, i.e. set of resources (i.e. nouns) and their interconnections. Sounds like a graph, doesn't it? Having a formal scheme definition, GraphQL bumps a fairly complex server and client on top of it. The thick client allows querying (QL stands for query language) custom and compound resources. The thick server knows how to populate the response based on the client's query and the domain schema.

Thus, GraphQL API basically consists of a single endpoint. I.e. no more bloated APIs. At the same time, its super-flexible and tailored queries help to avoid fetching unnecessary data from the server. Additionally, developers still benefit from the formal entity-oriented domain schema. But magic always comes at a price (c). And the price in that case is the utter complexity of the GraphQL client and server sides. So, yeah, it's all about trade-offs...

What's up with protobuf, JSON, etc?

What could help with wrapping your head around the API development realm is the proper categorization of the moving parts. At this time we already know that REST and RPC are just architectural styles. And GraphQL, well... I tend to consider it as a style as well, even though technically it's a formal language backed by a runtime.

But if GraphQL is backed by a particular implementation, we probably could expect the same from RPC and REST. And indeed, there is a bunch of prominent RPC frameworks - gRPC, Apache Thrift, and Apache Avro to name a few. Surprisingly though, there seem to be no clear leaders among REST frameworks. Probably because REST is technically super-close to HTTP and almost every web framework already plays well with it.

Hm, but what's the deal with protobuf? Apparently, it's just one of the ways to serialize your data before it will be sent over the wire or stored somewhere. It just so happened that gRPC, a particular RPC framework, fully relies on protobuf. And since they often used hand in hand and have been released altogether, lots of people negligently use the terms gRPC and protobuf interchangeably. However, it's perfectly fine to use protobuf in your RESTful API as an encoding format and we do it currently at work for some of our services. Thus, protobuf should not be compared with REST or RPC frameworks but instead, it can be compared with XML or JSON.

To confuse you even further, Apache Thrift has its own serialization format called... Thrift! I.e. there is gRPC over protobuf and Thrift over Thrift 🙈

Summarizing, there are different architectural styles to design an HTTP API. The three most popular ones are REST, RPC, and GraphQL. Every style in turns can be implemented in many ways and there is a bunch of well-known frameworks such as gRPC or Apache Thrift. Under the hood, they rely on lower-level mechanisms like data serialization (protobuf, JSON, XML) or protocols (JSON-RPC, XML-RPC). And these lower-level bits sometimes can be reused between styles making the whole realm of API development looking pretty convoluted at first sight.

Further reading