Qafoo GmbH - passion for software quality
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

:Author: 		Tobias Schlitt
:Date:			Thu, 13 Jun 2013 10:35:27 +0200
:Revision: 		3
:Copyright:		All rights reserved


=======================
Embedding REST Entities
=======================

:Description:
    During my talk at IPC Spring I showed an approach for embedding
    entities in REST responses. This methodology might be useful if the
    standard use-case for your API is to request multiple related entities
    together. The downside of such an approach is the raised complexity of
    caching and especially purging logic. In this blog post I will further
    elaborate on the approach of resource embedding.
:Abstract:
    During my talk at IPC Spring I showed an approach for embedding
    entities in REST responses. This methodology might be useful if the
    standard use-case for your API is to request multiple related entities
    together. The downside of such an approach is the raised complexity of
    caching and especially purging logic. In this blog post I will further
    elaborate on the approach of resource embedding.
:Keywords:
    rest, webservice, resource embedding, restful, standards, caching

During `my talk at IPC Spring`__ I showed an approach for embedding entities in
REST responses. This methodology might be useful if the standard use-case for
your API is to request multiple related entities together. The downside of such
an approach is the raised complexity of caching and especially purging logic. In
this blog post I will further elaborate on the approach of resource embedding.

__ /blog/047_pragmatic_rest_bdd_ipc.html

Entities
========

As an example I chose the resources *Product* and *Category* from the e-commerce
domain. A product entity could be encoded through the media type
``application/vnd.com.example.product+xml`` as shown as follows::

    <?xml version="1.0" encoding="UTF-8"?>
    <product
      xmlns="urn:com.example.product"
      xmlns:atom="http://www.w3.org/2005/Atom">
      <atom:link rel="self"
        type="application/vnd.com.example.product+xml"
        href="http://example.com/products/23" />
      <name>Glow Stone Driveway</name>
      <description>Awesome ...</description>
      <atom:link rel="collection"
        type="application/vnd.com.example.category+xml"
        href="http://example.com/categories/geek_toys" />
      <!-- More links ... -->
    </product>

A product has a ``name`` and a ``description``. Besides that, the representation
provides Atom__ links as hyper media controls, using `relations as recommended
by the IANA`__.

__ https://tools.ietf.org/html/rfc4287
__ https://www.iana.org/assignments/link-relations/link-relations.xml

Corresponding, the following example shows an encoded category entity::

    <?xml version="1.0" encoding="UTF-8"?>
    <category
      xmlns="urn:com.example.category"
      xmlns:atom="http://www.w3.org/2005/Atom">
      <atom:link rel="self"
        type="application/vnd.com.example.category+xml"
        href="http://example.com/categories/geek_toys" />
      <name>Geek Toys</name>
      <products>
        <atom:link rel="item"
          type="application/vnd.com.example.product+xml"
          href="http://example.com/products/23" />
        <!-- ... -->
      </products>
      <!-- Links: overview,  paging, sorting, ... -->
    </category>

Note that here is a list of product items linked from a category, which could
for example be paged, sorted, filtered and so on using URL parameters.

The Use-Case
============

Imagine the major use-case for the API is to retrieve the top 20 products from
a category. To retrieve these together with product details, a client needs 21
``GET`` requests on the first go. For each subsequent execution of the use
case, the 21 requests must be repeated, although the response might be that the
entities did not change.

Alternatively, the server can set expiry times for the entities, so that
clients do not need to re-check that frequently. But such time spans are hard
to calculate or even guess and can easily lead to stale caches.

So, depending on the requirements for data accuracy and clients, this might
easily lead to a huge number of requests.

Resource Embedding with HAL
===========================

A promising idea to solve the named issue is to deliver resources in an
embedded way while making explicit that the delivered data partially belongs
to a different resource that can be found at a different URI. Searching the web
reveals that there is already an approach for this out there, which is called
`Hypertext Application Language (HAL)`__.

__ http://stateless.co/hal_specification.html

Embedding products into a category using a HAL encoding is shown in the
following example::

    <?xml version="1.0" encoding="UTF-8"?>
    <resource href="/categories/geek_toys">
      <name>Geek Toys</name>
      <products>
        <resource rel="product" href="/products/23">
          <name>Glow Stone Driveway</name>
          <description>Awesome ...</description>
          <link rel="category" href="/categories/geek_toys" />
        </resource>
        <!-- ... -->
      </products>
    </resource>

HAL encodes any resource using the ``<resource>`` tag and has its own
``<link>`` tag. As can be seen, the products which are part of a category are
embedded into its representation, but the categories that are assigned to a
product are linked.

While this approach appears charming at first glance, it has some flaws:

First of all, HAL is meant to encode all entities in a standardized manner of a
``<resource>`` tag. That conflicts with one of the fundamental ideas of hyper media: to
encode an entity in a semantically meaningful way.

Second, the HAL specification does not provide its own namespace, making it
ugly to re-use among other XML languages. Finally, the specification introduces
its own ``<link>`` element instead of re-using existing standards like Atom__
or XLink__.

__ https://tools.ietf.org/html/rfc4287
__ http://www.w3.org/TR/xlink/

Better Resource Embedding
=========================

Since the idea of HAL is essentially quite nice, I made up my mind to make the approach
more standards-compliant and to mitigate the named issues. The basic idea is to
re-use the Atom ``<link>`` elements, as in the original drafts, and embed
resources as their children::

    <?xml version="1.0" encoding="UTF-8"?>
    <category xmlns="..." xmlns:atom="..."
      xmlns:p="urn:com.example.product"><!-- ... -->
      <products>
        <atom:link rel="item" type="..." href="...">
          <p:product><!-- ... -->
            <p:name>Glow Stone Driveway</p:name>
            <p:description>Awesome ...</p:description>
            <atom:link rel="collection"
              type="application/vnd.com.example.category+xml"
              href="http://example.com/categories/geek_toys" />
            </p:product>
        </atom:link>
        <!-- ... -->
      </products>
    </category>

This example left out some unimportant details, it is basically derived from
the original example in this post. However, in addition to the category
namespace, the namespace for encoding product entities is also imported using
the shortcut ``p``. Inside of the ``<atom:link>`` for a product, the entity
representation itself is embedded using this namespace.

Naturally, the mentioned drawbacks of HAL are mitigated: Clear semantics are
kept, each media type ships with a decent XML namespace and a standard
``<link>`` element is used (Atom). Luckily, the Atom specs allow any external
element to occur inside of ``<atom:link>`` so embedding the product entity is
perfectly valid.

Furthermore, if clearly documented, that would allow the REST provider to
transparently switch on and off embedding of specific resources. A client then
must not rely on embedded resources and must be capable of fetching them through
their link references. However, if available, it can use the embedded
information to cache an embedded resource transparently.

Bottom Line
===========

Of course, resource embedding is nothing I would recommend to you in
general. It can be considered a *hack* for special use-cases. One of its
drawbacks is to weaken the caching capabilities RESTful web APIs provide: The
category resource must be re-fetched every time one of the embedded products
changes. This of course also affects your application's caching logic, because
you need to purge caches accordingly.

So, if you think about making use of embedded resources, make sure you analyzed
the use-cases of your API carefully and keep the hack as local as possible.

..
   Local Variables:
   mode: rst
   fill-column: 79
   End: 
   vim: et syn=rst tw=79


Trackbacks
==========


Comments
========