With Ryan McCue leading off the day at A Day of REST, we were able to learn about the history of web APIs, APIs within WordPress, and where the WordPress REST API fits in.
Ryan's goal was not just to talk about what the API is, but why various API decisions have been made. He started with the history of WordPress APIs. Here are his slides if you'd like to follow along. You can view all videos as we release them on the A Day of REST video archive.
A brief rundown of existing APIs
Ryan talked about APIs in WordPress (some live, some ideas that failed) as well as other influential web APIs, to lay some groundwork for the WordPress REST API and what has influenced its design.
WordPress's most powerful API before now, XML-RPC
The XML-RPC API has been in WordPress since the very beginning, since before b2 was forked to WordPress. While the API is powerful, it's not the easiest to use.
RPC stands for Remote Procedure Call, and as the name indicates, it is designed to work with the XML syntax. There was a little known effort three years ago to port functionality from XML-RPC to a JSON format, called JSON-RPC. But Ryan notes that it was not as easy to work with as it may sound, considering XML-RPC just wasn't meant for that use case. That effort was abandoned eventually, in favor of creating the full API we have today.
The Admin Ajax API in WordPress gives developers the ability to interact with the server through remote calls, and has a lot of flexibility to do pretty much whatever one would want. However, “everything you build on top of it is completely custom.” And since nearly anything can be done with it, each use of it ends up with little consistency.
While a powerful developer tool that's been helpful for features like WordPress's Heartbeat API, it's not got the kind of structure and standards that would make it reusable, but rather keeps it as a developer tool to use for specific features.
RSS (and RSS.js)
RSS has been one of the most powerful distribution tools of the open web for ages. WordPress has awesome RSS support, and it's used all over the place: feed readers, podcast distribution, Facebook instant articles, and much much more.
RSS.js was also a thing, but, according to Ryan, not a very good idea, and didn't really catch on.
Stripe, the API-first payment processor
Ryan uses Stripe as an example of a company with a great REST API. They focus all of their features around an API-first methodology. And the API fully embraces both REST and JSON. He says that their API is both one of the best designed and best documented APIs on the web.
Twitter, one of the first REST APIs
Ryan says Twitter was one of the first companies to push a REST API, and prove that an API could be a, “core competency.” Twitter is also a good example of politics playing a big role in an API, as Twitter kept their API relatively open for a long time, and a huge ecosystem of third party developers utilized it, and then Twitter gradually ratcheted down that access and what third parties could actually do, making many in their community angry. Twitter has since tried to open back up a bit more, though many developers are skeptical, considering the burned bridges of the past.
Facebook, one of the best REST APIs
Facebook has one of the largest (or maybe the largest) REST APIs on the web, and Ryan says it's one of the best out there. And their REST API fits well with the rest of the application.
Facebook is really pushing their API, and they're also inventing new technologies, like GraphQL, that are quickly gaining adoption. GraphQL allows, “complicated, SQL like queries.” GraphQL is actually included in Drupal 8 — which is a rapid adoption to a major CSS — and the WordPress REST API team has looked at it as well, but is holding off because it's not standardized enough yet.
WordPress.com REST API
Of course, WordPress.com has its own REST API, and it also ships with Jetpack for self-hosted WordPress websites to utilize. The WordPress.com API has been out for a long time, but Ryan notes that it's not a fully open API. While some of the elements are available via Jetpack, much of its functionality relies on internal Automattic services and servers.
The WordPress.com API is also pivotal to the Calypso project, a single page application interface for WordPress.com.
Ryan says it's also not “exactly” a RESTful API, partly due to its age, but also because of compatibility issues. However, the WordPress.com API has been influential on the WordPress REST API and compatibility between the two will be important in the long term.
Automattic also paved the way for what both APIs call “enveloping” — that helps maintain client compatibility — as well as figured out solutions for various WordPress-specific issues that arose in development of the API.
Both Automattic's API team and the WordPress REST API team are trying hard for the two APIs to work well together, and be able to work similarly with other services.
The story of the WordPress REST API, so far
The WordPress REST API conversation started at the 2012 WordPress community summit, where a number of long reaching ideas were discussed. Ryan was one of about a hundred community members to attend, and from then on, his mind was set.
He says he “somewhat naively” thought a fully functioning REST API could be accomplished by WordPress 3.6. The very first version of the API was an attempt to port some popular XML-RPC functionality to a JSON format. Though with the evolution of WordPress, some XML-RPC stuff was, “left in the corner.”
Ryan calls JSON, “almost a reaction,” to XML as a format. JSON, “maps very well into the support that's built into a lot of languages.” It works well with many programs and, “how they want to store data.” That means that a JSON formatted API can be quite portable between very different types of applications.
Whereas XML was based on HTML and meant for documents, JSON is not. He also notes that XML doesn't mesh too well with the concept of REST, which is based on resources or objects in the database.
JSONP and a mega vulnerability
JSON isn't a perfect format either, Ryan affirms. He sites a horrible vulnerability that affected nearly the entire web, from small services to Google. JSONP enables services to talk cross-domain, and while it's a controlled setup, it's what Ryan calls a “controlled XSS” vector.
And a bug in Flash caused a big problem, where a simple hack made nearly every JSON service vulnerable. Fortunately it was an easy fix, but the Rosetta Flash debacle was a valuable lesson and a reminder that JSON has its issues as well.
Ryan calls REST a, “conceptual shift,” with great browser support considering the vast majority of RESTful systems communicate over HTTP. He says that RPC doesn't really map well to a CMS, whereas REST is “almost perfectly designed” for a CMS, though not without challenges.
He says there's a, “long tail of servers and proxies that don't support all the methods involved.” Many RESTful services won't mind this, but WordPress is best friends with the long tail of the web, and will definitely have to deal with such issues.
The organic growth of WordPress
Because WordPress has experienced “organic growth” and evolution, versus working off some master “design”, not everything (or almost nothing) is straightforward when working with the WordPress REST API.
Ryan used the example of IDs, slugs, and content for various WordPress objects to highlight these inconsistencies. The REST API attempts to normalize things, so uses
id for, well, IDs. But WordPress uses
id for posts,
comment_id for comments, and
term_id for terms. And the API uses
slug instead of
post_name, (or in the case of users,
user_nicename), which is odd considering it's not a name at all. The REST API has renamed these things to try and create consistency.
Ryan used enveloping as an example of a WordPress-specific API issue that needed to be solved.
Enveloping is a methodology borrowed from the WordPress.com REST API, which Ryan calls a layer on top of the response format. Since, “We can't trust servers. We can't trust proxies. We can't even trust the HTTP clients we're working with. They don't all work.” So enveloping treats the body of the response as its own object inside the main object, making sure that the return status is a 200, and able to, “sneak past proxies.”
The REST API infrastructure is, “what ties all of the APIs [in WordPress] together, quite tightly.” Merged in WordPress 4.4, the infrastructure is the foundational layer of the API.
Ryan says they went about creating the infrastructure differently than other parts of WordPress have been created. It's based on request and response, versus relying on global state. It makes many things simpler, and makes for much better reusability of code.
Ryan says, “your endpoints shouldn't need to be concerned with server compatibility, for example,” and cites decoupling the infrastructure as one of the team's best decisions in creating the API.
Meta in the API
By now, many readers know about the struggles with meta in the WordPress REST API, but at the time of this talk, Ryan touched on some of those challenges, and how because WordPress makes it really hard to know just what might be in meta data (via registration). It makes it incredibly difficult for the API to then determine what that data is as well, or what to do with it.
“Authentication is really hard to do right.” And as of yet, it's not actually in the API that's proposed for core. OAuth 2 would be the preference to include, by far, but it requires HTTPS, as it leaves encryption to the client, and therefore can't be relied upon for all sites. Joe and I talked more about this toward the end of our last podcast. There are multiple authentication methods available, and it's a problem that will need a continued effort.
Ryan did a really nice job starting the day off and providing both history and challenges for the REST API. In our next videos, we'll dig more into how people are using the WordPress REST API in real life.
The videos at A Day of REST were shot and edited by Rhys Alexander. The conference was organized by Human Made, and you can stay up to date with A Day of REST — which is going to Boston — and learn about other potential future events. There were eight great talks at A Day of REST, and you can read my review of the event as a whole, and look out for the next video to be posted here soon.