Saturday, December 19, 2015

EVE Online XML API Endpoint Proxy

In an earlier post, I announced the release of the EVE XML API.  This is a Java library (at least the third or fourth by my count) which provides access to the EVE Online XML API server endpoints.  If you're writing third party code in Java, you might try this library out.  It's complete as of the latest release (not officially named -- but everyone is calling it the Frostline release).  It was also battle tested in a slightly different form as part of EveKit.

For small or one-off projects, it can be a pain to install a library.  Or, maybe you're using a language which doesn't have a ready made library for accessing the EVE Online XML API endpoints.  It's for these use cases that we've created the EVE XML API Proxy, a simple web service which exposes a REST API which in turn uses the EVE XML API library to call the EVE Online XML API server endpoints on your behalf.  In other words, a proxy for your EVE Online server requests.

To make all of this easier to use, we've annotated the proxy using Swagger.  This makes it very easy to view high quality documentation for the REST API, try out API calls directly, or generate proxy clients in a variety of languages.  We'll demonstrate each of these features in this blog post.

Resources

Live Proxy Site
GitHub Page
Swagger UI for Proxy
Swagger Editor/Code Generator (see instructions below)

A Brief Introduction to Swagger

I discussed Swagger in an earlier post, which I will briefly recap here.  Swagger is an API tooling ecosystem designed for REST based APIs.  It is open source with core libraries and tools available on GitHub (see the appropriate link under "Resources").

The Swagger tool chain is driven by an API specification written in a variant of JSON schema.  You can write a specification by hand, but it's also possible to auto-generate a specification by annotating an existing REST API.  We use the latter approach to generate a Swagger specification for the REST API exposed by our proxy.

There are three main tools which consume a Swagger specification:
  • Swagger Editor: This tool lets you create a Swagger specification online, test it out, then export a client or server back end in a variety of languages.  We use this tool below to show you how to generate a Python client for our proxy.
  • Swagger UI:  This tool turns a Swagger specification into online documentation.  The online documentation can also be used to try out the API (i.e. make API calls).  We use this tool next to introduce the proxy API and make a sample call.
  • Swagger Codegen: This is a build tool which converts a Swagger specification into a client or server back end in a variety of languages.  Much of the same functionality is built into the Swagger Editor, but the Codegen tool is usually more up to date and supports more languages.  Note that code generation for Javascript is usually not done statically.  Instead, you use the swagger-js Javascript module which generates a Javascript API dynamically from a Swagger specification.  We use the swagger-js module below.  We won't be using Swagger Codegen in this writeup.
Swagger has an active developer community and good support for many languages.  Unfortunately, it doesn't support XML very well, which is why I failed to create a direct Swagger specification for the EVE Online XML endpoints.  It's also worth noting that Swagger has competition in the form of RAML (RESTful API Modeling Language).  RAML has some nice features and better support for XML, but a much less active developer community with fewer languages supported.  I also ran into problems creating a direct specification for the EVE Online XML endpoints using RAML (I haven't written up that experiment yet).

Anyway, back to the proxy...

Viewing Documentation and Trying out the API

Swagger makes it very easy to view high quality documentation for REST APIs.  The easiest way to view the documentation for the proxy is to use the Swagger UI online demo pointed at our Swagger specification.  This link will do that.  If you click the link, you'll see a view like this:

This view shows each of the API sections exported by the proxy.  Clicking on the Server section, and then again on the first entry, will produce a detailed view of the "ServerStatus" endpoint:
This view gives a detailed description of the endpoint including a description of the expected return values.  In this case, there are three possible return codes: 200 (OK), 404 (server error), or 500 (service error).  The model schema is shown for each response type.

You can try calling this endpoint by clicking the "Try it out!" button, which will give a display like the following:
Each part of the response is shown with the content sent back from the server.

This example illustrates a few features worth discussing more carefully:
  1. You can cut/paste the "Curl" line to call the proxy end point from the command line.
  2. Every call has three possible results:
    • 200 (OK) is always the result when the call to the EVE Online XML server succeeds.
    • 404 (Not Found) is always the result when the call to the EVE Online XML server succeeds, but EVE returns one of the error codes defined in the error list.  In this case, the result body contains the error code, message and time fields.
    • 500 (Internal Server Error) is always the result when the proxy service itself has an error, either because there was an IO error attempting to reach the EVE Online XML server, or there was an internal error in the service.  In this case, the result will contain a simple error message.
  3. HTTP headers are used to relay certain EVE Online XML server fields
    • The "Date" HTTP field contains the "currentTime" field returned by EVE
    • The "Expires" HTTP field contains the "cachedUntil" field returned by EVE
    • The "EVEXML-Version" custom HTTP field contains the EVE API version parameter returned by EVE
  4. Calling the API from the Swagger UI doesn't show the Date and EVEXML-Version API headers (but they are there).  If you use the curl command line and pass the -v option you can view the HTTP headers directly.
You may have noticed an optional parameter for this endpoint called "server".  This optional parameter is available on every endpoint and provides a way to tell the proxy to call a server other than the default production EVE Online XML server (e.g. https://api.eveonline.com).  For example, if you want to invoke the test server you can enter "https://api.testeveonline.com" in the server field and get a result like the following:
Endpoints which require credentials will have parameters for passing api keys and vCodes.  For example, the CharacterSheet endpoint:

Example: Using the Proxy from Javascript

We've seen how to view proxy documentation and try out a few calls manually.  Now let's look at how you can use the proxy from within a web page using Javascript.  This is a simple example, but it's worth doing because Swagger provides a dynamic generator for Javascript clients.  This is different than generating clients for other languages which is mostly done statically (as shown in the Python example below).

The following HTML document illustrates how to retrieve and display server status (our GitHub page contains similar documentation):

<!DOCTYPE html>
<html>
<head></head>
<body>
  ServerStatus Call Example Using EVE XML API Proxy

  <div id="mydata"></div>
  <script src='https://cdn.rawgit.com/swagger-api/swagger-js/master/browser/swagger-client.min.js' type='text/javascript'></script>
  <script type="text/javascript">
    var url = "https://evekit.orbital.enterprises/xmlproxy/api/swagger.json";
    window.swagger = new SwaggerClient({
    url: url,
    success: function() {
    // On success, fetch server status and display it in the DIV above
    swagger.Server.requestServerStatus({}, {}, function(data) { document.getElementById("mydata").innerHTML = JSON.stringify(data.obj); });
    }});
  </script>
</body>
</html>

Note the use of rawgit.com to pull in the Swagger javascript module.  This is necessary because GitHub doesn't attach proper content types to raw files.  Rawgit does this for us so that we can pull the raw module directly into our code.

Loading this page will product output like the following:
The format of a Javascript library call is:
swagger.<api_section>.<method>({<method args>}. {<header args>}, success_callback, failure_callback) 
where "api_section" is one of the tags defined in the Swagger specification (e.g. Server, Character, Corporation, etc.) and "method" is an operation ID defined in the specification.  Unfortunately, you have to look at the Swagger specification to determine operation ID as neither the Swagger UI nor the Swagger Editor show this information in the documentation they generate.  To make it easier to use the proxy, we've consistently set the operation ID to be "request" plus the name of the XML end point (e.g. requestServerStatus).

Method arguments are set in the first argument to the request method.  For example, the following call sets the optional "server" parameter in the server status request:
swagger.Server.requestServerStatus({server: "https://api.testeveonline.com"}, ...)
Header arguments can be set in the second argument to the request method.  You can use a header argument to do things like specify the content type of the response.  We don't use header arguments for the proxy (we only support one response type).

Finally, all method calls are asynchronous following the usual Javascript convention.  The result of a call is an object with several useful fields:
  • status: the HTTP status code for the response.
  • headers: an object containing the HTTP headers returned on the response.  This is where you'll find the "expires" header describing the expiry time of the EVE server result.
  • obj: a Javascript object representation of the result.  The format of this object is determined by the schema specified in the Swagger specification.
  • data: a JSON encoded string representation of the result.
  • url: the URL to which the request was made.
In the example above, we convert "obj" to a string for display on the web page.

Example: Using the Proxy from Python

Now let's look at an example where we generate and use a Python client.  In this case, we'll need to generate the client using the Swagger Editor.  By default, the editor loads with a sample specification.  To load the specification for the proxy, select "File -> Import URL..." and enter "https://evexmlapi.orbital.enterprises/api/swagger.json":
Click "Import" then wait a few seconds for the editor to process the specification.

The left side of the editor will show the specification in YAML format.  The right side of the editor shows a view similar to the Swagger UI, including the ability to try the API directly.  Code generators are selected from the menu at the top.  For this example, we'll select "Generate Client -> Python".  This will download the Python client code.

The client download will consist of a zip file containing a single directory called "python-client".  Within this directory there is a setup.py and a README.md.  The setup file will install the library as "swagger_client" including any needed dependencies.  The README file gives sample code for using the library (note: the instructions in the README file do not instantiate ApiClient correctly.  See the example below for the proper way to create an ApiClient).

I'm not going to install the library for this example.  Instead, I'm going to just load and use the library directly from the "swagger_client" folder.  To do that, we need to make sure we have all the dependencies listed in setup.py.  In my case, I was missing urllib3 so I installed that.  Once you've installed all necessary libraries, it is straightforward to make calls to the API:

>>> # The local directory is "python-client"
>>> import swagger_client
>>> api_client = swagger_client.ApiClient()
>>> server_api = swagger_client.ServerApi(api_client)
>>> server_api.request_server_status()
{'online_players': 20845, 'server_open': True}

The call format is similar to the Javascript example except that the method name consists of "request_" followed by the name of the end point converted to lowercase and separated on word boundaries with "_".

By default, API calls are synchronous (this is shown in the example above).  To make an asynchronous call, define a callback function and pass it to the method call:

>>> def cb(obj):
...  print(obj)
...
>>> server_api.request_server_status(callback=cb)
<Thread(Thread-4, started 9632)>
>>> {'online_players': 22299, 'server_open': True}

Method arguments are passed as named arguments.  For example, to use the test server:

>>> server_api.request_server_status(server="https://api.testeveonline.com")

The result of a call is a typed Python object matching the schema defined in the Swagger specification.  You can find Python class definitions for these objects in the swagger_client/models directory

Parting Words

The proxy we've described here uses Swagger annotations to provide nice documentation and a convenient interface for ad hoc use of the EVE Online XML end points.  If you already have a favorite third party library, then this proxy may not be for you.  But if you're looking for a library, or if you're looking to use a language that isn't commonly supported, then our proxy with Swagger generated clients might be right for you.

0 comments:

Post a Comment