Alastair Callum Unity Developer

Unity & Graph Maker

Make something already!

Ok, so we're gonna be pulling down loads of information from the CREST API so it's going to have to be displayed in a sensible way and there's nothing more sensible than graphs. Graph Maker is a fantastic plugin for Unity that allows the creation of fully dynamic 2D graphs and charts, honestly, the work put into this is just incredible - oh and it's fully supported by the new Unity UI framework.

The first thing to do is set up a graph ready for holding the information. In this case a line graph. The graph contains a whole load of configuration options for auto scaling axis, colour, grid display, labels, legends - anything you can think of. After the required series where added to the graph object (it's a good idea to break the prefab instance so you don't overwrite it!) you can start populating it with data!

From our previous progress, the history has been deserialised into the history object (RootObject). The following populates each series by firstly clearing any current values, and adding one for each history entry with the average, min/max price, and volume.

series1.pointValues.Clear();
series2.pointValues.Clear();
series3.pointValues.Clear();
series4.pointValues.Clear();

int _i = 0;
foreach (Item _item in hist.items) {
    series1.pointValues.Add(new Vector2(_i, (float)_item.avgPrice));
    series2.pointValues.Add(new Vector2(_i, (float)_item.highPrice));
    series3.pointValues.Add(new Vector2(_i, (float)_item.lowPrice));
    series4.pointValues.Add(new Vector2(_i, (float)_item.volume));
    _i++;
}

Frustratingly GraphMaker uses the Vector2 type to store it's point values - but this causes problems when casting the Double values from the history into floats; this could be a problem because of EVE's currency commonly enters into the billions. I may look at converting the Vector2 types to a custom container that supports Doubles.

Anyway - when the application runs the graph is populated with the desired data. Nice.

undefined

And with a little time configuring the various GraphMaker prameters, we have a pretty snazzy looking graph. The bar chart at the bottom is a second graph with one series and only 200px high. Both Y axis auto expand or shrink based on the data provided, maximising the real-estate of the graph area. The X axis, time/date, is currently fixed to the 12 month period, in time this will be configured dynamically based on the search period.

Links in this post

JSON to C#

You shall not parse!

With a knowledge of the CREST API endpoints, requests can start being made for data - in Unity this is as simple as initialising a new WWW object with the specified endpoint URL. For example, a request for the history of an item on the market can be made with the following endpoint:

_www = new WWW("https://public-crest.eveonline.com/market/10000002/types/34/history/");

This will return a rather impressive JSON object containing nearly 400 entries for the price history of Tritanium in a market region known as The Forge. The problem here is that this string is pretty useless without being restructured in memory, the solution then is to parse or deserialise it into an object oriented .NET structure.

There are many ways to do this, SimpleJSON is a parser and builder written in C# but relies on knowing what index you are after. .NET's JSON Serialisation does a similar thing but converts directly to and from .NET Framework types. For the amount of data returned from the CREST API the latter solution would be preferred; being able to navigate a class structure representative of the JSON string.

And how about Unity?

Unity 5.3 comes to the rescue again, in this release the engine can now serialise and deserialise classes and structs marked with the System.Serialisable attribute. Unfortunately the class or struct type is needed before the deserialisation can be done with the Unity JsonUtility.

public static T FromJson<T>(string json); 

So how can the desired class structure be created when we do not know the structure of the source data? The Unity Manual suggests the following:

Deserialize the JSON into a class or struct that contains ‘common’ fields, and then use the values of those fields to work out what actual type you want. Then deserialize a second time into that type.

Not very helpful. Please welcome json2cshap.com. This tool will create the C# class structure from a JSON string - there's only a couple of things that need to be looked at.

By default the tool will create the members of any classes in the JSON string as properties, with getter and setter attributes. This appears to dissagree with Unity, so remove any getters/setters from the class and leave them as fields. The System.Serialisable attribute needs to be added to the classes too.

[System.Serialisable]
public class RootObject {
    public string totalCount_str;
    public List<Item> items;
    public int pageCount;
    public string pageCount_str;
    public int totalCount;
}

[System.Serialisable]
public class Item {
    public string volume_str;
    public int orderCount;
    public double lowPrice;
    public double highPrice;
    public double avgPrice;
    public double volume;
    public string orderCount_str;
    public string date;
}

This makes it possible to deserialise the item history into the above class structure using the code below.

RootObject _history = new RootObject();
_history = JsonUtility.FromJson<RootObject>(_www.text);

By default the json2csharp.com tool will use generic class names such as RootObject and Item, all that needs doing is changing these to something more semantic such as ItemHistory and HistoryEntry for example.

Links in this post

CREST API: Making the first request

Is anyone there?

The CREST API has a fistful of endpoints available to 3rd party developers. Interestingly, sending a request to the root endpoint returns a JSON string of all available endpoints; pretty handy.

https://public-crest.eveonline.com/

Most browsers will return this object as a download as there is no filetype associated with the response. When sent through an application however, the string can be read as normal. This stuff is pretty straight forward as all that needs doing is identifying the desired endpoint, some examples are outlined below.

https://public-crest.eveonline.com/market/10000002/types/34/history/

Will return the history of the item ID 34 (Tritanium) from a market region ID 10000002 (The Forge).

https://public-crest.eveonline.com/market/10000002/orders/sell/?type=34/

Will return all current sell orders of the item ID 34 (Tritanium) from a market region ID 10000002 (The Forge).

More information can be found on the EVE Online wiki.

Anything else?

The CREST API is still being implemented so a lot of the endpoints do not work, but in future it is planned to give access to any public game data from characters, industry, market, corporations and alliances. There are also plans for an authentication only (OAuth) API for account private account data.

Rate Limits

Rate Per Second: 150
Burst Size: 400
Maximum concurrent connections: 20

For your requests, this means you can send an occasional burst of 400 requests all at once. If you do, you'll hit the rate limit once you try to send your 401st request unless you wait.

Your bucket refills at a rate of 1 per 1/150th of a second. If you send 400 requests at once, you need to wait 2.67 seconds before you can send another 400 requests (1/150 * 400), if you only wait 1.33 seconds you can send another 200, and so on. Altrnatively, you can send a constant 150 requests every 1 second.

EVE Online Static Data Export (SDE)

Show me what you've got!

EVE Online does something wonderful with it's data, it doesn't hide it away behind fancy graphics and menus; it puts it on display for the whole universe to see. Almost everything you see in many of EVE's user interfaces is pulled from a database with ID's, put in a table for you to filter and analyse as you see fit.

However these ID's are not shown in the game, instead their name and description are used to identify items in a market search for example. Requests to the CREST API require an ID, so how can we locate the ID of a given item name such as the common mineral Tritanium? If you've spent any time with EVE APIs you'll probobly know this as 34 - but what about Kernit? or Heavy Missile Launcher I?

The solution here is to make use of EVE's Static Sata Export (SDE). A 170mb database of all the items, stations, solar system, regions, and corporations that exist in the game. Information about the SDE can be found on the EVE Developer Resources. They have also been generous enough to allow a download of the Image Export Collection (IEC), allowing 3rd party developers the ability to show icons of the items in their applications.

Making use of this database would allow a developer to locate the ID for a given item, Kernite - 20, Heavy Missile Launcher I - 501.

EVE Online CREST API

Developer or Gamer?

As a developer it becomes difficult to see games in the same way as gamers. The perfect example of this for me is EVE Online - the fantastically technical MMO. I've tried to play it many times - and like many others found it difficult to really get into; now this isn't a criticism of the game, I guess it's just not my play style. But what I do love is it's technical magnificence.

I've always been fascinated by the 3rd party applications people have made to monitor, estimate, and plan their entire progress in the game. The very fact there has been such a direct link between online and offline play goes to show how well structured (you'd hope) the game is.

I won't lie, I've been out of the loop for some time but recently saw a trailer for an expansion coming Spring 2016; EVE Citadel. So I found my account, logged in and flew around a bit (trying to remember where I left off) and then logged out again. It's just not me. So instead I started looking at the updates CCP have made to the game since I've been away and began reading about the new CREST API.

For most people it might not sound very exciting, but to me, this was my opportunity to get into (quite literally) EVE Online.

CREST API

There's not a whole lot of information out there, but the CREST API is essentially a direct link to the game's server, Tranquility, allowing much more up to date information requests.

Most details come from the Third Party Developer Blog, and the EVE Wiki:

The Carbon RESTful (CREST) HTTP API is a read and write API to the EVE Universe game cluster. It gives the ability to interact with the cluster in a scalable way from clients/applications other than the EVE game.

The API returns JSON strings based on the requested endpoint. In the past 3rd party applications relied on 3rd party APIs to request data and these are often only updated at certain times of the day. With the ability to requst data directly from the game server, the CREST API offers new opportunities for 3rd party developers.

Newer posts → Home