Learning Python, JSON and Network Graphs in Middle Earth

Learning Python

As a data analyst early in my career, I haven’t had a chance to work with many programming languages. Most of the work I have done has been using SAS, so when given the chance to have a go with Python I jumped at the opportunity.

The immediate problem that hit me, was that I’m quite messy with my code and don’t stick to convention very well… so finding out that Python was both case and indent sensitive was a worry! I went through the basics using a course on Kaggle, which was good because after each lesson there was a Jupyter Notebook for you to solve problems by writing your own code, allowing you to put what you learnt immediately into practice.

Following brushing up on the basics, I dived into the deep end, producing some networks from JSON data. Python is an open source programming language which comes with a couple of advantages, there are lots of modules available to install for free, and also there is the answer to any problem you’re having somewhere on the internet. 

JSON

JSON stands for JavaScript object notation, and is a way of storing and sending data. It’s the main way that data is sent in application programming interfaces (APIs), so I first came across it a while ago while attempting to do analysis on Fantasy Premier League data. 

It’s initially very daunting as JSON data looks nothing like other formats of data that I was used to. Once I got my head round it, it’s a very logical way of storing data and the main benefit is that it’s all text which means it can be used and stored in a large variety of ways. 

Network Graphs in Middle Earth

I can’t share the network work I was originally doing, so decided to demonstrate with an example. I utilized a module called NetworkX to analyse networks.

In Middle Earth, it’s 3017 and Gandalf and Aragorn are preparing for the return of Sauron. They have a sense that in their midst there is a traitor, so they hack Sauron’s phone to see who he has been in touch with, while also tracking the phone of the suspicious Denethor and downloading their own phone data. They decide to hire Butterfly Data to help them to analyse this data.

This comes as JSON data, which was analysed in Python. Every single event was reduced to the caller and the recipient, and this was studied as a network, allowing visualisation of who had contacted who (for all data and Jupyter Notebook have a look at our Github).

Upon sending this network graph back to Aragon and Gandalf, they realise their old friend Saruman has been in touch with Sauron, and is the traitor in their midst. They move quickly to arrest Saruman, no hobbits are captured, Helms Deep is never put under siege and the One Ring is easily destroyed as all the powers of Middle Earth unite together, all thanks to the insight provided by Butterfly Data.