Geo Spaial Data Visualization

and Critical Metrics Predictions for Canadian Elections

Code | Paper

Problem Statement

  • Abundance of open-source election databases
  • Lack of concise and insightful data interpretation tools
  • User Inability to process information in a consistently comparable way

Proposed Solution

We proposed a data interpretation tool with following components:

  • Data Visualization
  • Trend Analysis
  • Prediction

Challenges

For this case study, we required to extract validated dataset of Canadian elections from different resources. The database developed by Dr. Anthony Sayers at the Department of Political Science at the University of Calgary is one of the most reliable, consistent, and accessible databases for Canadian elections.

  • No direct access to the election-dataset.
  • No uniform method to retrieve the data from server

Component: Scraper

For our data extraction task, we separate the Canadian Election Database webpages into two categories.

  • Static pages and
  • Dynamic pages.
  • The federal election part of the website loads static pages upon clicking on any of the link to a federal election.
  • The Provincial election part of the website makes the browser send an AJAX request to the server and dynamically update the webpage with the information received.
  • For the first phase of the data extraction, we store all the election data into different csv file in our server. We have used Beautiful Soup to extract information from the static pages and store them.
  • In the second phase of data extraction, we retrieve the information from the dynamically loaded webpages for provincial elections. Our second scraper function sends an AJAX request with appropriate parameters to the websites’ server on behalf of the browser. Upon receiving the AJAX request, the server sends all the associated response in a JSON file format to our function.

Component: Geo-Spatial Visualization

In the second component of the tool, we have generated Geo-spatial map using R script. We needed get the actual map data with longitude and latitude information. In the case of Canada, central statistics agency provided map shape files that we could use. The Shape File format (.shp) is the most widely-used standard for maps.

Conclusion

Open data is meant to be used by the public and provide data-driven decisions. However, visualization tools are required to make the data interpretable.

One of the other challenges to provide the analysis and visualization tools for open data is the different formats of the data that are published by various parties in separate databases.

In this paper, we provided architecture and the technical details of an open-source tool that we developed for collecting data, and visualizing and analyzing information.

Although the tool is developed explicitly for Canadian Election data, the technical details and the approach can be used by researchers from various fields and developers to address the issue of open data, such as having separate databases with no interpretation tool

Questions?

Ask

Presentation