Getting Started With Open Data and ASP NET MVC
Getting Started With Open Data and ASP NET MVC
WHITE PAPER | UTILIZING THE SOCRATA OPEN DATA API WITH ASP.NET MVC
Table of Contents
About the AuthorBrian McKeiver
Install SODA.NET
10
Introduction
10
Intended Audience
10
10
The Helper
11
Who is Socrata?
The Model
13
The Result
13
13
Paging
13
Filtering
14
Sorting
16
The Controller
17
Socrata Setup
The View
18
Troubleshooting
20
API Limits
20
Whats Next
21
Summary
21
About Socrata
21
WHITE PAPER | UTILIZING THE SOCRATA OPEN DATA API WITH ASP.NET MVC
Introduction
This series of walkthroughs and tutorials is designed to illustrate creating a web application
that works with government-based open data that is most easily accessed via tools powered by
Socrata, a company specializing in open data. The core of this exercise is to walk through what it
takes to setup a sample application, to connect to a remote dataset, and finally to consume that
dataset inside of a web application. Tools such as Visual Studio, ASP.NET MVC, and C# will be
used to create the application.
At the completion of the eBook you will be able to programmatically access government-based
datasets easily inside of a Microsoft technology platform-based application. Doing so will open up
a wealth of open data resources from governments, non-profits, and NGOs around the world.
When I was first asked to write this eBook, I was truly honored and humbled to do so. I will do my
best to give you all of the knowledge needed to understand the concepts of creating a Microsoft
ASP.NET application using the Socrata Open Data API.
The full source code of this eBook can be found at: https://github.com/mcbeev/
SocrataSodaNetSample
A live demo of the sample application can be viewed at: http://sodanetsample.
azurewebsites.net
WHITE PAPER | UTILIZING THE SOCRATA OPEN DATA API WITH ASP.NET MVC
Intended Audience
The intended audience of this eBook is experienced ASP.NET
developers who are new to using an open data dataset and
API. A developer interested in this tutorial should have the
following skills:
Familiar with an object oriented programming
(OOP) language
Familiar with ASP.NET web forms or MVC development
methodologies
Familiar with web development concepts (HTML,
CSS, JavaScript)
Familiar with JSON representation of data
Familiar with REST API design pattern and related
HTTP verbs
Familiar with Microsoft Visual Studio IDE
Who is Socrata?
WHITE PAPER | UTILIZING THE SOCRATA OPEN DATA API WITH ASP.NET MVC
http://data.cityofchicago.org/resource/uupf-x98q
The hostname of data.city.ofchicago.org is the main location
of the hosted dataset. Each city, state, or government agency
has its own. From there, all resources are accessed through
a common base path of /resource/ along with their dataset
identifier.
This paradigm holds true for every dataset in every SODA API.
All datasets have a unique 4x4 identifiereight alphanumeric
characters split into 2 four-character phrases by a dash. For
example, uupf-x98q is the identifier for business licenses.
The 4x4 identifier can then be appended to the /resource/
endpoint to construct the API endpoint.
5
Online: www.socrata.com | Phone: 206 340 8008 | Twitter: @socrata
WHITE PAPER | UTILIZING THE SOCRATA OPEN DATA API WITH ASP.NET MVC
WHITE PAPER | UTILIZING THE SOCRATA OPEN DATA API WITH ASP.NET MVC
Socrata Setup
One of the first things you will need to do is to create your
account on the Socrata Open Data platform.
WHITE PAPER | UTILIZING THE SOCRATA OPEN DATA API WITH ASP.NET MVC
Once the Manage link is clicked, you are taken to the Your
Applications page. From this page you can either create a
new application or edit an existing one.
WHITE PAPER | UTILIZING THE SOCRATA OPEN DATA API WITH ASP.NET MVC
Now that you have chosen your dataset and know a bit
about accessing an endpoint, lets work on creating a sample
application to put it all together. To do this, open up Visual
Studio 2013. Create a new project by following these steps.
WHITE PAPER | UTILIZING THE SOCRATA OPEN DATA API WITH ASP.NET MVC
Install SODA.NET
Once you have the library installed you have should now
be able to reference the main SODA.NET classes. The most
common namespace you will be using is SODA.
using SODA;
var client = new SodaClient();
10
WHITE PAPER | UTILIZING THE SOCRATA OPEN DATA API WITH ASP.NET MVC
The Helper
The code snippet below shows the main concept of creating
a client and returning data.
using
using
using
using
PagedList;
SODA;
System.Linq;
WebApplication.Models;
namespace WebApplication.Helpers
{
public static class SODAHelper
{
#region Constants
/// <summary>
/// App Token from Socrata registered app - get yours at https://opendata.socrata.com/login
/// </summary>
private const string _AppToken = 362kGXXXXXXXXXXXXjkozax;
/// <summary>
/// HostName of http://data.cityofchicago.org/resource/xj6s-q5eb.json;
/// </summary>
private const string _APIEndPointHost = data.cityofchicago.org;
/// <summary>
/// Socrata 4x4 Identifier
/// </summary>
private const string _APIEndPoint4x4 = xj6s-q5eb;
#endregion
#region Methods
/// <summary>
/// Gets sorted list of all Business Locations in the dataset by filter
/// </summary>
/// <param name=SearchQuery>Query to filter on</param>
/// <param name=PageNumber>Current page number</param>
/// <param name=PageSize>Number of items per page</param>
/// <param name=OrderBy>Column name to sequence the list by</param>
/// <param name=OrderByAscDesc>Sort direction</param>
/// <returns>object PagedList</returns>
public static PagedList<BusinessLocation> GetBusinessLocations(string SearchQuery, int PageNumber,
int PageSize, string OrderBy, bool OrderByAscDesc)
{
//Create client to talk to OpenDat API Endpoint
var client = new SodaClient(_APIEndPointHost, _AppToken);
//get a reference to the resource itself the result (a Resouce object) is a generic type
//the type parameter represents the underlying rows of the resource
var dataset = client.GetResource <PagedList<BusinessLocation>>(_APIEndPoint4x4);
//Build the select list of columns for the SoQL call
string[] columns = new[] { legal_name, doing_business_as_name, date_issued, city, state,
zip_code, latitude, longitude };
CONTINUED ON FOLLOWING PAGE...
11
WHITE PAPER | UTILIZING THE SOCRATA OPEN DATA API WITH ASP.NET MVC
//Column alias must not collide with input column name, i.e. dont alias city as city
string[] aliases = new[] { LegalName, DBA, IssuedOn };
//using SoQL and a fluent query building syntax
var soql = new SoqlQuery().Select(columns)
.As(aliases)
.Order((OrderByAscDesc) ? SoqlOrderDirection.ASC: SoqlOrderDirection.DESC,
if(!string.IsNullOrEmpty(SearchQuery))
{
soql = new SoqlQuery().FullTextSearch(SearchQuery);
}
}
}
return pagedResults;
12
WHITE PAPER | UTILIZING THE SOCRATA OPEN DATA API WITH ASP.NET MVC
The Model
The Result
using System;
namespace WebApplication.Models
{
public class BusinessLocation
{
public
public
public
public
public
public
public
public
public
Paging
Because open data datasets usually contain large amounts
of rows, showing everything on one web page may be
problematic. In the sample application for this eBook I
implemented a simple PageList type built for ASP.NET MVC.
To install the PagedList.MVC NuGet library, open up Visual
Studio to your solution. From the Tools menu, select NuGet
Package Manager and then Package Manager Console.
In the Package Manager Console window, make sure the
Package source is nuget.org and the Default project is <Your
Solution Name>, then enter the following command:
Install-Package PagedList.MVC
13
WHITE PAPER | UTILIZING THE SOCRATA OPEN DATA API WITH ASP.NET MVC
Once you have installed the package you simply have to add
a using statement to your C# class that returns a PagedList
collection, add an @model declaration line to your view that
you want to use the pager on, and link up the CSS file that
makes the pager look nice.
@model PagedList.IPagedList<WebApplication.Models.BusinessLocation>
@using PagedList.Mvc;
<link href=~/Content/Styles/PagedList.css rel=stylesheet type=text/css />
Filtering
To filter a dataset you have two options. The first option is a
FullTextSearch method the SodaClient object offers. This
method takes your query and translates it into the SoQL
syntax of:
http://data.city.org/resource/xxxx-yyyy.
json?$q=SearchForTermGoesHere
The full text search will perform a search through all columns
in a dataset for the value specified
The second way to filter a dataset is to use the .Where()
clause in your fluent C# SoQL call. For full information on the
options available, check out the queries documentation at
http://dev.socrata.com/docs/queries.html.
14
WHITE PAPER | UTILIZING THE SOCRATA OPEN DATA API WITH ASP.NET MVC
There are two parts to making the filter work. First the View
must have a simple HTML form as seen below:
<div class=row>
<div class=col-md-12>
@using (Html.BeginForm(Index, Home, FormMethod.Get))
{
<p>
Filter Name: @Html.TextBox(SearchQuery, ViewBag.CurrentFilter as string)
<input type=submit value=Search />
</p>
}
</div>
</div>
15
WHITE PAPER | UTILIZING THE SOCRATA OPEN DATA API WITH ASP.NET MVC
Sorting
In the screenshot below, is the dataset ordered by the
Legal Name column instead of the natural sort order. This
is handled on the backend of the code through the .Order()
clause of the SoQL query.
16
WHITE PAPER | UTILIZING THE SOCRATA OPEN DATA API WITH ASP.NET MVC
The Controller
The controller in the sample application is a pretty simple
one. The class handles setting up the configuration for
paging, sorting, filtering, and calling the helper class of
GetBusinessLocations. The only noteworthy part of this
code is the fact that the SortOrder parameter handles
which column the dataset is sorted by, as well as which sort
direction is going to be used.
Since the main focus of this eBook is about using the SODA.
NET API this code is a good example of how to simply call out
to the SODA API and return a PagedList collection, the rest is
pretty boilerplate code.
using System.Web.Mvc;
using WebApplication.Helpers;
namespace WebApplication.Controllers
{
public class HomeController : Controller
{
public ViewResult Index(string SortOrder, string CurrentFilter, string SearchQuery, int? CurrentPage)
{
//Handle Paging
int pageSize = 20;
int pageNumber = (CurrentPage ?? 1);
//Handle Sorting by Column
if (string.IsNullOrEmpty(SortOrder))
{
SortOrder = ;
}
ViewBag.CurrentSort = SortOrder;
ViewBag.NameSortParm = (SortOrder == legal_name) ? legal_name_desc : legal_name;
ViewBag.DBASortParm = (SortOrder == doing_business_as_name) ? doing_business_as_name_desc :
doing_business_as_name;
ViewBag.ZipSortParm = (SortOrder == zip_code) ? zip_code_desc : zip_code;
ViewBag.IssuedSortParm = (SortOrder == date_issued) ? date_issued_desc : date_issued;
bool sortAsc = (!SortOrder.Contains(_desc)) ? true : false;
//Handle Filtering
if (!string.IsNullOrEmpty(SearchQuery))
{
CurrentPage = 1;
}
else
{
SearchQuery = CurrentFilter;
}
ViewBag.CurrentFilter = SearchQuery;
if (!string.IsNullOrEmpty(SearchQuery))
{
SearchQuery = SearchQuery.ToUpper().Trim();
}
Online: www.socrata.com | Phone: 206 340 8008 | Twitter: @socrata
17
WHITE PAPER | UTILIZING THE SOCRATA OPEN DATA API WITH ASP.NET MVC
The View
Once the data is returned to the View, there is little work left
to display the table on the web page. Basically, the models
data is bound to a row in a HTML table via standard ASP.
NET MVC practices. Note that the correct declarations were
applied to make the PagedList collection work correctly at the
top of the page then render the PagedListPager html element
below the main table thanks to the PagedList.MVC library.
@model PagedList.IPagedList<WebApplication.Models.BusinessLocation>
@using PagedList.Mvc;
<link href=~/Content/Styles/PagedList.css rel=stylesheet type=text/css />
@{
}
<div class=row>
<div class=col-md-12>
<h2>Chicago CrossFit Gyms<small><a href=https://data.cityofchicago.org/Community-Economic-Development/
Chicago-Crossfit-Locations/xj6s-q5eb target=_blank><i>source data</i></a></small></h2>
</div>
</div>
<div class=row>
<div class=col-md-12>
<hr class=compact />
<table class=table table-striped table-hover >
<thead>
<tr>
<th>
@Html.ActionLink(Legal Name, Index, new { SortOrder = ViewBag.NameSortParm,
CurrentFilter = ViewBag.CurrentFilter })
</th>
<th>
18
WHITE PAPER | UTILIZING THE SOCRATA OPEN DATA API WITH ASP.NET MVC
19
WHITE PAPER | UTILIZING THE SOCRATA OPEN DATA API WITH ASP.NET MVC
Troubleshooting
API Limits
Another note worth mentioning is Socrata and the Socrata
Open Data portal does have a set of API limits. You can make
a certain number of requests without an application token,
but they come from a shared pool and youre eventually
going to be cut off. When you include an App Token that limit
is raised.
The API Limit is 1000 per hour when using an App Token
20
WHITE PAPER | UTILIZING THE SOCRATA OPEN DATA API WITH ASP.NET MVC
Whats Next
Summary
This eBook has shown you how to use the Socrata Open Data
API with ASP.NET MVC from a data consumption standpoint.
The API does also allow for creating new datasets, adding
items into datasets, and updating items inside of datasets.
For more on how to accomplish these advanced tasks, please
see the Socrata Developers site at: http://dev.socrata.com/
publishers/getting-started.html
About Socrata
Socrata is the world leader in cloud solutions for open data and data-driven governments. Its innovative customers include the
cities of New York, Chicago, San Francisco, Los Angeles, Melbourne and Eindhoven; the states of New York, Illinois and Texas;
US Health and Human Services; Centers for Medicare & Medicaid Services; the UN and the World Bank. Socratas solutions
including the recently launched Open Data Network which unleashes the full potential of government data to help drive
connected communities around the worldassist government leaders in improving transparency, modernizing citizen access to
information and bringing data into every decision, all with unprecedented speed and cost savings. Delivered as turnkey services,
Socratas technologies unlock data trapped in enterprise silos, mobilize and transform it into useful information that everyone
can easily access, visualize, share and reuse. To learn more about Socrata, visit www.socrata.com.
21