Saturday, November 14, 2009

Spatial / Temporal MBTA HeatMap

MassDOT issued a visualization challenge, where it is calling on developers to use released CharlieCard data to visualize "A Day in the Life of the MBTA". The following three applications use the Flex API for ArcGIS as a response to this challenge. The first application (click here to view it) consists of two maps. The first map contains a dynamically generated heat layer that morphs based on the selected hour. The second map is an isometric projection, but the T lines are represented in a schematic layout rather than a geographical layout. A schematic layout is what you see when you step up to a metro map at the station. You, the user, can rotate the schematic map by holding down the left mouse and dragging it left or right (no zoom in, sorry). At the bottom of the application is a horizontal slider. As you drag the slider, you are changing the "current" hour and minutes of the day. On the geographic map, the heat layer will reflect the CharlieCard counts per station. As to the isometric map, bars will extrude from every station, where the height of the bar is proportional to the CharlieCard count. There is a PLAY button, to play the hours of the day for you to step back and watch. The second application (click here to view it) is a simplified version of the first, where I embedded the clock in the map and removed the isometric map. This was nice and uncluttered, perfect for a dashboard on a large screen, say at the MBTA headquarters (hint, hint :-). The third application (click here to view it), is the same as the second one, but I added a table that lists all the active stations in descending count order.

Now, let me tell you how I built these applications. As I mentioned earlier, These are all based on the Flex API for ArcGIS. The base map image tiles are retrieved from ArcGIS Online. The geographical and isometric lines and stops are read from embedded shapefiles. The hour statistics are post-processed using ye-good-olde Excel from the MBTA released data. The result is a tab separated fields file, where each line in the file has the station name, the hour, and the sum of CharlieCards for that hour at that station. Again, that file is embedded in the application. Might take a bit of time to load the application, but all the spatial and temporal information is now on the client for local manipulation, after all this _is_ an Rich Internet Application.

What enabled me to quickly create three views of the same application is what I would like to talk to you next and is the guiding principles that I use to develop most of my "simple" applications. Three letters: M, V, C. You are now saying to yourself "yes, yes, what framework did you use ?" Actually....none! I can hear now. "Oh no, the (nasty word)... created _yet_ another framework". Actually I did not. I do have to admit that I did look at the usual suspects (Cairngorm, PureMVC, Swiz, Mate) but really did not need the extra swcs and the weight. What I _do_ need, is the MVC philosophy. But, can I do it with just the Flash and Flex provided classes? Yes, so here is the micro architecture and thought process. The Model is a singleton (O' oh, Singletonitis... hold on, hear me out) that holds the application state. It contains [Bindable] properties. Views (subclasses of UIComponent) are bound to the Model using "{}" in MXML (one of the beauties of Flex) or through ChangeWatcher instance in AS3. If a view or a non-view element wants to change the Model, it dispatches an event encapsulating all the values to change the Model. A controller instance will be listening for that event type and in its event handler code will process the encapsulated values and modifies the Model. As the view is bound to the Model, then the view will reflect any changes in the matter that it desires such as a set of map graphics or rows in a data grid. And the process (create an event, dispatch the event, controller listens for the event, processes the event and modifies the Model, view is bound to the Model and changes) repeats itself, over and over. You _will_ be temped to modify the Model directly from the view, but that will be an instant gratification solution. As the application development progresses, you _will_ find yourself needing to encapsulate that Model modification, thus the introduction of a controller. So, my advise is do it right from the beginning. Now, to enable that central event dispatching, we take advantage of the Application being an instance of EventDispatcher. To not confuse the Flex events with my events, all my event types are suffix with "$" such as "complete$". Last but not least, is the instantiation of the controllers and making them event listeners. As a matter of convention, all controllers names are prefixed with the event type, such as "CompleteController", and are declared as immediate children tag to the mx:Application tag in the MXML. For example:


What does this mean? This tells the runtime engine to create anonymous instances of these controller classes. Each controller should have an empty constructor that registers it as a listener to the event to process and modify the Model.

public class CompleteController
public function CompleteController
Application.application.addEventListener( "complete$", handler );
private function handler( event : Event ) : void
// Process, process....modify Model.

With this process in mind, you now are structured to take every application feature and break it down into these three pieces which makes you focus on the task at hand. In addition, with unit testing (another blog topic) feature closure will emerge enabling you to move on to the next task. Yes, you do become what I call a "code monkey", but this is where the usage of for example TextExpander on my mac can automagically generate code based on a template. Side Note - One of these days, gotta write an eclipse plug-in :-)
Finally, have to give credit where credit is due, as I would have not been able to put this application together so quickly without "borrowing" code from:

Edwin van Rijkom for the shapefile library.

Juan Sanchez for the clock.

Michael VanDaniker for the heat map.

Keith Peters for the isometric code.

Nahuel Foronda for the Brownie skins

If I missed any person - it was not intentional. And like usual, you can download the source code from here. Hope you enjoyed this and tell me what you think.


Adam Mollenkopf said...

Excellent work Mansour!!!

ThunderHead said...

Glad u like it :-)

Sir Don of Bedford said...

Excellent work

odoenet said...

Thanks Mansour. Great "clean" MVC example.

Greg Knight said...

Awesome my friend...

Willy said...

Excellent as usual :)
I've tried to extract the layer map to put on my map and succeeded :) however where I'm zooming some of the heat point seems to be overlapped by the map tile... for example just half of point show up.
Assuming that I've not too much experience here I could have missed something but my guess was the layer order should not change so the heatmap should be always on top of tile map.

ThunderHead said...

Hum - somebody else reported this :-( will look more into it

Harley said...

Wow! really great.

I was just asking my self this morning, how the heck can i write better code?


ThunderHead said...

Hope it will help :-)

andregastelum0104 said...


Ole said...


chris said...
This comment has been removed by the author.
datadriven said...

This is very cool, but wouldn't the tool be more useful if the clock had numbers and if AM and PM were clearly labeled. It would also be useful if the Play progress bar had time of day scale so you could easily jump to time of day that is of interest. And it would also be useful to see a legend or a dynamic readout of raw data at particular stops. As it is, then meaning of different colors of the heat map, in terms of say, number of riders, is not clear.

Ravi said...

Cannot see your example on your site.
Please help...