AS3 Socket Issues With Large Data on MacOS

So... I was writing an MXML/AS3 application that uses a Socket class instance to consume large amounts of data from a Java socket server program. The server is supposed to write very rapidly 1MB data chucks to a connected AS3 client. Now the funny part is that when the packet size is less that 64K, the server would write these packets very quickly, but when it was writing 1MB chucks, it would "stall" between writes while the client was reading the data in 64KB chuncks. So, I start pulling my old TCP tuning trick back from my 'C' days to tune the socket server. But no difference. I can only tune the server side as the client AS3 socket class does not offer any tuning properties and I'm assuming that the folks at Adobe would implement sockets correctly. It is this last statement that kept nagging me. "How often have I said to you that when you have eliminated the impossible, whatever remains, however improbable, must be the truth?" (Sherlock Holmes). So I wrote a simple Java socket client to test my server and guess what...no stall, A-Ha ! So there _is_ an issue with AS3 Sockets...but still...could not believe it, I mean...my MacBook Pro is used for development, God knows how many libraries I've installed and I have two VPN clients and again, God only knows what they setup in the kernel to make the tunnel. So, I took my Java server jar and my AS3 AIR application and installed them on a "clean" MacBook. Started the server and then the AIR application, and guess what....same result. The server stalls between writes as the AS3 Socket client is reading in 64K chucks the data. Now this is messed up. Dusted my old Dell with XP, installed the latest updates (that took forever) and then dropped the java server jar and AIR application. Started the server and then the AIR application, and guess what ? NO STALL. No way....my trusty Flash Player implementation on MacOS has issues. Just to make sure, started the server on XP and AIR on MacOS; stall. Started the server on MacOS and AIR on XP; No Stall.
I'm writing this to share my experience, the binaries and the source code hoping that somebody out there tells me that I've done wrong or if there is indeed an issue.
Please test it yourself. You need to have Java and AIR installed. The AIR package is here and the server jar is here. The java source code is here and the FlashBuilder project is here.
From a command line, start the java server as follows:
$ java -jar SocketServer.jar
As to the client, just double click on the installed application. I'm printing to the console the number on packets that have been sent. On my Mac, I see a print statement every one second. On my XP, all is done in one second.
Thanks Y'all - in the meantime will do my development off my XP. Happy New Year.

How to use Esri Flex API on Android and iPhone

Update (Jun 2011) : For more updated info, check out my ADC article here.
At MAX2010 (Awesome BTW) Adobe introduced Flex SDK Hero and Flash Builder Burrito which enables you to build and deploy Flex mobile applications on Android by default. In addition, using the revitalized Packager for iPhone, you can postprocess the swf into an ipa that you can deploy on an iPhone using iTunes.

In this post, I will share with you the steps that I took to use the Flex API for ArcGIS to build a mobile mapping application.

First and foremost - RTFM - here are some links that u _need_ to read specially for iPhone provisioning:

http://opensource.adobe.com/wiki/display/flexsdk/Hero
http://labs.adobe.com/technologies/flashbuilder_burrito/
http://labs.adobe.com/technologies/packagerforiphone/

The cool part of this whole process is now you have one code base to build upon and deploy on Android and iPhone and in the future whatever mobile platform Adobe decides to support.
You can download the source code of the sample application from here.
This simple application queries the states demographic layer in arcgis.com and displays the result as a list. By taking advantage of the new state syntax in Flex 4, if the user rotates the phone to view the info in landscape mode, the list is replaced with a map showing the selected states. Cool ?
There is no new Flex API for ArcGIS to learn. I'm using the same old Query, QueryTask, Map, ArcGISTiledLayer, GraphicLayer and SimpleFillSymbol classes from the released Flex API.
Now, to compile the application using Flash Builder Burrito, make sure to include in the libs project folder the flexapi swc and make sure to copy from the Flex sdks folder (under the burrito application folder) the mx.swc and the sparkskins.swc files. For the next release of our swc will not require the last two swc, just add them for now. Your package explorer in Flash Builder should look like this:

Now you can compile, deploy and run your swf on the USB attached Android. Make sure that you have the latest AIR runtime on your phone. If not, then delete AIR using the Settings view and redeploy the application, the latest runtime engine will be automagically installed.

Now to the iPhone. Currently, Burrito does not have a built-in wizard to postprocess the swf into and an ipa. This must be done via command line tools. So, download and install the latest iphone packager for your platform (windows or mac). In my case, I had to re-compile the source code to reproduce the swf. Here is the content of my bash shell script:

FLEX_HOME="/Applications/Adobe Flash Builder Burrito/sdks/4.5.0"
"${FLEX_HOME}/bin/mxmlc"\
-load-config "${FLEX_HOME}/frameworks/airmobile-config.xml"\
-compiler.include-libraries+=libs\
-sp src\
-o bin-debug/MyMobileApp.swf src/MyMobileApp.mxml

Next is to convert the swf to an ipa. Here is the content of my shell script:

~/packagerforiphone_v2_mac_101110/bin/pfi\
-package\
-target ipa-test\
-provisioning-profile ~/YourProfile.mobileprovision\
-storetype pkcs12\
-keystore ~/YourFile.p12\
-storepass YourPassword\
MyMobileApp.ipa\
bin-debug/MyMobileApp-ios.xml\
bin-debug/MyMobileApp.swf\
bin-debug/assets/Default.png\
bin-debug/assets/Icon29.png\
bin-debug/assets/Icon57.png\
bin-debug/assets/Icon512.png

Here is the content of MyMobileApp-ios.xml:

<?xml version="1.0" encoding="UTF-8"?>
<application xmlns="http://ns.adobe.com/air/application/2.0">
<id>ThunderHead</id>
<filename>MyMobileApp</filename>
<name>MyMobileApp</name>
<version>v1</version>
<initialWindow>
<renderMode>gpu</renderMode>
<content>bin-debug/MyMobileApp.swf</content>
<fullScreen>true</fullScreen>
<aspectRatio>portrait</aspectRatio>
<autoOrients>true</autoOrients>
</initialWindow>
<supportedProfiles>mobileDevice</supportedProfiles>
<icon>
<image29x29>bin-debug/assets/Icon29.png</image29x29>
<image57x57>bin-debug/assets/Icon57.png</image57x57>
<image512x512>bin-debug/assets/Icon512.png</image512x512>
</icon>
<iPhone>
<InfoAdditions><![CDATA[
<key>UIStatusBarStyle</key>
<string>UIStatusBarStyleBlackOpaque</string>
<key>UIRequiresPersistentWiFi</key>
<string>NO</string>
]]></InfoAdditions>
</iPhone>
</application>

Again - if you want to know what all the above options mean - you _must_ read the packager documentation. The pfi command will run for a long time - do not worry - it will eventually finish - again, this is still in its early stages, and I hope that Adobe will make this easier and faster in the future.
Now drag the ipa file onto iTunes and sync your iPhone. you should in theory have the app on your iPhone.
I hope this works for you and tell me what you think.

Dynamic Legend Generation From An Image Map Services

You can see the national weather warnings that NOAA publishes here. If you look at the page, you will notice that _all_ the legend swatches and text are presented under the map and you are left to your own means of locating the associated colors between the map and the legend grid to figure out what is on the map.
It would be nice if the legend grid could be dynamically generated to list only the swatch colors and their descriptions based on what is on the map. This is what this post tries to solve.
So NOAA publishes the national weather service maps as Esri ArcGIS REST endpoints. Using an instance of an ArcGISDynamicMapServiceLayer subclass, I can intercept the reponse image. Once I have the image bitmap data, I can get a histogram of the color distribution. Now using NOAA color map, I can associate a color with a description and populate a legend UI component. Check out the application in action here. The legend located on the top right corner is not fully transparent, roll the mouse over it to make it fully opaque. And like usual, you can download the source code from here.
Update: Looks like my usage and processing of the histogram does not lead me to a unique set of colors. So I reverted back to good old fashion looping through the pixels and creating a unique bucket of colors. Works pretty fast and more importantly correctly :-) A colleague at Esri was wondering if you can click on the map and popup an info window containing the label of the weather at that location, so I updated the code to handle this request - Thanks Mehul.

Image Flare Symbol For Clustered Graphics

This is a long overdue post following the clustering post where I wanted each flare element in a cluster symbol to flare to a different image symbol when the flare expands out. In addition, the default flare symbol expands upon a rollover mouse event, by me giving you the source code, you can customize it to expand on click, double click, shift click, whatever you want - you have the source code. The same applies to the flare elements. The ImageFlareSymbol has the same properties as the basic FlareSymbol but has two additional properties (backgroundWeights, backgroundColors) to color code the clusters based on their weight value. Each cluster is composed of one or more Graphic instance. In this implementation, each graphic must be symbolized with an instance of ImageFlareElementSymbol. The latter has a property ‘href’ that reference a bitmap url that will be displayed as a flare element. Here is a sample application to show the image flare in action. And like usual, here is the source code.
Tweet

Weighted Map Feature Clustering With Attributes

Was asked if the WeightedClusterer can cluster features not just by proximity but by proximity and attribute property value. The idea is that there should be at least one cluster per unique attribute property value. Let me explain further:



The above snapshot is a clustering of cities based on proximity and country - note that despite the proximity of Belgium and The Netherlands, two distinct clusters have been created one for BE and the other for NL - now for France - there is a cluster containing one feature over Paris, however due to the distance from Paris (at that scale) a cluster was created based on 3 cities in the south of France. Make sense ?

In the implementation of the AttrClusterer class that extends the ESRIClusterer class, we override the clusterGraphics function. This function is invoked indirectly by the Flex framework during the update of the display list. In this function, we overlay a virtual grid on the map. This grid acts as a spatial index. Features that are in the same cell will be initially clustered together. Next a process of searching and merging will be executed till no cluster overlaps another cluster. Let me explain this process; for each cluster, search the immediately adjacent cell for any overlapping cluster (that is why we need that spatial index grid :-). If a neighboring cluster is found, then the two cluster merge into one cluster and the location of the new cluster is based on the weight of each original cluster. the new cluster is spatially indexed and the original two clusters are removed. BTW, this process is guaranteed to converge to a steady state solution, so there is no need for a terminal condition like a counter. Now something of importance here is the index used to locate neighbors. The spatial index is implemented as a hash map, the cluster key is a combination of the cluster position relative to the center of the map and the cluster attribute - remember - that what started this whole implementation. You can see the application in action here. I’ve loaded the top 1000 cities by population from geonames. And like usual, you can find the source code here.

Merging Table Of Content and Legend Info

Initially, while working on Gulf Of Mexico Response Map, the legend and table-of-content were two separate windows. That took too much real-estate, and a TOC layer was hard to correlate with the legend swatch by name only. So I set upon merging the legend window and the table-of-content into one window. There is plenty of TOC code as a visual tree component, but it is the legend UI that causes us trouble as there is no REST endpoint to get the layer swatch information. Well with the advent of FeatureLayer somebody can argue that this could be done. But you still have to go back and forth for each layer and get the rendering information. BTW, this will be remedied in 10.1, as a new REST endpoint will be introduced to get _all_ rendering information of a map service. In the meantime, I had to do something. As you can imagine the project map services were very dynamic and the layer content was changing constantly, so having a static legend is out of the question. There were attempts to create an ArcObject proxy, where the client application connects to that proxy who issues a SOAP request to an ArcGIS server for the legend info. The SOAP result is converted to JSON and forwarded back to the client for decoding and rendering. The code was written in .Net and have a Java based server. I wanted to be platform independent and have a two tier architecture (at least from a 10,000 feet perspective) and the introduction of a proxy was out of the question. So....I have to issue SOAP calls directly from the client to ArcGIS server. This is where the advent of FlashBuilder with its data connectivity option was a blessing. FlashBuilder can generate correctly stub code for ArcGIS SOAP methods based on a map service WSDL. This was impossible in FlexBuilder. Yes, it does bloat the code base a bit, but it is a small price to pay for such a dynamic system. The TOC code is based on the FlexViewer TOC code. I modified the item renderer to be of variable height to accommodate the renderer swatches as child elements. You can see the application in action here. And like usual, you can download the source code from here. Small note: First time through, it take a few seconds to get the swatches, so if you open the tree, it will look “plain” and somehow AGS returns blank swatches for tiled map services. not sure why ! have to ask the core team about that. Oh one more thing, in addition to adding the Flex API for AGS swc, make sure to include the Adobe provided fiber swcs. Until 10.1 (hopefully soon) hope this can help.

PS: If you have a 10 server - there is a REST way to get all the layers published in a map server that could be used to create swatches from the rendering information.

Pixel Bender For Map Projection

A customer came up to me at the User Conference and asked me if I can apply the same overlay method that I used in my OverlaySymbol application to position a geographic referenced image on a web mercator tiled base map; since esri is migrating most of its published maps on arcgis.com to web mercator. After talking it over with my friend Kerry (who is the final authority when it comes to map projections), he recommended that this should be a pixel by pixel operation due to the exponential latitude elongation nature of web mercator. So basically the algorithm is; for each output mercator pixel, convert it back to geographic, find out in the geographic image source what is the pixel color at that geographic location and assign that color to the output mercator pixel. Straight forward, I mean the web mercator transform is well known. I just have to do it a million time, and... that is when I started panicking. A million time loop in AS3 - that will take forever (well actually half a second) in user space. Then I remembered PixelBender. That is what it does for living very well - massively process pixels at the GPU level. So I downloaded the toolkit, went over the tutorials (all awesome) and I started writing my own filter. PixelBender can operate in a filter, blender or filler mode. It is the last mode that I'm interested in, as it enables me to fill any region in a display object based on a set of parameters. So I approached this overlay method differently than my previous experiment. I created a layer rather than an symbol to overlay the geographic image. The source of the image will be a property of the layer. The layer will load the compiled PixelBender filter code and on an extent change, PixelBender will be invoked to fill the layer graphics context based on the image source and extent as parameters. And it is fast :-). The following is a snippet of the mercator filter source code:

void evaluatePixel()
{
float2 pt = outCoord();
pt.x = pt.x * targetWidthFact + targetXMin;
pt.y = (targetPixelHeight - pt.y) * targetHeightFact + targetYMin;

pt.x = degrees(pt.x / 6378137.0);
pt.y = degrees(2.0*atan(1.0) - (2.0 * atan(exp(-1.0 * pt.y / 6378137.0))));
if( pt.x < -180.0 || pt.x > 180.0 || pt.y < -90.0 || pt.y > 90.0)
{
  dst = float4(0.0,0.0,0.0,0.0);
}
else
{
  pt.x = (pt.x - sourceXMin) * sourceWidthFact;
  pt.y = (sourceYMax - pt.y) * sourceHeightFact;
  dst = sampleNearest(src,pt);
}
}

So the evaluatePixel() function is invoked by Pixel Bender on each pixel in the target image. the built-in outCoord() function return the coordinated in the target image that is currently being processed. In the next two lines, I convert the pixel values into mercator values - this is a linear transformation. Then I convert the mercator values into geographic value. If the values are 'outside' the geographic boundaries are return a fully transparent pixel value. Otherwise I linearly convert it to a pixel coordinate based on the image source width and height and I invoke the built-in sampleNearest() function to get me the pixel color value and return that value to pixel bender who will paint the target pixel location with that value. Cool, eh ?
You can see the application in action here. It shows precipitation image data from NOAA. And like usual you can download the source code from here. BTW, this requires the new Flex API for ArcGIS Server.

Augmented Reality, iPhone and ArcGIS Server

I am a HUGE fan of Augmented Reality, and one of my favorite applications on the iPhone is Layar. In this post, I will "walk you" through how to setup a simple ArcGIS Layar Point Of Interests (POI) layer and how to expose it through the iPhone application. In addition, I created a small Flex application that will enable you to add POIs to that feature class through a python geo-processor task.

BTW - this is heavily geared to using ESRI desktop and server tools and I'm assuming that you are familiar with these tools.

First, request a developer key from http://dev.layar.com/. This might take a couple of hours to get to you.

So, in the meantime, let's get started. Using ArcCatalog, create a new point feature class called 'Layar' in a geodatabase with the following fields: Title (text), Line2 (Text), Line3 (Text), Line4 (Text), Attribution (Text), Type (Integer), ImageUrl (Text:512).

Import that layer into a map document using ArcMap. Save and publish the MXD as a Map Service. Just to be on the safe side, clear your REST endpoint service directory cache and list the available services. You should now see the new map service and you can now query using REST the newly created 'Layar' layer.

Next, we need to populate this 'Layar' layer. With the Advent of ArcGIS 10, this could easily be achieved using the new FeatureTask endpoint. However, I'm still using a 9.3.X server and I will have to rely on a geoprocessing task to add point features to the Layar feature class. For the sake of simplicity and quick deployment, I decided to implement this GP task using python. So, using ArcCatalog, create a new GP Toolbox and add to the toolbox a new python script. You can download the script from here. Again, I'm assuming that you are familiar with all these tools. This script will have two parameters. The first input parameter should be named 'featureSet' of type 'FeatureSet' and its schema should reference the Layar feature class. The second parameter should be named 'objectID'. It has an output direction and is of type LONG. Basically what the script does; it expects a feature set with one feature. That feature is read using a cursor and placed in memory. Next an insert cursor is created on the Layar feature class and populated with the in memory information and executed. Upon a successful execution, the OBJECTID of the last inserted feature is retrieved and returned back as a parameter. The whole script is surrounded by a try: except: in case of an exception, where a -1 as an OBJECTID value is returned back. Simple but not simplistic :-) Publish this Toolbox, clear the cache and now you should have new GPService task.

To use this GP task interactively, I created a Flex based web application.

This application consists of a side-by-side map and form component enabling the user by zooming and panning to click and define POI coordinates and to populate that POI attributes. By clicking the Save button, the above publish GP task is invoked and upon a successful completion, the map is refreshed to show the newly added POI. The Flex application source code can be downloaded from here.

Now that we have created and authored this Layar information, we need to publish and consume it on the iPhone Layar application. The Layar folks have described in detail the JSON publishing format - http://layar.pbworks.com/GetPointsOfInterest. So what we need now, is a middleware that accepts an HTTP request from an iPhone, convert the input parameters to an ArcGIS REST query, invoke a query task whose URL is the Layar layer URL in the above publish map service, read the ArcGIS JSON response and convert to a Layar JSON response. Easy, eh ? That is why we have computers :-) I accomplished all this using Java Web Application - you can download the source code from here. This web application is basically a servlet that relies on the Apache HTTPClient library for http communication and the Jackson JSON processor library for fast consumption and JSON production. You might wonder why I decided to write this in Java versus say python. well...First, I'm a Java nut. Two, I can whip this up fairly quickly, Three, wanted to experiment with Google Java App Engine so I can be "in the cloud man !" - LOL, last one is just an artifact of the first two - but it is cool ! When the iPhone Layar application requests POI, it does so on a predefined URL (more on this later) with the following HTTP GET parameters, lat: your current latitude location, lon: your current longitude location, radius: the search radius in meters plus other parameters. In this application I'm just decoding lat, lon and radius.

Now that you have an in-the-cloud POI Layar endpoint and assuming that you have gotten your Layar developer key, you can sign in to http://dev.layar.com/ and create your own layer. The form will request a POI URL, enter your in-the-cloud poi servlet endpoint. All of the above was leading this point. I highly recommend that you test your layer using the Layar test page http://dev.layar.com/api20test/layarTestPage/.

To test this on your iPhone, go to "Settings" add add your developer ID and key to the Layar application settings. Launch the Layar application, and you should see your newly published layer.

This is just the tip of the iceberg. There is so much more that you can do, adjust and customize. I just wanted to share this "little" experience with you. Hope you found it useful and tell me what you think.

Heat Map Layer Revisited

Following this post, Glenn G. found an "issue" when panning the map. I revisited the project and indeed there was a bug. Thanks Glenn. I updated the project and here is the result. In this project, I downloaded from geonames all cities with population over 15000 and plotted them on the map. The clustering of the cities (over 21,000) generated a cool heat map. Check it out and tell me what you think. And like usual, you can download the source code from here.

How Oracle Can Improve Java

Found this great article "15 Ways Oracle Can Make Java Better (and Improve Its Stance with Developers)" - here is my "in the weed" take:

1. Keep JavaONE - with heavy focus on developers.
2. Embrace OSGi down in the core VM.
3. Kill JavaFx - Work closer with Adobe on Flex / BlazeDS / LCDS.
4. Improve Dynamic Language Support - groovy / scala / JRuby / Jython /etc...
5. Talk to Google about ME (seriously?) / Android
6. Kill NetBeans - Eclipse may be King, but IntelliJ is God !
7. Introduce new language primitives XML / Function / JSON.
8. Improve language - set/get bean, native String ( switch myString {case "foo": ...}) as for casting.
9. LINQ for Java - LCDS can help here.
10. Talk to the Spring folks - there-is-only-one-framework-zuul :-)
11. No "Effing" store !

So....what do you think ?