Thunderhead Explorer

This post is dedicated to my Esri colleagues Ajit D. and Philip H. for their invaluable help. This is work in progress - but I've put a good dent in it that I would like to share it with you.  In this post, we will go through a complete cycle, where from ArcMap, we will: Export a FeatureClass to an HDFS folder Register that folder as a Hive table Run command line Hive queries Execute Hive queries from ArcPy and show the results in ArcMap Execute a MapReduce Job as a GP Tool Import an HDFS Folder (result of MapReduce Job) as a FeatureClass This post brings everything that I have been blogging about so far into a nice story, so here we go: BTW - I am assuming that you have a Hadoop instance running somewhere and are familiar with ArcMap. You can download a Hadoop demo VM for local testing. Download the ArcMap extension in this file and unzip its content into your ArcGIS\Desktop10.1\java\lib\ext folder - The jars have to be children of the ext folder. Make sure...

So at the last Strata+Hadoop World , Cloudera introduces Impala - I downloaded the demo VM , and install the TPC-DS data set (read the impala_readme.txt once the VM starts up) and tested some of the queries. Was pretty fast and cool - As of this writing, UDFs and SerDes are missing from this beta release, so I cannot do my Spatial UDF operators, nor can I read JSON formatted HDFS record :-( One of the setup tables was a customer_address table, though it was missing a lat/lon field, it included the standard address field. Will be cool to invoke an Impala query on that table and mach up the result in ArcMap using ArcPy. So, I downloaded and installed onto my Windows VM (now remember I work on a real machine, a MacBookPro :-) the ODBC driver and downloaded and installed pyodbc . Small thing, the documentation keeps talking about 'You must use the 32-bit version'. A bit of googling revealed that they are referring to the Odbcad32.exe file is located in the %systemdrive%\Window...