Example: Connect to an ESRI GDB File Geodatabase

In this topic we create a data source by connecting to an ESRI GDB file geodatabase.    The project we create will be used in the Example: Connect Through Manifold ODBC to a Third Party and Example: Connect LibreOffice Through Manifold to an ESRI GDB example topics.

 

The well-known, American GIS company ESRI utilizes several different "geodatabase" formats for data, in three main types:  

 

 

 

 

File geodatabases are recommended by ESRI over personal databases and are becoming one of the most frequent formats used to publish spatial data for use by ESRI applications.   ESRI provides an SDK, used by Manifold, to connect to ESRI file geodatabases for read/write access.     The ESRI SDK is not thread-safe, so Manifold cannot utilize it for parallel computation, but even with that limitation it provides reasonably fast and reliable connection to ESRI file geodatabases using ESRI's own code.  

 

This topic utilizes a data set published by ESRI.   It illustrates a water distribution network in Naperville, Illinois, in the US.    Like most file geodatabases, the data is published in the form of a .zip file.   When unzipped the example creates a folder called WaterDistributionNetwork.gdb that is populated with over 160 files, the large number of files being characteristic of file geodatabases, which store various different parts of a data set, such as indexes, within separate files.   

 

Visiting the WaterDistributionNetwork.gdb folder in Windows Explorer we see the many files that comprise the data set.   

 

 

We pause for a moment to consider what has a more staggering impact on good taste: the idea of saving a single GIS project as 167 files, or the bloated ribbon interface used for Windows File Explorer.   

 

Lucky for us, Manifold requires no guesswork of which of those 167 files to use to connect to the GDB.   

 

To fit into this documentation, illustrations show an artificially small Manifold desktop, with only a few panes, docked to the right side.   In real life we use a much larger Manifold desktop, and all panes would be turned on, with some panes docked to the left and others docked to the right.

 

 

 

We begin by launching Manifold.   To connect to the GDB we choose File - Create - New Data Source - More...

 

 

We enter Water as the name for the data source, choose File: gdb as the Type and then click the [...] Browse button to find and choose the file we will use as the Source.

 

 

We navigate to our folder where we have three file databases stored.    ESRI uses an unusual style of naming file geodatabase folders as if they were files, using a three-letter extension.  We double click the WaterDistributionNetwork.gdb folder to enter it.  Note that the files of type box is already loaded with GDB files, since Manifold knows from the preceding dialog we want to create a file: gdb data source.

 

 

Mercifully, Manifold filters out all of the 160+ files in the folder that are not gdb files and shows us the single file called gdb.   We click on the gdb file and then press Open.

 

 

Taking a moment to review that everything in the New Data Source dialog is as we want, we press Create Data Source.

 

 

 

 

Manifold instantly creates a new data source called Water in the project.  We click on the + box to expand the Water data source.

 

 

 

 

We click again to epand the WaterDistribution folder.  

 

We see the GDB contains the various drawings and tables for those drawings that are in ESRI's WaterDistributionNetwork.gdb example.   

 

Drawings are named using a table.field pattern, where the name of the field used for geometry for the drawing is appended to the name of the table.   

 

 

For example, the wCasing.SHAPE drawing is called that because it takes its geometry from a table called wCasing, from the SHAPE field in that table.

Viewing a Drawing

We can work with the GDB as we could with any read/write data source in Manifold.

 

 

 

 

We scroll down to the wHydrant.SHAPE drawing and double-click on it to open it.   This drawing shows all of the fire hydrants in Naperville as points.  The illustration above uses the Style pane to set a point size of 3.5 points, and a Stroke size for points of .5 points.

 

 

 

 

Double-clicking the drawing's wHydrant table we see that it provides attributes for each of the hydrants.   Note that the SHAPE field provides the geometry data for each point as a Manifold geom.   The Manifold dataport for ESRI GDB file geodatabases automatically translates ESRI geometry into Manifold geometry to and from.  If we edit this field, the changes made within the ESRI file geodatabase will be made in the native ESRI geometry formats.

 

 

 

 

To see where all this is located we first create a map using a Google streets image server as a base layer.  We right-click on the wHydrant.SHAPE drawing in the project pane, and then we choose Create - New Map in the resulting menus.

 

 

We choose Google Maps Street Map as the base layer.  The wHydrant.SHAPE drawing we want to use is already checked, so after choosing the base layer all we need do is press Create Map.

 

A new map appears in the project, within the GDB Water data source.  We Double-click the new Map component to open it.

 

 

 

 

Manifold knows that if we have a map with a drawing that uses one of the default image server layers as a base layer, we probably want to zoom to the extent of the drawing and not to the whole world.  The window opens zoomed to the wHydrant.SHAPE layer in the map.

 

 

 

 

The dots cover up much of the underlying Google streets layer at first but if we zoom further in we can see that yes, indeed, the dots really do show the locations of fire hydrants in Naperville, Illinois.  We can see Naperville North High School in the view.

Using the Data

Anything we want to do with data in Manifold we can do with this linked data source.   Manifold's ESRI GDB file geodatabase dataport is fully read/write.   We can work with the drawings and the table as if they were native drawings and tables within the Manifold project, or stored within some other Manifold project nested into this one as a data source.

 

For example, we can make selections.   We switch to the wHydrant table by clicking the table's tab.

 

 

 

 

We use the Layers pane to hide columns to get the above, simpler display.  The LASTSERVICE field gives the date when each fire hydrant was last serviced.    We will select all fire hydrants that were last serviced in the year 2007.

 

To do that we choose the  Select pane.  If the Select pane is not one of the panes we have turned on, we can turn it on by choosing View - Panes - Select, or by using the keyboard shortcut Ctrl+4.

 

 

 

 

With the focus on the wHydrant table window, in the Select pane we choose the LASTSERVICE field, and then we double-click the Search template to launch it.

 

 

 

 

In the Search template, from the Use parameter pull down menu we choose date and time.

 

For the Condition, from the pull down menu we choose between.

 

For the At least parameter we enter the first 1-Jan-2007 and for the At most parameter we enter 31-Dec-2007.

 

To see a preview, we press the Preview button.   Manifold will preview the result of the between operation using blue preview color in the table, adding the name of the Select template in the upper right corner of the table to indicate what is being previewed.

 

 

 

 

If we like what we see in the preview, to make that selection in the table we click the Select button.   

 

 

 

 

We have selected all of the records for hydrants last serviced sometime in 2007.

 

 

 

 

The same selection automatically appears in the wHydrant.SHAPE layer in the map.

 

 

 

 

If we like, we can add another imageserver layer, such as a Google satellite layer.  Zooming in, we can see where the various hydrants are located.  In the illustration above we have used Style to format the points in green color and to increase the point size slightly, to make it easier to see hydrant locations against the satellite photo background.

 

Notes

Many files -  The very large number of files used in a file geodatabase, with essential elements of the data set being stored in separate files that must travel together for the data set to retain coherency, poses obvious risks for fragility.   It is wise when transferring a geodatabase to do like ESRI often does, zipping all the files together into a single .zip file.   Given there are very many files involved, all of which must be synchronized, it is also highly unwise to attempt to  modify an ESRI file geodatabase using anything other than ESRI code.   Manifold uses ESRI's own SDK to interact with ESRI file geodatabases.  

 

Modern GDB vs Deprecated GDB - The ESRI SDK used by Manifold supports connections to ESRI's modern GDB file geodatabase format, in current use for some years.   Many years ago ESRI used an older GDB file geodatabase format that is no longer supported by the ESRI SDK.   Some GDB format data sets are still in circulation on the web that use the old GDB format.   To read those, use Manifold's ability to connect through the GDAL library to formats that GDAL supports.  

 

Before GDAL began offering the ESRI SDK as an option to read modern ESRI file geodatabases, a GDAL-written, reverse-engineered module could read some information from the older ESRI GDB file geodatabase formats.   The GDAL module is not perfect in that it is read-only and it cannot extract all data from all file geodatabases, but at least it can extract some, if not all, data in many cases.  Better than nothing! The step by step procedure to use GDAL is given in the Example: Connect to an ESRI GDB usng GDAL/OGR topic.

 

Creating the Map -  We created the Map component by right-clicking on the hydrant drawing within the Water data source, that is, within the ESRI gdb, so that's where Manifold created both the Map and also the Google Maps Street Map data source.    

 

At the end of the example, we have added the Google satellite image server within the Water GDB data source as well.   We created the Map component within the GDB data source, and by doing so we automatically created the Google street map image server within the Water GDB data source as well.   To use the Google satellite image server within that map, it should also be created within the Water GDB data source.

 

When Manifold creates various Manifold accessories, like a map or a data source, within an ESRI gdb, it does so in a way that will not interfere with normal ESRI function, if we have colleagues working with that gdb using ArcGIS Pro or other ESRI software.   ESRI gdb format perfectly capable of storing tables that Manifold may create to save simple text fields, like the text field that specify what layers are in a map or how to connect to a Google image server data source.   

 

However, for Manifold users it is a great convenience to have various accessories within the project, such as image server data sources, saved within the gdb. That way, anybody else who connects to that gdb (including us, in the future) with Manifold will have things like style and components like maps and layouts ready to go.  

 

It could be, of course, that the administrators for our ESRI gdb might not want us to make any Manifold-specific changes to their gdb.  In that case, we could easily create a map in the main body of the project (right-click into the main body of the project, outside of the Water data source, and choose Create - New Map...) and then drag and drop the hydrants layer into the map.  

 

Videos

Manifold Viewer - View GDB Faster than ArcGIS Explorer - ESRI's ArcGIS Explorer product is a free viewer that allows ESRI people to view ESRI GDB Geodatabases, along with other data.  Unfortunately, ESRI has stopped producing Explorer and has replaced it with ArcGIS Earth, which cannot connect to and display ESRI GDB Geodatabases.  The video compares Explorer to Viewer side-by-side so ESRI users can see that if they need to continue viewing GDB Geodatabases for free they can reliably use Viewer for fast, high quality GDB viewing.

 

See Also

Maps

 

Selection

 

Web Servers

 

Select Pane

 

GDAL / OGR

 

Example: Spectacular Images and Data from Web Servers - A must see topic providing a gallery of views illustrating how Manifold can use web servers such as image servers and other free resources to provide a seemingly endless selection of spectacular background maps, satellite images and GIS data with nearly zero effort.

 

Example: An Imageserver Tutorial - An extensive tutorial showing step by step how to add new data sources that are imageservers, how to show them as layers in a map, how to create a new drawing that matches the projection of the map and how to trace over what is seen in an imageserver layer to create an area object in the drawing.

 

Example: Convert an ESRI File Geodatabase into a .map Project - How to convert data in ESRI's current file geodatabase format into a  Manifold .map project in a single step.  We convert ESRI's example NapervilleGas GDB geodatabase, all 857 files, into a single, unified Manifold .map project.

 

Example: Import ERDAS GIS with GDAL and Assign Coordinate System  - A multistep example that covers import of an unwieldy format using GDAL, followed by use of Assign Initial Coordinate System and Repair Initial Coordinate System to deal quickly with real-life complications.

 

Example: Create an ODBC Data Source with Windows - How to create an ODBC data source (a DSN) using dialogs built into Windows 10.

 

Example: Connect to Manifold from Release 8  - Step by step procedure to connect from Manifold System Release 8 to a Manifold .map file using Manifold's ODBC driver.

 

Example: Connect to an ESRI GDB usng GDAL/OGR  -  Instead of using Manifold's built-in ability to connect to modern ESRI GDB file geodatabases, use the Manifold GDAL/OGR dataport to take advantage of the GDAL library's ability to connect to deprecated GDB formats.

 

Example: Connect Through Manifold ODBC to a Third Party  - With Release 8, use an ODBC connection to a Manifold .map to connect through the .map project to a third party, external data source, an ESRI GDB file geodatabase.  We use Manifold facilities as an intermediary to give Release 8 capabilities it does not have on its own, to link into data stored within an ESRI file geodatabase.

 

Example: Connect LibreOffice Through Manifold to an ESRI GDB - A companion example topic to the Example: Connect Through Manifold ODBC to a Third Party  topic.  Shows how to connect LibreOffice Base, the database part of LIbreOffice, through Manifold to link an ESRI GDB file geodatabase table into LibreOffice.

 

Example: Connect to Manifold from QGIS - Step by step procedure to connect from QGIS 2.8.9 to a Manifold .map file using Manifold's ODBC driver.