Example: Connect to Data Collected in the Field

Use the Web Server: kobo dataport to connect to KoBo servers, using your KoBoToolbox account's API key to automatically authenticate your connection to your KoBo projects on the server, with automatic linking of data from those servers as tables and drawings in the Manifold project.  Use KoBo data to create a map of restaurants surveyed in Chartres, France.   Add new data and refresh to see new locations appear in the Manifold map.


This topic is a continuation of, and uses the data collected in, the Example: Create and Use a Survey Form topic.   See the video version of that topic in the 10 Minute Tutorial - Create and Use a Survey Form video.   See the 10 Minute Tutorial - Connect to Data Collected in the Field video for a video version of this topic.


To connect to our KoBoToolbox account from Manifold, we must copy the API key for our account.  Each account has its own API key.   To get the API key, we login to our KoBoToolbox account, and we click on the user account icon (a large blue circle with the first letter of our account name in it)  in the upper right of our user account page.  



In the illustration above, the user's account name begins with the letter A, so that is what is shown in the user account icon.   Click the user account icon and then click on Account Settings.   



The API token will be masked with a series of dots.  Press the eye icon to unmask it.



That will show our API token as a long string of letters and numbers.  Highlight the API token and then Copy it.


In Manifold, choose Edit - Create - New Data Source to launch the New Data Source dialog.



Enter a descriptive Name for the data source, and Paste the API key copied from the KoBoToolbox page into the API key box.   The API key is unique to each Kobo account, so it is a quick way of both authorizing access via the API as well as serving as user credentials for access to the data.  No need to enter Login and Password info when using the API key.


Press Create Data Source.



A new data source appears, using the name given.   Expanding the data source will show the contents of projects within the data source as tables.  If any of the projects have acquired data that includes point, line, or area data, a drawing will automatically be created for each such data acquired.


In the example above, we have connected to the project and data acquired in the Example: Create and Use a Survey Form topic.   The form used in that topic had a Location question that was used to collect data on the location of a restaurant in Chartres.    The Restaurants.Location drawing created from the table uses the Location field from the table for the point geometry of the drawing,



Opening the Restaurants table, we see it includes a single record.  The fields are shown with gray background, which indicates they are read-only.


The Location data has automatically been captured as a geom data type, a point.  If the form had also included questions to capture line and area data, using the geotrace and geoshape widgets available within KoBo forms, those fields would have appeared in the table as geom types for lines and areas, with drawings created from them as well.


We can see all of the fields in the table using the Layers pane.



By default, KoBo automatically records the beginning and ending time for a form's completion.  It also captures a unique identifier for each record collected, the version of the form used, the submission time and a status field.


Tables and drawings within the Kobo data source are read only and cannot be modified, and they cannot be styled.    If we would like to style the drawing, we can do that easily by Copying the drawing within the Kobo data source and Pasting it in the main part of the project, outside of the Kobo data source.



The drawing copy that is pasted into the main part of the project takes its data from the Restaurants table within the Kobo data source, but because it is a component created in the fully read/write part of the project it can be styled.


We can right-click onto the drawing copy and create a new map from it.   We can do that since the focus is in the read/write part of the project so we can create a new map there as well.



We create a new map using the Restaurants.Location drawing copy, with a Bing satellite layer as the background.  In the illustration above we have styled the drawing to show points as large green dots.   


We can see from the map that the point brought in from the Kobo data source is indeed located on the Le Serpente restaurant.  The Bing satellite image was acquired in the summer, as Le Serpente's wonderful outdoor dining terrace is in full swing, with patrons having a wonderful view of the cathedral.  


We can add more restaurants using the KoBo data acquisition toolset.   We will continue to use the web form, which could be used from any browser on any smartphone, tablet or portable computer.


We launch a browser on a computer and enter the URL that we copied from our KoBoToolbox form for direct access to the form.   We enter data into the form as was shown in the Example: Create and Use a Survey Form topic.



We enter data for the O Tire Bouchon restaurant, around the corner from Le Serpente.   When we have finished entering data, we click the Submit button on the form and the data is automatically uploaded into the KoBo server.


Back in Manifold we can update our data source to capture any changes.



Right-click on the Kobo data source and in the context menu choose Refresh.   That will immediately refresh the data source to link in any new projects we might have created in our KoBo account and will update any tables and drawings with any new data.



Back in the map, we choose View - Refresh to update the map.  A new green dot appears at the location of the O Tire Bouchon restaurant.



The table is also, of course, updated with a second record, the new record we added using the form.



Tables can be styled even if they are inside the Kobo data source.  For example, we can right-click on the column header for the end date, right click Style in the context menu, and then in the Style dialog choose a display style for datetime values that uses the names of the days of the week and the names of months.  We can even choose French language to use, so that the days and months are displayed using French language.   Manifold can display dates in hundreds of language-country code combinations.  



We can also create Labels from the Name field in the Restaurants.Location drawing copy in the main part of the project, and then drag and drop the labels into the map.  


We will add a third restaurant to the KoBo database.



Back in the web form, which could be running on our smart phone as we walk about in Chartres, we fill out the form for the L'Estaminet cafe/restaurant that is near the tourism center, and just around the corner from the open air market square.


We fill out the form and press Submit.  That adds a third record to our Restaurants data project in the KoBoToolbox server.


Back in Manifold, we right-click the Kobo data source and choose Refresh.



That updates the Restaurants table within the Kobo data source to add a third record, and it also updates the geometry data used by drawings which take data from that table.



With the focus on the map window, we choose View - Refresh to update the map.    A green dot and label appear for the L'Estaminet restaurant.


In this example we have left the data acquired as a linked table within the Kobo data source, which means the data resides on the KoBoToolbox server.  That is very convenient for updates, and to be able to use the data from any Manifold instance anywhere, or a free Viewer instance anywhere.    If we wanted a local copy of the data, that would be easy to do, by Copying both the Restaurants table and the Restaurants.Location drawing, and then Pasting into the main part of the Manifold project.



What does KoBo mean and how is it pronounced?  Kobo means transfer in the Acholi language, which is a Southern Luo dialect spoken by the Acholi people in northern Uganda.   The "o" is pronounced the same as the "o" in hello.   The KoBoToolbox organization capitalizes the "b" in kobo, so Manifold does that as well when referring to the organization or brand, with lower case "b" used otherwise.


What does Enketo mean and how is it pronounced? According to enketo.org: "Enketo means survey in Esperanto.  We are not sure how to properly pronounce it in Esperanto but you could say [\in-ˈke-tō]. The first "e" and second "e" should be pronounced the same because each letter has only one pronunciation in Esperanto."  Roger that.  Enketo is a cool thing however you pronounce it.


How the cathedral was saved during the war - The great cathedral in Chartres was almost annihilated by the US Army during WWII, and was saved by the bravery of Colonel Welborn Griffith, who lost his life on the mission to save the cathedral.  See more in the Notes to the Example: Locations topic.



10 Minute Tutorial - Create and Use a Survey Form - The first of two videos showing how to collect data in the field with Manifold Release 9.  We create a new survey form using the KoBoToolbox ecosystem, the same ODK/Enketo based technology used by Esri's Survey123.  We create a form to collect information and locations for restaurants in Chartres, France.  We use the form to capture data for a restaurant, automatically syncing collected data into our Kobo server.  The next video shows how to connect Manifold and automatically harvest collected data.  


10 Minute Tutorial - Connect to Data Collected in the Field - The second of two videos showing how to collect data in the field with Manifold Release 9. The first video showed how to create a survey form and to use that to acquire data for restaurant locations. This video shows how easy it is to form a live connection to the KoBo server storing our survey data, automatically linking tables and drawings into our Manifold project.  We create a map of restaurants surveyd in Chartres, France, using a satellite layer for background.  We add labels and see how when we add more data using mobile or other devices, we can refresh to automatically capture those new locations in the Manifold map.   Works in the free Manifold Viewer, too!


See Also

File - Create - New Data Source


Collecting Data in the Field


KoBo Servers


Example: Create and Use a Survey Form - Create a new survey form on the KoBoToolbox site, and then modify the form to provide a starting view for map widgets used to collect locations.  Use the form to collect data in the field, automatically syncing collected data into our KoBo account on the KoBo servers.