Server: Share Data

Manifold Server, called Server for short, is a read-only, parallel, spatial database server that also includes a TCP data sharing server, an HTTP map server and WMS server.  This topic discusses use of Manifold Server as a TCP server to share Manifold projects to Manifold clients.

 

Server can run in three modes:

 

 

 

 

See the Manifold Server, topic for basic instructions on installing, activating, and running Manifold server in either TCP or HTTP mode, including a complete list of command line options.

 

To use the Manage Services dialog,  a Server license must be installed for all users.   If the license is not installed for all users, it will not be possible to run new services from the dialog.  Check that setting in the Help - About dialog to verify that the license has been installed for all users.

TCP Server

Manifold clients connecting to Server running as a TCP server are regular Manifold Release 9 sessions (any Manifold Release 9 edition, including Viewer)  that can be launched either on the same computer or on different computers, connecting through a local area network or Internet.

 

A Manifold client connects to a Manifold TCP server using the same dialogs and commands as connecting to any data source.  The connection to the Manifold server appears as a data source, just like any other data source, for example, like linking to a Manifold .map project file.  The Server data source can be expanded to view and use the contents within.

 

A Manifold client session can connect to multiple different Server instances, either on the same machine or on different machines.   Multiple different Manifold client sessions can run at the same time on the same machine, with each client session connecting to the same or to different Server instances.

 

Server allows many different Manifold users at the same time to use the same components that are stored in .map files.   Because Manifold projects can be nested to many levels without losing performance, with one .map file containing links to other .map files, the .map file with which Server is launched can refer to hundreds or thousands of other .map files, making it easy for an organization to present many terabytes of archived GIS data for mujltiuser use through Server.   

 

Organizations with at least a Manifold 9 Universal license can serve the contents of many .map files through a single Server instance by opening that Server instance with a .map project that contains references to other .map projects as nested data sources.  

 

Organizations with a Manifold 9 Server license can launch multiple Server instances on the same machine with each Server instance sharing out a different .map project.  Each instance can use a different network address and port, and also different login and password credentials.  That makes it easy to share out data to many different groups of users by using different Server instances to control how different groups have access to different collections of data.

 

Manifold clients connect to Server through many threads for greater performance.  Many Manifold functions, such as query execution and preparing data for rendering, automatically occur server-side within Server to reduce the need to send large amounts of data through the network, and to take advantage of more powerful hardware that is normally used for servers.  Server automatically provides tunneling from clients to data sources within the .map projects shared by Server.  Shared cache on Server helps improve performance for all clients when the same data is used by multiple people within an organization.   

 

Organizations can run Server on several different machines to share out different constellations of .map files.  Clients can then connect to several different Server data sources and use data from several different Server installations as layers in their maps.  Using more than one Server machine will provide more processing power than a single machine.   Each machine running Server requires a Server license, either a Manifold 9 Universal or a Manifold 9 Server license.

 

For example, if a map includes three different layers from three different Server installations on three different machines, when that map is panned and zoomed each of the three different layers will be prepared for rendering and otherwise serviced by the Server machine sharing that layer, so that the computational resources of three different machines will be utilized.   Instead of parallel processing on a single computer handling all three layers, parallel processing on four different machines, the three Server machines plus the client machine, will be used to speed up the rendering pipeline for the map.  That can result in visibly faster response with very big layers.

Launch a TCP Server from a Command Prompt

Server can be launched from a Command Prompt window when using a Manifold 9 Universal or a Manifold 9 Server license.  

 

When using a Manifold 9 Server license, Manifold Server instances can  be launched as Windows services using the new Tools - Manage Services dialog within Manifold.  That is a lot easier than launching Server from a Command Prompt window as illustrated below, although a Manifold 9 Server license also allows launching Server from a Command Prompt window.   That is a useful option since dealing with Windows Firewall can be easier if we launch Server the first time after a new installation by launching from a Command Prompt window.

 

This topic shows examples launching from a Command Prompt window.  For examples launching Server using the Tools - Manage Services dialog, see the Tools - Manage Services topic.

 

The server machine in this example is a Windows 11 machine, with rounded corners on windows.   Client sessions in this topic will show Manifold being launched on a Windows 10 machine, with right-angle corners on windows.   In this example a portable Manifold installation is being used for both server and client.

 

 

Open a Command Prompt window and navigate into the Manifold installation folder containing the manifoldsrv.exe file.  Launch the manifoldsrv.exe file without any arguments to see syntax and options.

 

 

For this example we launch Server using the meteorites.map example project from the Release 9 Downloads page.  We want Server to listen to the IP address used by the machine on the local area network.   The command line used is:

 

manifoldsrv -addr:192.168.2.12 "C:\mfd\projects\meteorites.map"

 

We only need quotes around the path to the file name if any part has a space character in it.    There are no spaces in the example above, but we use quotes anyway since they do not hurt and to get in the habit of using quotes for when we will use a file name or path with a space character in it.

 

Server starts up and announces the options it is using.   

 

Leave the Command Prompt window open and do not log out from the account being used.  Server will continue to run so long as the Command Prompt window is open, we do not press Ctrl-C or Ctrl-Break, and we do not log out of the account being used.   If we do not want other people using a computer that has an opened Command Prompt window, lock the screen by simultaneously pressing the Windows logo key and L.

 

 

Windows immediately raises a Windows Defender Firewall dialog.   Check both boxes and choose Allow access.   

 

Server is now running and ready to service client Connections.

Connect to Server

Connect to Server using Manifold as a client, running on a different machine.   The client machine in this example is running Windows 10, with right-angle corners on windows.

 

Connecting to Server is easy:  create a new data source for the Manifold Server database.  That is the same as connecting to any database, such as SQL Server or PostgreSQL.

 

 

Launch Manifold as usual and choose File - Create - New Data Source - More... to launch the New Data Source dialog.

 

 

In the New Data Source dialog provide a memorable Name for the data source, if desired.   In the Type box choose Database: manifoldserver.  

 

Click the browse [...] button to launch the Database Login dialog.

 

 

In the Server box, enter the IP address that Manifold Server is listening on, for example, 192.168.2.12.  The IP address reported by Server for the Network Interface when it was launched is the IP address to use.  Also use the IP address when launching Manifold on the same machine running Server.  Do not use localhost or a NetBIOS/WINS name for the machine.

 

To specify a connection to Server that uses a port other than the default 9099, specify the port along with the IP address in the Server box in the Database Login dialog, in the form

 

<IP address>:<port>

 

...for example, 192.168.2.12:9098   

 

It is a good idea when connecting to nonstandard ports to use the name of the port or the name of project being provided into the data source name used in the Name box, to make it easier to keep track of which data source is which.

 

If a login and password have been required for Server, check the Use login and password option and specify the Login and Password.

 

Press OK.

 

 

Back in the New Data Source dialog, press Create Data Source.   

 

Note that we do not have to check the Open as read-only box, because Manifold Server connections are always read only.

 

 

A new, read-only data source appears in the Project pane.  The database cylinder icon for the data source has a lock sub-icon to indicate it is read-only.

 

The client-server connection is not formed until we expand the data source.   When we expand it, Manifold connects to Server and the data source is populated with the contents of the .map project with which Server was launched.

 

In this example a very simple project is used.   There are only two data sources in the project, webserver data sources for Bing and Google Satellite.   When Server is used to share layers in an organization, the project with which Server is launched usually would contain many components that are to be provided, perhaps organized into folders, and it might contain data sources that are other Manifold .map projects linked in.   

 

The project shared by Server can also contain data sources like connections to enterprise databases such as Oracle, MySQL, PostgreSQL, and SQL Server, so that those connections could be provided to users as well.

 

Saving a project that includes a Manifold Server data source will save that data source and connection information just as other database data source connections are saved.   The next time we open that project, the Manifold Server data source will be ready to go.

The Database Box

The Database box in the Database Login dialog allows us to connect to a nested data source within the project shared by Server as if that was the only data source in the Server project.   Data sources are shown in Manifold projects with a cylinder database icon, because in Manifold data sources are themselves databases.

 

To connect to a nested data source within the project shared by Server, enter the path to that data source in the Database box.  Levels nested further down are separated by double :: colon characters.  For example, suppose Server is sharing a project called Europe.map, and that project has within it nested data sources that are links to .map project files for various countries.  Each such country in turn has links to .map files for various provinces within the country.

 

Connecting to the root of the Europe.map file we would leave the Database box empty.  Opening up the Server data source will show the entire contents of the Europe.map project as though we had opened it directly.

 

Entering [France] in the Database box when creating the new Server data source will create a Server data source that when opened will show the contents of just the France.map file, as if we had launched Server to share only that one projects.  None of the other data sources for the other countries in the Europe.map project will appear.  

 

Entering [France]::[Centre] in the Database box will create a Server data source that when opened will show the contents of just the Centre data source within the France data source that is in the Europe project.

 

Entering [France]::[Centre]::[Chartres] in the Database box will create a Server data source that when opened will show the contents of just the Chartres data source within the Centre data source within the France data source that is in the Europe project.

Clients Monitor their Connections to Server

When we create a Manifold Server data source in a Manifold project, the data source does not attempt to connect to the Server database until we open the data source.

 

The data source then attempts to connect to the Server database in background. If the connection cannot be established in a reasonable time, the data source logs an error and stops further attempts to connect.   If the data source connects to the Server database successfully, after the connection is established, the data source will monitor the connection.   If the Server database breaks the connection unexpectedly, for example, because on the server machine the manifoldsrv instance has been stopped, the data source logs an error and becomes disconnected.

 

When disconnected, either because an initial connection failed or because Server broke the connection, the data source will not attempt to reconnect until the parent data source, which is usually the .map file being used by Manifold session, has been closed and then reopened.   Manifold expects in future builds to allow reconnecting a Server data source using the View - Refresh command.

Client Connections Reported

When a client connects to Server, back on the Server machine the Command Prompt window is updated with an additional line showing the client connection.  The IP address used by the client is reported.

 

 

When a client connects to Server, back on the Server machine, the Command Prompt window is updated with an additional line showing the client connection.  The IP address used by the client is reported.

Connections are Parallel

Manifold running as a client to a Server database connection launches multiple, parallel threads to connect to Server, which processes the connection with multiple, parallel threads as well.  The result is a very efficient, very fast,  parallel connection for exchanging large amounts of data.   It is far faster than connection technologies like ODBC.

 

A Manifold client data source connected to a Server database sends multiple requests to the server in parallel using multiple threads.  A request does not have to wait until all previous requests have been processed by the server or even have been completely received by the Server instance.  Server processes multiple requests from the same client connection in parallel if there are worker threads available.  If worker threads are not available, because all worker threads are busy processing requests from other connections, requests from the Manifold client are queued for servicing when worker threads become available.

 

Both sides, both the Manifold client and Server, split big data packets into parts and communicate with each other to optimize work with those parts to avoid overwhelming the network with too much data sent at once.  Both sides prioritize transferring small data packets, such as those involved with getting the schema for a table, before transferring big data packets, such as fetching the next group of many records from a big table.  Prioritizing small packets minimizes the number of requests waiting in queues. Both sides can have an unlimited number of requests waiting in queues outside of available memory, to ensure no requests are ever lost.

 

Advice:  The data connection between Manifold as a client and Server can only run as fast as the speed and bandwidth of the network allow.  Use a fast network to connect to Server.   Avoid connecting through VPN services, which usually are much slower than Manifold parallel connections can run.

 

If a client machine is set up to use a VPN when connecting to Internet, either use local network IP addresses such as 192... that do not go through the VPN to connect to Server, or set up Manifold as an exception to the VPN so it does not connect through the VPN when connecting through regular Internet IP addresses to Server.

Server Side Features Leveraged

A Manifold client data source connected to a Server database supports all features of spatial indexes, including those that enable faster rendering: access to intermediate levels for raster data, thinning for vector data and thinning for point cloud data used with LAS / LAZ files.   That means rendering data shared by Server will use all the intelligence that rendering from a .map project directly opened by the Manifold client on the local system would use.   The usual result is rendering that performs much faster than rendering from a general-purpose database, such as Oracle, SQL Server, or PostgreSQL.

Read Only Data Source

The Server data source is read-only.  We can open components in the data source to view them, but we cannot change anything about them.   For example, we could double-click a map in the data source to open it, and we could pan and zoom within that map, but we could not turn layers in the map off and on, nor could we change the Style of any of the layers.

 

For more convenient interactive use of components within a Server project, copy them from within the Server and paste them into the read/write main part of the project.  This is the same technique that would be used with components stored within any read-only database, for example, if we have read-only access to our organization's GIS assets stored in the organization's PostgreSQL database server.

 

 

For example, click the Map in the Server data source and then press the Copy button in the Project pane toolbar, or press Ctrl-C for Copy.   Next, click into the Project pane outside of the Server data source to move the focus there.

 

 

Press the Paste button in the Project pane toolbar, or press Ctrl-V for Paste.   A copy of the Map appears in the project outside of the Server data source.   

 

That copy of the Map automatically has had the Properties adjusted to refer back to the originating tables and layers in the Server data source.   However, the new Map itself has fully read/write infrastructure for things like layer tabs.  

 

Double-click the new Map to open it:

 

 

The Map opens showing the meteorites map published in the meteorites.map example project.   Note how the names of layer tabs are now fully qualified to show that the layers are coming from the Manifold Server data source.

 

 

We can double-click layer tabs to turn the layer off and on.   In the illustration above we have clicked off the layer showing all meteorites and have clicked on the layer showing only those meteorites that came from the planet Mars.  

 

Some meteorites were created when a large body impacted onto the planet Mars with such power that some material was ejected from the resultant impact/cratering event with such force that escaped the planet's gravity and went into orbit around the Sun.  On rare occasions some such material has intersected the Earth's orbit and has crashed down onto the surface as a meteorite.  Meteorites that originated on Mars can be identified with certainty as parts of the planet Mars by microscopic inclusions of Martian atmosphere, in which the ratios of gases and isotopes match the unique properties of Martian atmosphere as measured by landers sent to Mars.

 

 

We can zoom into the Martian meteorite found in France.  With the focus on the martian meteorites tab, we alt-click the symbol to pick that object, displaying its attributes in the Info pane.

 

 

That particular meteorite, the famous Chassigny meteorite, was found in 1815, a meteorite fall that was observed by many people when the meteorite entered the atmosphere and streaked into an impact in a farmer's field, with multiple pieces of meteorite resulting from the fall.

 

The gray background for fields in the Info pane indicate the fields show read-only data.   

Stopping Manifold Server

We stop Server by closing the Command Prompt window in which the Server session is running, or, with the focus on the Command Prompt window, pressing Ctrl-C or Ctrl-Break.   Logging out from the user account used for the Command Prompt session will also shut down Server.

 

 

In the illustration above we have pressed Ctrl-C to shut down server.  All connected clients will be disconnected.  

 

Administrators will normally check with users before shutting down Server, to give users a chance to save and close their projects.   There are many chat or team messaging applications available. Pick one and use it in your organization to allow administrators to broadcast a shutdown warning to users.  

Changing Manifold Server Projects

Server shares whatever .map project was specified in the command to launch Server.   While Server is sharing that project, other Manifold sessions cannot open that same project.   Other Manifold sessions could connect to Server as a client to use data from that project, but they could not open that same project.  

 

Changing the content of a project shared by Server:

 

  1. Shut down Server.

  2. Open the project in a regular Manifold session.   

  3. Change the project as desired.

  4. Save the project, and then close the project.

  5. In a Command Prompt window, launch Server again with that project.

 

To change the .map project that is shared by Server, shut down Server and then launch it again using the different .map project.  Use keyboard shortcuts in the Command Prompt window to avoid lots of unnecessary keyboarding.  

 

For example, suppose we have Server running in a Command Prompt window.   We want to edit the file it is sharing out.  The fast way to do that is to press Ctrl-C in the Command Prompt window to stop Server.   Next, without closing or doing anything in the Command Prompt window, in a regular Manifold session open the .map project that was being shared by Server.  Change the project as desired, save it, and then close the project in the Manifold session.

 

Back in the Command Prompt window, press the up arrow key on the keyboard.  That will reload the last-used command line, which was used to launch Server with that .map project file.  Press Enter and Server is now running just as before with the updated .map project file.

Data Source Tunneling

When Server shares a Manifold project, it automatically provides connections through Server to data sources within the project being provided, excepting web data sources such as image servers.   Connecting to a nested data source in a Server database creates a new connection to Server and tunnels data through that connection.

 

Tech tip:  Since each data source in a project shared by Server requires a new connection, having many data sources in projects shared by Server can rapidly use up the 20 connections allowed when running a Manifold 9 Universal edition limited version of Server.    It is therefore wise to avoid sharing generic data sources, like Bing or Google image servers, in projects that are provided by Manifold 9 Universal edition Server installations.   Alternately, use Manifold 9 Server edition for Server installations where many data sources will be provided, since the unlimited version of Server in Manifold 9 Server edition licenses allows an unlimited number of connections.

 

Manifold projects can have three types of data sources, based on whether they can be reached through the web without any local bindings or not:

 

 

 

 

For example, if the project being provided contains a SQL Server or MySQL database data source,  the Manifold client can reach into that database through the Server connection without having to independently login to that database data source.   For example, if Server shares a .map project which has a link to a JPEG 2000 file, when a Manifold client connects to Server and within the Server data source opens the image created from that linked JPEG 2000 file,  the client asks Server to open the JPEG 2000 file and read its contents on behalf of the client.   The client and Server form another connection and data from the JPEG 2000 tunnels through that connection to the client.

 

Projects shared by Server can contain Manifold Server data sources, with nested Server data sources automatically setting up pass-through tunneling.   For example, a project shared by Server on machine A might include Server data sources using Server installations on machines B and C.   A Manifold client connecting to Server A will see the Server B and Server C data sources within that project and will be able to open them and use the data and links they contain.   Server A will automatically arrange pass-through tunneling to Servers B and C, which will in turn automatically arrange pass-through tunneling to the data sources they share.

 

Using the meteorites.map project from the examples above, suppose we shut down the Server session sharing that project, and then we open the project in a Manifold session.  We add a connection to a PostgreSQL database, the same one used in the Connect to PostgreSQL topic.   We also use File - Link to link in a Manifold project that contains many ArcGIS REST web server data sources.  We save the project, and close it.   Next, we launch Server again using that new, updated meteorites.map project.

 

When a Manifold client opens data sources within the Server project that are web sources, those web data sources are not tunneled, but instead are passed to the client for a direct connection.  For example, if a map within the provided project uses a Bing streets image server data source, that image server data source automatically will be handled by the client session.  Instead of Server pulling tiles from Bing and then passing them to the client, the client instead will pull tiles from Bing directly, not wasting network bandwidth in the Server connections for what it could get directly from Bing.

 

However, if a Manifold client creates a new data source for Server that uses the Database box option to explicitly connect to a web data source within the Server project, then Server will tunnel data from that web data source to the client.

 

Tunneling to nested data sources applies recursively from the root database operated Server to all child data sources at all levels. Since each nested data source may potentially create a separate connection, if the project served by Server includes many nested data sources, use a Manifold 9 Server edition license and launch Server with the connection limit increased to more than the default 20 connections.

 

Tunneling to a nested web data source in a Server database automatically turns off the option to save cached data between sessions, because the parent database used to store cached data might be read-only.  If the web data source has been configured to cache data at all, the cache will still be used, but it will be cleared when Server exits.

Example

We launch Manifold on a different machine and we create a data source that connects to the Server that shares the meteorites.map project.   Or, if we have previously saved a project that includes that Server as a data source we just open that project again.

 

 

When we expand the Manifold Server data source, we see it contains two additional data sources, one for the linked ArcGIS_REST_Servers project, and the other for a PostgreSQL database connection.

 

Data sources are tunneled only after they are expanded or otherwise opened.   Data sources shown with a gray database cylinder icon have not yet been opened.  Each data source that is tunneled and expanded will use a connection to Server.

 

 

Expanding the ArcGIS_REST_Servers data source shows it contains, in turn, many more data sources, for various ArcGIS REST servers.   The ArcGIS_REST_Servers data source is tunneled because it is not a web data source but is instead a link to a .map project.   The individual data sources within the ArcGIS_REST_Servers data source are not tunneled, since they can be passed to the client for direct connections between the client and the web data source.

 

Whether the data source is tunneled or not, we can display any of those layers directly, as if they were part of the local project.

 

 

Collapsing the ArcGIS_REST_Servers data source and expanding the Postgres data source, we can see the contents of the PostgreSQL server used in various topics in this documentation.

 

 

Double-clicking the public.MexicoPM drawing opens it, as if it were a local component within the project.   The Manifold client-server system is automatically tunneling through Server to reach through the project Manifold is sharing into a PostgreSQL data source within that project, and then tunneling into the PostgreSQL server to display a component stored there.

 

If we wanted to alter the style of the public.MexicoPM drawing, we could copy the drawing within the PostgreSQL server (just the drawing, not also the drawing's table) and then paste it into the local part of the client session's project.    We could then alter the style of that local copy, with the data automatically being brought into the drawing through the data source tunnel from the table within the PostgreSQL server.

Running Queries on Server

Queries and possibly scripts will run on the Server machine as follows:

 

 

Running queries on Server can be a huge performance boost when working with bigger data because the results of queries are usually much smaller than the bigger data that is utilized within the query.  Instead of having to send much data to the client for a query running on the client, Server can send only the results of the query to the client.

 

In the examples below we connect using a Manifold client to a Server instance that is sharing the example meteorites.map project used for examples in this topic.

Queries in the Server Project

Queries in the project shared by Server will execute on the Server machine.

 

 

Suppose the meteorites.map project has been modified to include a query called Biggest Query that finds the biggest meteorites in the worldwide database.  A drawing called Biggest has been created from that query as well.

 

 

Opening the query shows it with gray, read-only background, since the Manifold Server data source is read-only.    Executing the query by pressing the ! Run button in the main toolbar will run it on the Server machine.

 

 

Opening the Map, which includes the Biggest drawing as a layer, automatically runs the query to populate the drawing with those meteorites in the database with a mass greater than or equal to 500 kilograms.

Command Windows in Server Context

Queries written within Command Windows launched in the context of the Server data source will execute server side.

 

 

Launch a Command Window in the context of a Server data source by right-clicking the Server data source and then choosing New Command Window.

 

 

A new Command Window opens that running in the context of the Server data source.  What we do in that Command Window operates as if we had a remote desktop connection into the Server machine.

 

 

For example, if we drag and drop the meteorites Table from within the Server data source into the lower right pane of the Query Builder, the entries appear without a [Manifold Server]:: prefix the way they would if the table were dragged and dropped into a Command Window running in the context of the local project.

 

The query text pane has white background since Command Windows are dialogs that show transient text that is has not been saved into any queries.  We can write whatever we want into the Command Window even though the query we write will be executed within a read-only data source.

 

We write a query:

 

SELECT * FROM [meteorites Table]

   WHERE [mass] >= 500000;

 

The query text is the same as if we had launched the project locally within the Server machine.    

 

Running the query by pressing the ! Run button in the main toolbar will execute the query on the Server machine, with results being sent to the client machine for display.  

 

 

The results table shown by running the query has gray, read-only background since the table used is in a read-only data source.  We cannot edit through the results table to change data within the read-only data source.

Queries using EXECUTE

Queries written outside the Server data source will execute within the Server machine if the query text is wrapped within an EXECUTE [[ ]] ON statement that names the Server data source.

 

 

For example, suppose in the local part of our Manifold client project we create a Biggest Query analog to the query in the shared project.

 

 

Opening the query shows the SQL text we have written:

 

EXECUTE [[

SELECT * FROM [meteorites Table]

   WHERE [mass] >= 500000;

]]

ON [Manifold Server];

 

The query text has white background since it is in the local, read/write part of the project.   Note the syntax used for the reference to the meteorites Table as well as the reference to the Manifold Server data source following ON.   Because the query will be executed within the data source, we write [meteorites Table] and not [Manifold Server]::[meteorites Table].  

 

Note also that we have populated the lower right pane of the Query Builder by dragging and dropping the name of the data source into the lower right pane and also by dragging and dropping the meteorites Table from within the data source into the lower right pane.   Manifold has automatically added [Manifold Server] to the names used.   That is very convenient when we are writing queries that execute on the client, the default situation.  It is also convenient for server-side queries if we want to use the names of fields in the query builder with a double-click.  But it is less convenient for server-side queries that are written on the client using EXECUTE since we will want to remove the [Manifold Server]:: portion of component names.

 

Running the query by pressing the ! Run button in the main toolbar will execute the query on the Server machine, with results being sent to the client machine for display.  

 

 

The results table shown by running the query has gray, read-only background since the table used is in a read-only data source.  We cannot edit through the results table to change data within the read-only data source.

Copy and Paste Server Data Sources

We can copy a Server data source we create in one Manifold session and then paste it into a different Manifold session.  That saves the effort of creating it in the other Manifold session.

 

 

Suppose we already have created Server data source in one project.  We have given the Server data source a name that reminds us which machine is being used for that Server data source.

 

Click on the Server data source to highlight it and press Ctrl-C for Copy.  Or, we could copy using the Project pane toolbar Copy command, or we could right-click the data source and choose Copy from the context menu.

 

 

Switch to the Manifold session where we would like to paste the Server data source and press Ctrl-V to Paste.   Or, we could paste using the Project pane toolbar Paste command, or we could right-click into the Project pane and choose Paste from the context menu.

 

 

 

The Server data source appears in the Project pane.

 

 

We can expand the Server data source to see it is the same data source we created in the other Manifold session.  We can drag layers from the data source and drop them into the map.

 

Just like any other data source, a Manifold Server data source can be saved as a Favorite Data Source.   Save frequently-used Server data sources as favorites to make them available with a single click.

Multiuser Sharing of .map Projects

Linking a file or a .map project into a Manifold project by default links the file or .map project both for reading and writing, and restricts the use of that file or .map project to only one Manifold session.   If we link a file or a .map project read-only into a Manifold project, then other Manifold sessions can also use that same file or .map project so long as they also link that file or .map project into their projects read-only.

 

The above applies to files and to .map projects linked into the .map project that Server shares out.   

 

Suppose we have two Server instances running on the same machine, Server A and Server B.   If the .map project shared by Server A includes a linked file called myfile, then the .map project shared by Server B can only include myfile as a linked file if it is linked read-only.   If the .map project shared by Server B includes myfile as a linked file and it is not linked read-only, then if Server A is launched first the Server B instance will not provide access to the linked myfile data source.

 

Since Server instances are read-only for Manifold clients, it is good to get into the habit when creating .map projects that will be shared by Server to link all data sources read-only.  Whenever linking a file or a .map project into such .map projects, get into the habit of linking them by dragging and Ctrl-Shift-dropping the file into the Project pane.   

 

Drag and Ctrl-Shift-drop is the shortcut way of linking a file read-only.  This is a shortcut that is simpler than using File - Create - New Data Source to create a read-only linked data source.

 

The drag and Ctrl-Shift-drop shortcut also works to add .map projects as read-only data sources into a project.   By adding a .map project to another project as a read-only data source, we can share a single .map project for simultaneous use by multiple users on different machines.  That's very easy to do and it allows us to keep frequently used data in archived .map files.   

 

 It is easy to see if a data source has been linked into the project read-only.   Read-only data sources show a database icon with a lock, whether they have been opened or not.

 

See the Example: Multiuser Sharing of .map Projects topic for a quick example of sharing a .map project read-only by two users at the same time.

Scenarios for Manifold Server Sharing Data

The classic use case for Server as a TCP server is sharing out layers from a central archive for simultaneous use by different people.   Different users can use the same layers at the same time.   Server is installed and launched on the server machine, and users on the various client machines launch Manifold in the usual way on their desktops.   

 

A Manifold 9 Universal license allows one Server instance on a machine.   Universal licenses are good for using Server by individuals or by smaller workgroups.  Smaller workgroups, for example with four or five people, will often have a Manifold 9 Universal license for each person.   Each person on their machine will host a Server instance, and will also use Manifold interactively on their machine at the same time.  With four or five Server instances operating at the same time in the workgroup, that provides four or five shared sets of data to the organization.  With one level of nesting, that can provide a vast amount of data using licenses for each user that cost just slightly more than a Professional license that does not provide Server.

 

A Manifold 9 Server license allows use of multiple Server instances on the same machine, so an organization that has a server machine with good hardware might launch several different Server instances on that same machine, each using a different port, to share out different .map projects that each provide access to a different constellation of data.  Larger workgroups or organizations will usually have a few Manifold 9 Server licenses for their Server instances on a few machines, and then use Manifold 9 Professional licenses for most users, who will be just operating clients.

 

Server instances can run on the same machine that is also being used for Manifold desktop sessions.  A single Manifold license allows either one Server instance (Universal) or an unlimited number of Server instances (Server), and in both cases allows an unlimited number of desktop Manifold sessions to run simultaneously on that same machine.  Teams of GIS people working together might want to distribute their shared files to different user machines, perhaps so that fast SSD storage on any one machine is not used up by too many shared files, and then run Server instances on each of the different user machines so that other users can connect to them and use data the Server instances share out.

 

Saving frequently used, big layers on different machines and then sharing those layers with Server instances on those different machines has a performance benefit for clients that use several of those big layers in the same map.   When the map is panned and zoomed, the different layers will come in from different Server data sources, with each different layer being rendered by a different machine.  That usually results in visibly faster performance than sharing out all of the big layers from the same Server instance on the same machine, in which case the same machine will render all of the big layers as the map is panned and zoomed.

 

Server can also make sense for individual users.  Individual GIS users often have more than one project open at the same time, and they might want to use the same layers in those different projects.   A Release 9 Universal license includes both Server as well as Manifold as a desktop client, and both can be used at the same time on the same machine.   We can launch Server in a command prompt session that is minimized onto our Windows taskbar, and then launch multiple different Manifold desktop client sessions that can use layers shared out by the Server instance.

 

Using Server on our own machine to share files to our Manifold client sessions on the same machine is a handy way of sharing collections of very many read-only data sources, for example, a large collection of ArcGIS REST sources for GIS data from various public organizations.   If we save a Server data source as a Favorite Data Source, we can add it to a project with one click and then copy any desired component or data source from within it to our current project, and then deleting the Server data source if we no longer need it.   Use a Server data source in that way to get access to a collection of frequently used components avoids the problem of not being able to open the .map project that stores the collection of data sources if other sessions are using it.

 

When an individual is using Server to share out archival collections of files and data sources, nesting projects one or two levels deep, only those data sources that are being used will use up a connection, so it is possible to share out many terabytes of projects without running into the 20 connection limitation of a Manifold 9 Universal license, and thus keeping licensing costs very low.

Tips for TCP Server

The following tips can help performance and convenience when running Server as a TCP Server:

 

 

 

 

 

 

 

 

 

 

 

Read/Write Access

Server is currently read-only.   Manifold expects to expand Server into providing read/write services.  That would allow multiple users to edit the same layer at the same time.   Until Server gains read/write capability, the solution for organizations that want multi-user editing is to store their data in an enterprise database such as MySQL, SQL Server, Oracle, or PostgreSQL/PostGIS and to connect to that data from Manifold.  When connecting to data stored in such multiuser databases, multiple users can edit the same drawings or other components at the same time.

 

Queries and the TCP Server

The query engine always runs queries from data sources for the TCP server remotely.  That is a slightly different behavior than applies with data sources that are databases, because when a Manifold client connects to Manifold Server the system knows that Manifold Server can handle queries in Manifold SQL.  To understand that better, we can consider the logic of how Manifold handles queries within different data sources:

 

As is well known, Manifold allows accessing data from data sources other than the main .map file within a query.  For example:

 

--SQL9

SELECT * FROM [server]::[data];

 

The above query will return records from the table named data on the data source named server.

 

If data is a query, however, the logic is more complicated.  All data sources may run queries using the Manifold query engine, with Manifold SQL syntax. Some data sources, for example, databases like PostgreSQL or SQL Server, may also run queries using their own query engine, with their own syntax.

 

If the data source for server has its own query engine, the system checks whether data should be run using the server's native query engine or using the Manifold query engine. The system does that by inspecting the text of the query and looking for a $manifold$ directive.

 

If data is a native query, the system dispatches it to the server to run it. If data is a Manifold query, the system runs it on the Manifold client, since only the Manifold client knows how to run a Manifold SQL query, and a server other than Manifold does not know how to run a Manifold query.

 

The same logic also applies to:

 

--SQL9

EXECUTE [[ SELECT * FROM [data]; ]] ON [server];

 

In the above, it looks like the query asks the server to run SELECT * FROM [data]; ...That is superficially true, but to be precise, the system is asking the data source to run the SELECT.  The data source then follows the same logic as discussed for the first example query above: it checks whether data is a native query or a Manifold query, and then either asks the server to run it if it is not a Manifold query, or if it is a Manifold query, runs it on the Manifold client.  That makes sense if one assumes that only Manifold clients know how to run Manifold queries written in Manifold's implementation of SQL.

 

All of the above is good logic when non-Manifold servers do not know how to run Manifold queries.  But if the data source is Manifold Server running as a TCP server, then both of the above example queries will run on the server and not on the client even if the query was a Manifold query.  That is because, unlike other types of server data sources, Manifold Server can run Manifold queries.  The general logic used by data sources to decide where a query should be run has been updated in current Manifold builds to incorporate the exception that Manifold Server can run Manifold queries.

 

Before that enhancement, Manifold Server data sources already were running queries on the server in many circumstances. For example, if the served .map file had a query component, connecting to the server and then opening and running that query would run it on the server.  Connecting to the server with a Manifold client and then opening a command window for the server and running statements in that command window would also run those statements on the server.  

 

But opening a command window for the main .map file and running one of the above two example queries, or queries similar to those, would run the query on the client because of the previous logic that a data source should always run a Manifold query on the Manifold client.   That logic has now been updated so attempting to run queries on a Manifold Server data source will now always run them on Server

 

Notes

Credentials encryption - Credentials are stored in database and other data sources in plain text.   Manifold expects to provide encryption options so data sources can be shared with Server without exposing credentials.

 

TCP - Manifold clients connect to Server running as a TCP server through TCP protocol, a standard part of networking.

 

Exclusive use - A Manifold project being shared by Server cannot be opened or used by a different Server session or by a regular Manifold desktop session.   To update that project, the Server session using it must be shut down.  Server cannot share a project that is in use in a regular Manifold desktop session or which is being shared by a different Server session.

 

Exclusive binding - For security, Server binds to the network interface in exclusive mode.

 

Console title - When launched in a Command Prompt window, Server sets the console title to Server - addr:port - filename. This hides sensitive information like user logins and passwords and it also shows connection information even if that was not spelled out in the command line.   The console window will show the server type (HTTP or TCP) in the window caption when Server is run in a console window.

 

Cache - Server projects can be arranged to provide persistent cache for external data sources.  See the Cache in Server Projects topic.

 

Non-routable IP addresses - The examples in this documentation use non-routable IP addresses beginning with 192....  Such IP addresses may be visible on your local network, but they cannot be visited across Internet.  When connecting to Server through the Internet, either for sharing data using the TCP server or for publishing maps using the HTTP server, the computer should have a static IP address that is routable (visible over Internet), like 20.81.111.85, and the Server instances that are launched should use that IP address.

 

See Also

Manifold Server

 

Server: Publish Web Maps

 

Server: WMS Server

 

Cache in Server Projects

 

Favorite Data Sources

 

File - Create - New Data Source

 

Tools - Manage Services

 

Command Window

 

Query Builder