Tag Archives: ArcGIS

Kentucky’s Enterprise GIS Implementation – A Decennial Update

Background

A little over ten years ago an article entitled “GIS as a Utility: Kentucky’s Enterprise Implementation” appeared in Esri’s 2005 summer edition of ArcNews. The article provided an overview of Kentucky’s new Enterprise GIS implementation, discussed the challenges associated with making spatial data accessible in a networked environment, and highlighted how various agencies were beginning to leverage this new resource. It has been over a decade since that article’s publication, and the Commonwealth’s enterprise GIS has matured and become a critical component of business processes both within and outside State Government. This article outlines the efforts managed by the Kentucky Division of Geographic Information (DGI) that have made this implementation successful and sustainable throughout evolving technologies, tight budget cycles and changing policy directions.

The Keystone

One of the least desirable tasks in information technology is documentation. No one likes to document code, routine processes, or data for that matter. In the GIS world, creating “metadata” is the task that no one likes to see on their “to do” list but successful implementations require that it is completed on a layer-by-layer basis without exception. Like it or not, metadata is the keystone of Kentucky’s spatial data services. It has been tough at times to get agencies on this bandwagon, but once they’ve bought in, the value becomes more apparent. The policy states, “All spatial data resources shall have a complete metadata record in order to be included in the Commonwealth’s spatial data repository.” The existence of and strict adherence to this policy has had a positive impact on geospatial data integrity that carries on today.

Back in 2005, the Kentucky Geography Network (KyGeoNet) was based upon ArcIMS Metadata services, and there were less than 50 metadata records created by a half a dozen publishers. The whole concept of creating metadata was new and many just saw it as something else they had to do on top of everything else on their plate. For many GIS practitioners, it was a low priority until they realized that the Commonwealth’s spatial data repository would not grow if metadata was not created for the valuable resources scattered across the GIS community.

KyGeoNet Homepage
KyGeoNet Homepage

The value in making geospatial data accessible certainly outweighs the amount of effort required to complete a minimal metadata record and publish it to a repository. Essentially, this is a form of “cost avoidance” in that if users can find a resource, read the metadata to learn about its characteristics, and then download it for their use, they are less likely to call or e-mail with questions or requests. This becomes quite apparent once you’ve fielded 20+ calls about a given dataset and then realize that a published metadata record could have answered questions about the resource. Being asked the same question over and over again makes for grumpy GIS staff.

Fast forward to 2016 and there are now approximately 700 metadata records being actively managed by over two-dozen responsible publishers on what is known as the KyGeoPortal. The service is currently based upon a recent release of Esri’s open source product known as Geoportal Server. This solution includes the ability to search for and create metadata records associated with web mapping and image services, as well as downloadable datasets, static map images, standards documents and web mapping applications. It is a wonderful resource that is leveraged frequently by thousands of users on a monthly basis.

KyGeoPortal Search Page
KyGeoPortal Search Page

Average Monthly FTP Server Requests for 2016 (through July):
5.9 million requests/several thousand unique visitors

Enterprise Approach

As noted in the 2005 article, KyVector and KyRaster are the primary enterprise-based services that are accessible to the hundreds of ArcGIS Desktop and CAD users within Kentucky State Government. Each user is on the Wide Area Network (WAN) and enjoys fast, network-based access to these geospatial services hosted at the Commonwealth Computing Center in Frankfort, Kentucky. This level of service would not be possible without Commonwealth Office of Technology’s (COT) robust WAN and server infrastructure as the delivery mechanism.

Centralizing the data repository has reduced direct agency storage costs and eliminated the issue of outdated data residing on agency-based file servers. Agencies no longer have to expend resources on updating data that originates from external sources. All data is current, easily accessible, and includes associated metadata. This level of data availability and integrity promotes confidence in results when GIS technologies are utilized for inventory, planning, response, or business process decision support.

KyVector

KyVector, an ArcSDE-based server solution that runs in conjunction with Microsoft SQL Server is hosted on three separate physical boxes. One server (kysdewww) is dedicated to providing vector-based data to State Government agencies that host their own web mapping services and applications. Another server (kysdewan) is reserved for the hundreds of ArcGIS Desktop users on the WAN, while the third is used for staging and failover/outage scenarios. This proven configuration reduces contention for server resources and ensures that data delivery to users and applications is carried out in the most efficient and effective manner.

KyVector Layers in ArcMap
KyVector Layers in ArcMap

All data resources housed in KyVector have met the prerequisites of having a full metadata record and having been published to the KyGeoNet. In 2005, it was noted that 125 layers were in KyVector. Today, the number has more than doubled and stands at just over 300 layers. New layers are added periodically as they are created or if there is a need to share the resource across the enterprise.

Access to KyVector has been streamlined by using layerfiles that include connectivity properties, appropriate display scale thresholds, and suggested symbology settings. The organization, maintenance and distribution of the layerfiles is carried out by DGI. Users can start a new, or add to an existing, map document with ease by browsing a thematically organized folder of layerfiles.

Weekly data updates to KyVector, are fed from the agencies to the staging server then subsequently moved to production during the update procedure. Users access this resource with confidence knowing that it is the most current version available at any given time.

KyRaster

A decade ago, KyRaster was ArcSDE-based but newer technologies have prevailed. ArcGIS Image Server is now employed to provide users with access to raster-based data such as aerial photography, topographic maps, digital elevation models, and land cover. The primary benefit of ArcGIS Image Server over raster datasets stored in ArcSDE is the significant reduction in the time needed to process raster data so that it is accessible to end users. In the past, days or even weeks would be consumed just loading imagery into a raster dataset in ArcSDE. But now, with mosaic datasets and Image Server, we are able to get data out to the users in hours, or days at the very most.

KyRaster Layer in ArcMap
KyRaster Layer in ArcMap

Additionally, DGI’s raster storage footprint has been reduced by employing function chains within mosaic datasets. This functionality allows us to store the data once but serve it up in multiple projections, with different band combinations, or with special rendering options. For example, a function can be inserted into the chain of a digital elevation model (DEM) that allows it to be rendered as hillshade, shaded relief, or percent slope. The DEM is only stored once on disk, however the functions can be applied to each request on the fly.

As with KyVector, ArcGIS Desktop users connect to KyRaster using layerfiles that include connection properties, display scale thresholds, compression settings and custom rendering options. The performance of Image Server is on par with ArcSDE on the WAN, and those that aren’t on the State’s network have learned to change compression settings on the client-side to achieve better performance on slower networks. This approach has worked well in state field offices, at local governments, and by private sector users such as engineers, surveyors, and consultants.

Web Mapping Services

Both KyVector and KyRaster are the backbone of the Commonwealth’s publically accessible web mapping and image services. Dynamic and cached map services are hosted on kygisserver.ky.gov and provided in Web Mercator projection using ArcGIS Server. These services power hundreds of web mapping applications internal and external to State Government. The Commonwealth Base Map and the Commonwealth Street Base Map are the two primary cached services, but other statewide imagery and land cover layers have been cached as well.

The Commonwealth Base Map Cached Service
The Commonwealth Base Map Cached Service

The balance of the services are considered to be dynamic, as they pull data directly from KyVector to fulfill every single server request. Each time a user zooms in, zooms out, pans one direction or another, a request is made to the map server which in turn reaches out to KyVector for the data.

Average Monthly Map Server Requests for 2016 (through July):
9 million requests/several thousand unique visitors

Kentucky’s image and elevation services are hosted on kyraster.ky.gov in Kentucky Single Zone Projection using ArcGIS Server and the Image Server extension. Employees with ArcGIS Desktop in State Government are the primary users of this server, however there are a growing number of users outside the WAN. When ArcSDE was the server solution for imagery, only users on the WAN could gain access. Image Server now allows the Commonwealth to meet the needs of internal and external users with the same computing infrastructure yet still maintain a high level of performance and availability.

KyFromAbove Image Service Rendered Using an IR Function Chain
KyFromAbove Image Service Rendered Using an IR Function Chain

Average Monthly Image Server Requests for 2016 (through July):
7.5 million requests/several thousand unique visitors

KyGovMaps

Like many other users of geospatial technologies, DGI has incorporated ArcGIS Online (AGOL) into its solutions and workflows. AGOL is Esri’s scalable and secure software-as-a-service cloud-based mapping platform. It includes a rich collection of web mapping application templates, visualization and analytical tools, and ready-to-use data sources that make geospatial resources more accessible and user friendly. DGI’s organizational site on AGOL, KyGovMaps, showcases a wide variety of Kentucky-specific web maps, applications, and services. Data and maps that were once only usable by those with expensive GIS software are now available to most everyone. GIS isn’t just for propeller heads anymore!

KyGovMaps on AGOL
KyGovMaps on AGOL

A customized gallery page on the KyGeoNet makes it easy to locate featured web maps, mapping applications, and thematic galleries that interest a wide array of users. Lately, some of the most popular applications have been those in our collection of Story Maps. Recreational resources, imagery, historic landmarks, demographics, elevation data, and physiographic regions are a sampling of the topics covered using story map templates that we’ve customized.

Kentucky Weather Mapping Application
Kentucky Weather Mapping Application

All of the current web mapping applications hosted by DGI were built using a customizable Javascript-based template. These applications employ a responsive framework and thus can be used within any browser and on any platform. Find a trailhead at a State Park, check the weather, get the soil characteristics for where you are standing, or locate the closest place to put your boat in the water and do it using your desktop, tablet or phone!

Kentucky Bucket List Story Map
Kentucky Bucket List Story Map

The Next Decade

If the pattern continues, combined total server requests for the 2016 calendar year will easily exceed 250 million. This alone is a true testatment to the success of the Commonwealth’s Enterprise GIS implementation. The growth in adoption and usage of geospatial technologies during the past decade has been astounding, and Kentucky’s capabilities in this realm have kept up with the pace of advancement. It will be interesting to look back at this, and the preceding 2005 article, during the middle of the next decade to see where we stand. Live data feeds, UAV acquired imagery, and 3D visualization are just some of the new trends that will shape the makeup of the Kentucky’s Enterprise GIS down the line. Stay tuned!