Tag Archives: geographic information systems

KyTopo: Kentucky’s New Topographic Map Series

For many decades, the USGS Topographic Quadrangle Map series was “THE” base for anything mapping related. A wide array of Federal programs required that they be utilized for submittals, surveyors included them on plats as inset maps, and Kentucky-specific statutes relied on them for defining things like “blue line” streams. The “topo” or “quad” maps framed our perception of the nation’s surface and the planimetric features situated upon it.

In the mid 90s, USGS brought the topographic map into the digital age by releasing their Digital Raster Graphics (DRG) product. Finally, the basemap we’d relied on for ages was now in the background on our massive CRT monitors. For better or worse, our relationship to the maps was strengthened as we became mesmerized by the slow rendering raster image on our screens. GIS users quickly realized that the vector data they’d been creating and maintaining for years didn’t always line up perfectly with the DRGs. As a result, agencies quickly scrambled to adjust their points, lines, and polygons to the almighty map that we held as gospel. You know . . . they were made by USGS so they had to be correct!

USGS Kentucky DRG Image
USGS Kentucky DRG Image

Flash forward to December of 2011 when the Kentucky Aerial Photography and Elevation Program (KYAPED), or KyFromAbove effort, became a reality. With a focus on acquiring state-of-the-art LiDAR data and full-color, high resolution aerial photography, KyFromAbove promised a new view of the Commonwealth’s surface with a level of detail and clarity that had never been imagined. Since that time, and not surprisingly, one of the primary requests made by stakeholders in the GIS Community has been new LiDAR-derived contour lines.

In early 2017, as the impending reality of a statewide elevation model came into focus, the Kentucky Division of Geographic Information (DGI) began to investigate the creation of a statewide contour dataset and associated cartographic products and web services. Consultation with the Commonwealth’s GIS gurus prompted us to think outside of the box. Sometimes, bending the rules is necessary, even though it may lead to more complex processes and require more resources.

Although it was painful at times, going through this process was fruitful in that the KyTopo Map Series was conceived. The idea was to create a cartographic product that could be printed and shared as a cached web mapping service. Then, the primary derivative datasets (i.e. contours, hillshade, spot elevations, . . .) would be made available for download and published to the KyGeoNet. Two statewide contour datasets are slated for creation. One will be somewhat aligned with the USGS contour intervals in Kentucky (10’, 20, & 40’) and another statewide set of 5’ contours will leverage scale threshold settings and group layer functionality to adjust dynamically to the viewer’s map scale.

The KyTopo map series will be Kentucky-specific in several ways. First, an entirely new set of landscape-oriented quadrangle tiles have been developed. These new tiles align with our 5k tiling scheme and are in Kentucky’s Single Zone coordinate system rather than the traditional USGS UTM-based maps. The map area is exactly 30” wide (60,000’) by 20” tall (40,000’) and fits nicely on a standard Arch D (24” x 36”) printed page. The typical 1:24,000 (1” – 2000’) scale has been maintained and there are only 549 tiles as opposed to the 779 tiles offered by USGS. Importantly, the new tiles have square corners unlike the UTM version!

KyTopo Page Layout
KyTopo Page Layout

Next on the agenda was the task of coming up with names for the new map tiles. In many instances, the names from the old USGS quads were directly adopted. However, there were many cases where it just didn’t make sense. DGI staff studied the USGS methodology used to name its maps many years ago and employed that approach. Basically, the most prominent feature within a given tile was used for the name. For example, the largest city or the most centrally located place name (GNIS) was selected. In undeveloped areas, State Parks, State Forests, and natural features (i.e. streams, ridges, lakes, . . .) were used for naming the tile they dominated.

After pondering contour interval and index values, the standard USGS 10’, 20’ and 40’ intervals were embraced along with their associated 40’, 100’ and 200’ indices. Having these different interval levels is necessary due to great variance in elevation change as you move from east to west across the Commonwealth. Generally, the KyTopo intervals by tile, align fairly well with their USGS counterpart.

20’ Interval & 100’ Index
20’ Interval & 100’ Index

Weeks were spent gathering authoritative data for the project and fine tuning the map layout and symbology. Lots of thought was put into which layers should be included and which should be cast aside. As previously noted, all of the data is available in the public domain and much of it has been sourced directly from Kentucky State Government and a variety of Federal agencies. In fact, over 75% of the layers were already published to the KyGeoNet. It is anticipated that the maps will be updated on an annual basis so changes in the transportation network, forest cover, boundaries and other features can be updated accordingly.

The production hardware environment for the KyTopo project consists of one VMware-based 64-bit virtual server with 4 cores, 64GB of RAM, and 2TB of fiber-attached storage running Windows Server 2012 R2. Esri’s ArcGIS Desktop 10.3.1 and its Data Driven Pages functionality are the primary tools utilized for cartographic production. Additionally, Esri’s Production Mapping extension has proven useful in terms of handling layout constraints and customized map elements.

The map output process is done through a series of Python scripts that take advantage of the 64-bit background processing option within ArcGIS. Using this approach, the time taken to output each map was reduced by 10-15 minutes per map. Initially, draft maps were output to 300dpi PNG files for proofing and review. Feedback during this proofing process was used to further refine various aspects of the map series. Two types of georeferenced maps will be produced for distribution: collared and un-collared. The collared version will include all layout elements (i.e. legend, inset maps, title, . . . ) whereas the un-collared ones will be limited to the area within the data frame.

KyTopo Draft Map
KyTopo Draft Map

Significant effort was put into establishing an appropriate symbol set for this map series, and DGI staff consulted the USGS Topographic Map Symbols publication for guidance. Some of its symbols were adopted while others were adjusted to enhance readability. Once a symbol was selected, it was added to the KyTopo.style library using the ArcGIS Style Manager. This tool has proven to be an effective and efficient way to manage symbology for the production effort.

ArcGIS Style Manager
ArcGIS Style Manager

Typography, or cartographic labeling, is another aspect of this process that has received a tremendous amount of attention. Feature labels, titles, legends and notes are just some of the text-based components of a cartographic product. Using the correct typeface for each element is critical when compiling a quality map. Discounting the importance of typography during cartographic production can lead to a map that is difficult to interpret or one that draws the user’s attention away from the subject. Per the principles of thematic map design, serif typefaces were used for the map title, layout elements such as graticule labels, and hydrographic features.  Road shields, road names, elevation features, and some selected layout elements employ sans serif typefaces. Feature label placement was handled by enabling Maplex functionality in the map document. Hours were devoted to fine tuning the Maplex labeling rules so more prominent features were placed prior to those of less importance.

Every great map series includes both index and inset maps so users can see the location of each map tile in context. The KyTopo index map is situated in the upper right corner of the layout and highlights the given map tile on a statewide view. The inset map has been positioned below the legend and shows the eight tiles that surround each map. Including both of these components on the map has proven to be quite helpful. Using separate data frames for each, in conjunction with Data Driven Pages functionality, made this an automatable task.

KyTopo Index Map
KyTopo Index Map

One of the most frustrating layout elements to deal with was the legend. ArcGIS has some nice legend generation tools but DGI was unable to fully utilize its output. The legend created with the automated tools just didn’t produce the most desirable results. For example, the symbol for spot elevations would appear in the legend but the actual elevation value would not be placed beside it. Reluctantly, a partially manual process that uses Adobe Photoshop and Illustrator was put into place. It is a bit cumbersome, but the resulting legend is much nicer and better reflects the features within the map series.

KyTopo Draft Legend
KyTopo Draft Legend

At the time of the writing of this article, draft maps for KyTopo have only been generated for 394 of the 549 tiles. This is being driven by the current availability of LiDAR data for the Commonwealth. Statewide LiDAR coverage should be a reality by the end of 2017. Once that has been completed, contours will be generated for the remaining tiles and final map production will commence. It is planned that the map products and ancillary data elements will be made available on the KyGeoNet by Spring of 2018.

KyTopo Status Map
KyTopo Status Map

Stay tuned for final release dates and updated product specifications!

See you on the trail!

Kentucky’s Enterprise GIS Implementation – A Decennial Update

Background

A little over ten years ago an article entitled “GIS as a Utility: Kentucky’s Enterprise Implementation” appeared in Esri’s 2005 summer edition of ArcNews. The article provided an overview of Kentucky’s new Enterprise GIS implementation, discussed the challenges associated with making spatial data accessible in a networked environment, and highlighted how various agencies were beginning to leverage this new resource. It has been over a decade since that article’s publication, and the Commonwealth’s enterprise GIS has matured and become a critical component of business processes both within and outside State Government. This article outlines the efforts managed by the Kentucky Division of Geographic Information (DGI) that have made this implementation successful and sustainable throughout evolving technologies, tight budget cycles and changing policy directions.

The Keystone

One of the least desirable tasks in information technology is documentation. No one likes to document code, routine processes, or data for that matter. In the GIS world, creating “metadata” is the task that no one likes to see on their “to do” list but successful implementations require that it is completed on a layer-by-layer basis without exception. Like it or not, metadata is the keystone of Kentucky’s spatial data services. It has been tough at times to get agencies on this bandwagon, but once they’ve bought in, the value becomes more apparent. The policy states, “All spatial data resources shall have a complete metadata record in order to be included in the Commonwealth’s spatial data repository.” The existence of and strict adherence to this policy has had a positive impact on geospatial data integrity that carries on today.

Back in 2005, the Kentucky Geography Network (KyGeoNet) was based upon ArcIMS Metadata services, and there were less than 50 metadata records created by a half a dozen publishers. The whole concept of creating metadata was new and many just saw it as something else they had to do on top of everything else on their plate. For many GIS practitioners, it was a low priority until they realized that the Commonwealth’s spatial data repository would not grow if metadata was not created for the valuable resources scattered across the GIS community.

KyGeoNet Homepage
KyGeoNet Homepage

The value in making geospatial data accessible certainly outweighs the amount of effort required to complete a minimal metadata record and publish it to a repository. Essentially, this is a form of “cost avoidance” in that if users can find a resource, read the metadata to learn about its characteristics, and then download it for their use, they are less likely to call or e-mail with questions or requests. This becomes quite apparent once you’ve fielded 20+ calls about a given dataset and then realize that a published metadata record could have answered questions about the resource. Being asked the same question over and over again makes for grumpy GIS staff.

Fast forward to 2016 and there are now approximately 700 metadata records being actively managed by over two-dozen responsible publishers on what is known as the KyGeoPortal. The service is currently based upon a recent release of Esri’s open source product known as Geoportal Server. This solution includes the ability to search for and create metadata records associated with web mapping and image services, as well as downloadable datasets, static map images, standards documents and web mapping applications. It is a wonderful resource that is leveraged frequently by thousands of users on a monthly basis.

KyGeoPortal Search Page
KyGeoPortal Search Page

Average Monthly FTP Server Requests for 2016 (through July):
5.9 million requests/several thousand unique visitors

Enterprise Approach

As noted in the 2005 article, KyVector and KyRaster are the primary enterprise-based services that are accessible to the hundreds of ArcGIS Desktop and CAD users within Kentucky State Government. Each user is on the Wide Area Network (WAN) and enjoys fast, network-based access to these geospatial services hosted at the Commonwealth Computing Center in Frankfort, Kentucky. This level of service would not be possible without Commonwealth Office of Technology’s (COT) robust WAN and server infrastructure as the delivery mechanism.

Centralizing the data repository has reduced direct agency storage costs and eliminated the issue of outdated data residing on agency-based file servers. Agencies no longer have to expend resources on updating data that originates from external sources. All data is current, easily accessible, and includes associated metadata. This level of data availability and integrity promotes confidence in results when GIS technologies are utilized for inventory, planning, response, or business process decision support.

KyVector

KyVector, an ArcSDE-based server solution that runs in conjunction with Microsoft SQL Server is hosted on three separate physical boxes. One server (kysdewww) is dedicated to providing vector-based data to State Government agencies that host their own web mapping services and applications. Another server (kysdewan) is reserved for the hundreds of ArcGIS Desktop users on the WAN, while the third is used for staging and failover/outage scenarios. This proven configuration reduces contention for server resources and ensures that data delivery to users and applications is carried out in the most efficient and effective manner.

KyVector Layers in ArcMap
KyVector Layers in ArcMap

All data resources housed in KyVector have met the prerequisites of having a full metadata record and having been published to the KyGeoNet. In 2005, it was noted that 125 layers were in KyVector. Today, the number has more than doubled and stands at just over 300 layers. New layers are added periodically as they are created or if there is a need to share the resource across the enterprise.

Access to KyVector has been streamlined by using layerfiles that include connectivity properties, appropriate display scale thresholds, and suggested symbology settings. The organization, maintenance and distribution of the layerfiles is carried out by DGI. Users can start a new, or add to an existing, map document with ease by browsing a thematically organized folder of layerfiles.

Weekly data updates to KyVector, are fed from the agencies to the staging server then subsequently moved to production during the update procedure. Users access this resource with confidence knowing that it is the most current version available at any given time.

KyRaster

A decade ago, KyRaster was ArcSDE-based but newer technologies have prevailed. ArcGIS Image Server is now employed to provide users with access to raster-based data such as aerial photography, topographic maps, digital elevation models, and land cover. The primary benefit of ArcGIS Image Server over raster datasets stored in ArcSDE is the significant reduction in the time needed to process raster data so that it is accessible to end users. In the past, days or even weeks would be consumed just loading imagery into a raster dataset in ArcSDE. But now, with mosaic datasets and Image Server, we are able to get data out to the users in hours, or days at the very most.

KyRaster Layer in ArcMap
KyRaster Layer in ArcMap

Additionally, DGI’s raster storage footprint has been reduced by employing function chains within mosaic datasets. This functionality allows us to store the data once but serve it up in multiple projections, with different band combinations, or with special rendering options. For example, a function can be inserted into the chain of a digital elevation model (DEM) that allows it to be rendered as hillshade, shaded relief, or percent slope. The DEM is only stored once on disk, however the functions can be applied to each request on the fly.

As with KyVector, ArcGIS Desktop users connect to KyRaster using layerfiles that include connection properties, display scale thresholds, compression settings and custom rendering options. The performance of Image Server is on par with ArcSDE on the WAN, and those that aren’t on the State’s network have learned to change compression settings on the client-side to achieve better performance on slower networks. This approach has worked well in state field offices, at local governments, and by private sector users such as engineers, surveyors, and consultants.

Web Mapping Services

Both KyVector and KyRaster are the backbone of the Commonwealth’s publically accessible web mapping and image services. Dynamic and cached map services are hosted on kygisserver.ky.gov and provided in Web Mercator projection using ArcGIS Server. These services power hundreds of web mapping applications internal and external to State Government. The Commonwealth Base Map and the Commonwealth Street Base Map are the two primary cached services, but other statewide imagery and land cover layers have been cached as well.

The Commonwealth Base Map Cached Service
The Commonwealth Base Map Cached Service

The balance of the services are considered to be dynamic, as they pull data directly from KyVector to fulfill every single server request. Each time a user zooms in, zooms out, pans one direction or another, a request is made to the map server which in turn reaches out to KyVector for the data.

Average Monthly Map Server Requests for 2016 (through July):
9 million requests/several thousand unique visitors

Kentucky’s image and elevation services are hosted on kyraster.ky.gov in Kentucky Single Zone Projection using ArcGIS Server and the Image Server extension. Employees with ArcGIS Desktop in State Government are the primary users of this server, however there are a growing number of users outside the WAN. When ArcSDE was the server solution for imagery, only users on the WAN could gain access. Image Server now allows the Commonwealth to meet the needs of internal and external users with the same computing infrastructure yet still maintain a high level of performance and availability.

KyFromAbove Image Service Rendered Using an IR Function Chain
KyFromAbove Image Service Rendered Using an IR Function Chain

Average Monthly Image Server Requests for 2016 (through July):
7.5 million requests/several thousand unique visitors

KyGovMaps

Like many other users of geospatial technologies, DGI has incorporated ArcGIS Online (AGOL) into its solutions and workflows. AGOL is Esri’s scalable and secure software-as-a-service cloud-based mapping platform. It includes a rich collection of web mapping application templates, visualization and analytical tools, and ready-to-use data sources that make geospatial resources more accessible and user friendly. DGI’s organizational site on AGOL, KyGovMaps, showcases a wide variety of Kentucky-specific web maps, applications, and services. Data and maps that were once only usable by those with expensive GIS software are now available to most everyone. GIS isn’t just for propeller heads anymore!

KyGovMaps on AGOL
KyGovMaps on AGOL

A customized gallery page on the KyGeoNet makes it easy to locate featured web maps, mapping applications, and thematic galleries that interest a wide array of users. Lately, some of the most popular applications have been those in our collection of Story Maps. Recreational resources, imagery, historic landmarks, demographics, elevation data, and physiographic regions are a sampling of the topics covered using story map templates that we’ve customized.

Kentucky Weather Mapping Application
Kentucky Weather Mapping Application

All of the current web mapping applications hosted by DGI were built using a customizable Javascript-based template. These applications employ a responsive framework and thus can be used within any browser and on any platform. Find a trailhead at a State Park, check the weather, get the soil characteristics for where you are standing, or locate the closest place to put your boat in the water and do it using your desktop, tablet or phone!

Kentucky Bucket List Story Map
Kentucky Bucket List Story Map

The Next Decade

If the pattern continues, combined total server requests for the 2016 calendar year will easily exceed 250 million. This alone is a true testatment to the success of the Commonwealth’s Enterprise GIS implementation. The growth in adoption and usage of geospatial technologies during the past decade has been astounding, and Kentucky’s capabilities in this realm have kept up with the pace of advancement. It will be interesting to look back at this, and the preceding 2005 article, during the middle of the next decade to see where we stand. Live data feeds, UAV acquired imagery, and 3D visualization are just some of the new trends that will shape the makeup of the Kentucky’s Enterprise GIS down the line. Stay tuned!