Adding Data to our Maps
In today’s lab, we are going to explore the various kinds of data that we can add to our maps. Broadly speaking this data comes in two main types, Vectors and Rasters. Vectors come in Points (waypoints), Lines (Roads or Tracks from a GPS), and Polygons (Outlines such as lakes or buildings). Rasters are imagery this could be air photos or other data that has been rasterized. For example the Lidar we used in the first lab, started as point data and was rasterized into the digital elevation model we used.
Start a Project
As before create a new project in ArcGIS Pro, called Lab3 inside of your GEOG205 folder on your K: drive.
Use the catalog pane and add L:\GEOG205\lab03 as a folder connection.
Vector Data will be the primary source of data we will be using in this class and we will be looking at various ways of adding it. Our primary motivations for using Vectors are that we generally get smaller file sizes than rasters, the ability to store metadata about individual features, and a great deal of control over how to symbolize the data.
Shapefiles are an ageing file format but still can be thought of as industry standard, as nearly all GIS software will open a shapefile.
Take a look in L:\GEOG205\Lab03\shapefiles\unbc inside the catalogue pane of ArcGIS Pro you will see a variety of files listed. However, what will be perhaps more interesting is opening this same folder in windows file explorer you will notice that there are in fact many more files here as a shapefile is stored in multiple pieces
you will see a set of files there with various file extensions.
- .shp – File containing geometry
- .shx – Positional index of feature geometry
- .dbf – Contains attribute data for features
- .prj – File containing projection information (optional, but highly recommended)
- .sbn – Contains spatial indexes (optional file, but dramatically increases draw speed in large layers)
Inside of ArcGIS these are all presented as a single file, however, when interacting with shapefiles outside of ArcGIS care needs to be taken to keep these files together, and if one is renamed, they must all be renamed.
Add the features.shp, roads.shp and lakes.shp
features is a very generic name so to help keep our table of contents clean double click the layer in the drawing order panel and rename it to “UNBC Features”
The data we just added comes in three types of geometry: points, lines, and polygons. Each layer can only represent a single type of geometry, if we have a mixed data set, such as rivers where large rivers are represented by polygons, and creeks as lines, we would either need two layers or convert the geometry. (ie. Rivers as a line representing each bank as opposed to polygons.)
GPX or GPS Exchange format is a file type you may come across while mapping data collected in surveys. In comparison to shapefiles, the GPX is a single file, and not only that the file can contain all three geometry types at the same time (you will still only be able to load one geometry type per layer).
However ArcGIS Pro will not directly open these files in many cases (will not read line or polygon features). For this reason, we will Open QGIS
Once you have QGIS open, in the windows file explorer go to the folder L:\GEOG205\Lab03\gps_data and then drag and drop unbc_biodome.gpx onto QGIS.
The dialogue presented shows the layers inside of the GPX files as well as the number of features in each layer. In this case, there is a track (recording of where the GPS was) represented as both a line and points. In addition to 3 waypoints. Select all, and Add layers to the group and add.
In QGIS you will see the lower-left corner has a layers pane that works much the same fashion as Drawing Order in ArcGIS Pro. You can turn the layers on and off to change what you see on the map.
Right click on the waypoints layer >>> Export >>> Save Feature As
Save as an ESRI Shapefile, and using the … button to the right of the File name browse to your lab3 folder and save as unbc_biodome_waypoints.shp
Do the same for the track points and track line layers and then add all three layers into ArcGIS Pro.
KML is Keyhole Markup Language (KMZ is a compressed zip version of KML) and is the primary file type used by Google Earth, due to the ease of access of Google Earth KML / KMZ files are generally common in areas where data is produced without the need of a full GIS system, KML has an extra advantage over the other files we have looked at so far in that they also contain a very basic amount of styling information in the form of line colour and in the case of polygons fill colour.
Add L:\GEOG205\Lab03\kml\Northern Sports Centre.kmz to your project and notice that the polygon automatically appears in UNBC’s Green.
It is often common to get data in the form of spreadsheets, one such place to get such data is GeoNames. https://www.geonames.org/
On this website go to Downloads and Free Zip Postal Code Data
Download CA_full.csv.zip and extract it into your lab3 folder.
Open Excel and open an existing file, note that you will need to change the file type filter from Excel files to all files
Once you select the .txt file it will prompt with a dialogue to help excel understand what type of data is contained within the file.
On the first page make sure that the Original Data Type is set to Delimited, then proceed to Next.
On this page, we need to make sure the data preview looks like a spreadsheet. In this case, our Delimiter is a tab, and because there are some blank cells we need to make sure Treat consecutive delimiters is disabled so our data is not shifted after an empty row.
At this point you can press finish.
To help ArcGIS understand this data we will simply save it as a CSV file or Comma Separated values, essentially we have just replaced Tab with a comma in the file.
And open this file in ArcGIS Pro. You will at this point notice that it does not show up on the map, but rather at the bottom of the Drawing order as a “Standalone Table”. ArcGIS Pro knows it is a table but not how to interpret the data. Right-click on CA_Full.csv in the drawing order and choose open. The table will appear below the map, scroll to the side and make note of which Fields contain the Longitude and Latitude, we will need this information in the next step.
To make it show up on the map we need to add XY Point Data.
We will then tell ArcGIS which Fields contain the positional data.
As an alternative, you could add the .txt file to QGIS Directly and then export it as we did the GPX File before. There is always more than one way to solve a problem!
Before Proceeding, right click on the Postal Code layers and remove them from your project.
This CSV layer is relatively large and needs to be removed before proceeding.
The final form of vector input we will come to spatial databases. These range from file-based databases (Geodatabases, Spatialite) to Enterprise Database Management Systems DBMS (Such as Enterprise Geodatabase, PostgreSQL, MS-SQL with spatial extensions).
You have already been interacting with an Enterprise Geodatabase in this course while using the ArcGIS portal on campus. The ArcGIS Portal is storing all of its data in a PostgreSQL database; these systems are the highest performance option for storing data and provide the ability to work with large rapidly changing data. Outside of the portal we also operate PostgreSQL servers on campus that store and serve TRIM, OpenStreetMap (entire globe), and BC Vegetation Resource Index, all data sources that exceed the file limitations of Shapefiles. However you are unlikely to come across such systems outside of large organizations or smaller business that focuses on geospatial data.
File-based geodatabases are the default method in which ArcGIS Pro stores data, in fact, if you look in the Catalog pane you will find that there is already a lab3.gdb file in your lab3 folder, this is the geodatabase for your project, anytime you export data it will be placed into this file. Inside of ArcGIS, a geodatabase looks a lot like a folder, but to windows explorer, it is a single file, these are sometimes used to send someone multiple layers.
The other broad category of data that you will run into is raster, possibly more familiar referred to as images. Common examples of this would be air photos, satellite imagery, scans of paper maps, and digital elevation models.
We have already added raster layers to our maps in this class when we added the UNBC Campus, and the paper NTS map sheet. But now we are going to look at a scenario you may run into if an image either doesn’t contain spatial information or if this information is inaccurate.
Before we begin set the basemap to imagery so we have something to reference against.
Copy L:\GEOG205\lab3\raster\campus20cm_2010.jpg to your lab 3 folder, and open the copy in ArcGIS Pro.
Make sure your Map is set to ‘WGS 1984 UTM Zone 10N’ by right-clicking on map then properties and verify the Current XY setting. If it is not correct you can find the projection under Projected Coordinate System >>> UTM >>> WGS 1984 >>> Northern Hemisphere
The data we added should have appeared to the northwest of the campus somewhat skewed. To make this show up properly, this image must be ‘georeferenced’.
Select the image in the drawing order. Then go to imagery and georeference
This is a tool that we can use to show ArcGIS Pro where the imagery belongs. You may also need to do this process if the data is perhaps just poorly aligned.
This is going to be done in a 2 step process, first prepare, 2nd adjust (as a note if the image imported was just poorly referenced the prepare step could be skipped.
A great place to start is to zoom in on the campus, and press ‘fit to display’, this will cause the imported image to at least be in the general area of UNBC. You can then use the Move and Rotate tools to drag the image to the approximate location. Tip: Make sure the roads layer is still turned on and above the image in the drawing order so you can use it as a guide.
Next, we are going to Add Control Points, control points are created in pairs, first on the image we want to adjust, and second on the target or what we want to match against.
Click on Add Control Points, your cursor should change to “From point (source)”
Zoom in on UNBC and click as close as possible to the center of the campus compass between the buildings.
At this point your cursor should change to “To point (target)”
Hide the image you are trying to adjust in the Drawing order and click on the same point on the basemap.
When you turn back on the image you are adjusting you will notice it will snap into place. However the image may still have the wrong rotation or scale, to fix this we will add more points. You will have noticed that your cursor is once again at “From point (source)”
Pick 4 more control points to make. Some guidance in order of importance
- Control points must be in the same place between images
- Control points should be spread apart and distributed across the image
- Control points should be at ground level whenever possible, avoid trees, poles, and buildings as these can change with the camera perspective. If you must use these features pick a point at their base.
You should end up with something similar to this:
Notice how the road overlay is on top of the roads and relatively lined up.
When you are happy with your results select Save as New, save it in your folder using the TIFF file format.
What will be exported is a GeoTiff, which is a Tiff file with coordinates in the metadata allowing this image to be opened in GIS software without the need to georeferenced it again.
Some potential sources of raster data might be
If you are living outside Prince George your municipality may provide a similar service.
For Satellite photos: https://earthexplorer.usgs.gov/
Something you will very quickly discover if you have not already is that spatial data often contains errors that need to be fixed before it is suitable for mapping. This can come in several forms such as missing data that needs to be added, anomalous data to be removed, or data that just needs a little bit of an adjustment to its precise location.
Never edit original data, its always good to be able to go back if you need lets make a copy of the roads layer, right click, data, export features.
The only setting you should need to change is to provide the output name, by default it will save into your projects default geodatabase.
For this task select the roads layer and then the edit tab. Let’s start with a quick chat about the buttons available in the editor.
Before we can edit features we need to select what we want to edit, there is a drop-down under select for picking different selection tools when a rectangle selection is just not what you are looking for.
Attributes can be used to select features by values in the data table, for example, maybe you want to select all the creeks in a water courses layer. This is a feature better covered in GEOG 204 and 300.
And finally a Clear button, I would recommend always clicking this before selecting new features to make sure you are not editing features you have already finished with.
These tools work pretty much as you would expect, it is somewhat unusual to past within the same layer, however, one way to move features to a different layer is to cut, select the other layer in the drawing order then paste.
Here are two of the most important buttons, when we are finished editing save writes all the changes we made back to the original file. Discard is useful if you make a mistake potentially too far back to just undo actions, or just want to restart in general Discard will bring the layer back to the state it was in when you last pressed save.
This is the last section of the top ribbon we are interested in. Both Create and Modify will open new panels on the right over the top of the catalogue. While Delete will simply remove whatever features are selected
When the Create Features pane opens it will show you a list of editable layers, simply select the layer you wish to add features to. And you will be presented with a set of tools appropriate to the type of geometry contained in the layer.
Take a few minutes, try out the various tools. Do not Apply/Save your edits, we will discard them before moving on, the idea here is to get a feel for how the tools work.
Snapping is useful when we want to make edits without gaps. Snapping will assign our cursor coordinates to exactly match nearby features. Try turning it on, and hovering over a road feature. Does the cursor change to show what you are hovering on?
What happens if you hover over the middle of a line as opposed to the end?
Adding a road to the campus
Make sure snapping is turned on, and add a road to the roads layer leading to the powerplant on campus using the line tool, it should look like below. Ask your TA any questions you run into during this process.
Modifying features is possibly the most finicky process in the class, it’s not hard, but mistakes are common and there is no shame in the discard button and starting over!
Let’s clean up the road in the bus loop to make it better match the background.
Then in the Modify Features panel that appeared on the right find the Edit Vertices tool under Reshape.
Then click on the road in the bus loop once selected a new toolbar will appear at the bottom of the map.
The first tool from left to right is the lasso selection tool, you can use the drop-down to pick a different selection tool if you prefer.
Draw a selection over several vertices, then hover over one of the selected vertices until your cursor changes to “roads: Vertex” at this point you can click to drag the nodes.
The next tool is the add node tool, with this tool selected you can click on a line segment to add a vertex at that point and then click and drag to make corrections.
Following that is the Remove node tool, with this tool selected simply click on a vertex to remove it.
Finally in this section is the Continue Feature tool, this is much like the add vertex tool except instead of placing a vertex between existing vertices clicking on the map will create a vertex and connected to the most recently created vertex in the geometry.
Spend some time cleaning up the road and getting used to the tools.
Don’t forget if you make a mistake you can undo it with [Ctrl] + [z], or the back button in the upper left of ArcGIS Pro. Or make use of the Cancel button to undo all your recent edits.
Lab 3 Assignment
For this weeks lab you will be making a map. Your map will consist of several layers that you will need to import into your project and edit. As a minimum your map should also contain.
- Scalebar (with logical segments)
- Your Name and date
- Vector layers should be identifiable (ie. different colours and a legend) it it’s not perfect don’t worry we are doing symbology next week!
Your map will cover the trails near Shane lake with an extent similar to the photo below.
1. Add layers to the map
In the folder L:\GEOG205\Lab03\assignment_data you will find
- unbc2002.tif (add this to your map from here)
- trails.shp (copy this to your K drive as we will be editing it, and we don’t edit original data!)
2. Edit the trails layer to more closely align with the satellite photo
Do not edit the Raster, only the vector layer.
Notice the missing trail near the north of the map.
Notice that many of the trails are south of where they appear in the imagery
3. Find Additional Data
Part of working in GIS is locating the data sets that you need, use the website princegeorge.ca and find Water Enclosed Storage (This includes the water tower in Forests for the World) and add it to your map.
4. Add additional information to the map
Title, legend, scalebar, Your name, date.
5. Save your map as a PDF
E-Mail your completed assignment to Your TA.