Using data from our mobile phones, raster and vector data
We are going to look at some GPX data (the type that comes off your phone when collecting tracks and waypoints using GPSLogger, OpenGPX, and other methods (like GPS units). We will then take it into QGIS and ArcPro to do some analysis with it.
A GPX file is the standard file for sharing information from devices (GPS/GNSS device) or mobile apps. It is an XML based file format that records a point location at a certain interval – time based or distance based. Here is an example of the GPX file we will be playing with this week.
The top of the file:
Click the image for a larger view
End of the file:
As you can see the text in the file (much like we saw with the Postalcode text file) will create a system of points than can be joined to create lines. In fact for this lab, Scott has taken the line from the first track point:
<trkpt lat="53.89354394" lon="-122.82148698"><ele>779.975</ele><time>2020-08-20T22:46:10Z</time><speed>0.000</speed><sat>13</sat></trkpt>
and pasted it to the end of the file and saved it as a new file (we will see this in the lab), so a polygon can be created from the file.
Adding a GPX file to software
We are at the point where we want to use the software that makes life the easiest. ArcPro can read GPX files, but you need to use a processing tool – QGIS just opens the file (so we will use QGIS this week).
- Open QGIS
- Add in the GPX files in L:\GEOG204\lab6 folder (scott_roger_explore.gpx and scott_roger_explore_poly.gpx)
- Select the track points and tracks only for each dataset
- You will end up with two sets of data with almost the same names in the TOC, but you can tell them apart by the completeness of the track lines and the extra point in the “poly” file
- Don’t mistake the ‘poly’ in the name for a polygon layer: see by the icons in the table of contents that it is still a line layer
- Add in a web map background to see where the data is located – the ESRI Topo layer provides a nice view of the terrain
Investigate the data
You can see from looking at the data, we lost satellite reception often in our walk, but there are plenty of places to check out data under the track points. You can also see there are elevation values collected from the GNSS app in Scott’s phone. Lets compare it with a DEM from 2018 LiDAR data.
DEM data with point data
Sample raster data
Consider what we are doing by sampling raster data: wherever there is a point in the vector layer, the corresponding raster value will be saved into the vector layer’s attribute table.
- Add in the scott_roger_explore_dem layers into the project (Drag and Drop or Layer –> add Raster Layer)
- Find the sample raster values in the processing toolbox
- Fill out the panel appropriately – it does not matter which point layer you use as they are the same data
- Look at the new layer’s attribute table and note the column name that was added
Do some math
- Open the attribute table of the new layer
- Use the Field Calculator to create a new column that computes the difference between the GPX elevation and the LiDAR DEM data
- Style the layer to see the differences: if you are illustrating a range of values, what symbolization style would it make sense to use (single symbol, categorized, graduated, etc.)?
What kind of range do you get in elevation difference? Quite the error in elevation values from mobile phone GNSS data collectors!
Clean up your GPX data
- Have a look at the attribute table of your sampled and calculated layer, and take note of all the fields that are not useful to you
- Right click on the sampled and calculated layer –> properties
- Select the Fields tab in the panel
- Ensure the layer is in edit mode (toggle the yellow pencil icon)
- Select all the fields you are not interested in keeping (probably most of them)
- Hit the delete field button to remove the fields
- Turn off the edit mode – turn off the pencil
- Close the properties panel
Export as a new file type
Export the data in the usual fashion, BUT, we’re going to export into new format, a Geopackage:
- Export as a GEOPACKAGE
- Export the data in the local UTM projection (EPSG: 26910)
- Call the layer whatever you wish
Copy the Styling from the Original Dataset
- Right click > Styles > Copy Style > All Style Categories OR Symbology
- Right Click the new Geopackage > Styles > Paste Style > All Style Categories OR Symbology
Save the New Styling to Your Geopackage
- Right click on your geopackage layer > Properties > Symbology
- In the bottom left corner of the Symbology tab, click Style > Save Style
- Change Save Style to In Database (GeoPackage) –> give the style a name and description
Create a polygon and get some statistics
Using the GPX layer that has the potential to be made into a polygon (scott_roger_explore_poly.gpx), create a polygon layer using methods we’ve used in previous labs.
Once that is completed, sample under the polygon to get elevation and slope data.
Here are some suggested steps:
- Create a polygon from the track layer as you did in lab 2 (remember the Line to Poly and Polygonize tools?)
- Run the Fix Geometries tool on your new polygon layer
- Create a slope layer from the DEM layer as you did in lab 5 or check (search the processing toolbox) to see if you can do so in QGIS – it is very simple
- Sample under the polygon using the zonal statistics tool with the following parameters:
- include count, min, max, standard deviation, mean
- Sample both the DEM layer and the Slope layer: use the prefix “dem_” for the DEM sampling and “slope_” for the slope layer
- Check out the sample images below
Clean up the unnecessary attribute fields like we did earlier and save this as a second geopackage (no need to worry about styling)
Check against the track points
Check the elevation differences you sampled with the point data (where Scott and Roger walked)
- Find the Basic Statistics for fields tool
- Run this on the geopackage dataset for the sampled points (from Part 1) using the LiDAR elevation values (the field called something like SAMPLE_1) as the “Field to calculate statistics on”
- Click run, but don’t close the window yet: the report will be generated in the log pane
- How do the min, max and mean values compare here (from the point layer) to those calculated in the polygon layer using the zonal statistics tool and the DEM layer? You may wish to jot them down somewhere since you’ll have to navigate to another window.
- If you mistakenly close the window too soon, you can find the results from the tool you just ran in the Results Viewer pane by double clicking on Statistics:
Let’s review what we just did:
In Part 1, we sampled elevation data against the track points from the GPX data, then calculated the statistics for those sampled data just now. In the attribute table, each point had an entry and had its elevation saved after we sampled it. To see the statistics of the elevation data stored in the attribute table, we ran Basic Statistics for Fields, and created a separate output with the statistics.
In Part 2, we calculated the elevation statistics for the area covered by the polygon that used Scott and Roger’s hike as the perimeter. The summary statistics that we requested were then saved to their own fields in the resulting attribute table of the new polygon output.
New project and assignment
Save your project and start a new project.
Adding a DEM layer for Prince George
We are going to add a DEM layer for Prince George and test it against a trail layer. We first need to determine which DEM to use for Prince George.
Determine the mapsheet number
- Add the 250k_sheets_albers shape file from L:\DATA\250K_mapsheets_BC
- Set the projection of your canvas to UTM Zone 10 (26910). This is not necessary, but it is helpful.
- Zoom in a bit toward Prince George to help the web map load in the next step.
- Add in a web map (OSM standard).
- Set the transparency of 250K layer to enable viewing of Prince George.
- Determine which mapsheet we will need to use based on its A250K_TAG number – there are a few different ways we can do this:
- Use the Identify Features button and click on the polygon that includes Prince George.
- Select the polygon that includes Prince George, then go check out its attributes in the attribute table.
- Modify the Labels (the tab below Symbology in Layer Properties) to display the information that you want for each polygon.
- You should find the value 093G.
Grabbing the DEM to match this mapsheet
Back in the File Browser:
- Find the zip file containing the proper DEM from the mapsheet value in L:\DATA\dem25m_utm
- Unzip this file to your K drive (or you can use the unzipped one in this week’s lab folder)
- You should have two files unzipped: 93g-dem25m-utm10.asc and 93g-dem25m-utm10.prj
- Drag the .asc file into QGIS
- Also add the Trails layer from: L:\GEOG204\lab6 to the canvas
- Remove the 250K layer
Clean up the trails
You can see all the trails except Ferguson Lake (north of the DEM) fit on the DEM. Remove this trail loop by:
- Save the Trails layer to your K: drive as a geopackage
- Edit the layer –> remove the loop
- Save your edits
- Check if it worked by Zooming to Layer of your edited Trail layer – it should now exclude the Ferguson Lake trail loop, and all of your trail layer should fall within the DEM you just imported
Description of Assignment
This week’s lab is a bit more work and so will be worth 10% rather than 5% like the other assignments.
We want to see the difference between using a couple of tools for determining how difficult a trail may be to walk compared to checking points along a line sampled against a slope layer. You will carry out this check using the following steps:
- Use the Drape tool on your trail layer to get elevation values (“Z” values) from the DEM layer we just imported to create a 3D line layer (1 mark). Check out the description for each of these tools on the right hand side of the tool window to learn more about what each tool does.
- Use the Climb Along Line tool on the output to generate some climbing attributes for this layer (1 mark)
- Create a new line layer by using the Densify by Interval tool with a 20 meter spacing using your trail layer as input (1 mark)
- Create a new point layer by extracting these new vertices created from the densified line using the Extract Vertices tool (1 mark)
- Create a slope layer from the DEM (1 mark)
- Sample the new point layer against the slope layer (1 mark)
- Style the climb layer (use the ‘descent’ field) and the sampled point layer (use the SAMPLE_1 field) as graduated using the same colour scheme with the same number of classes for both layers in order to compare them visually (2 marks: 1 for each layer’s styling, shown in your screenshots)
For your write-up, describe the steps you took to get your results (1 mark). Use the tool descriptions to help you describe what you are doing in each step. Feel free to include screenshots of the different steps, but this is not mandatory. Do include screenshots of your two resultant layers (one line layer, one point layer) against the slope layer you created as a base layer (2 marks as mentioned in the final bullet point above).
Below is an example of this type of output. It’s a zoomed in portion of the trails, but for the assignment we would like to see all of the Prince George trails (excluding Ferguson Lake). Make sure to include your Layers panel in the screenshot so we can see the classes you sorted by (1 mark).