This Weeks Lab
- Data organization for DEM creation with TRIM data
- DEM creation from TRIM data
- Data organization for DEM creation using TRIM data
- DEM creation from trim data
- More organizing your data
TRIM data for DEM creation
You could use NTS if TRIM data is not available – we are not going to do this in the lab this week though:
I have left the following NTS part in for those that may need it later.
We can get free data from the the Canadian Geographic Data Infrastucture (CGDI) via GeoGratis at:
GeoGratis Advanced Download Page
You should all be familiar with getting data form NRCAN in this fashion by now. I (Scott) prefer to go to the NRCan data download directory
To prepare these data we would:
- get the data from the ftp site – you could use wget to help with more than one sheet
- unzip these data into a directory
- select only the sheets you need
- once they are downloaded for all the sheets, you need to merge mapsheets layers together (i.e. contours with contours)
- reproject into the projection you want to work in and clip to the area you are interested in.
- Why would we want to use TRIM data over NTS data?
- Does this seem like lots of work?
Getting the TRIM data organized to use in our lab – do this section of the lab
We are going to log into the PCI server as usual but the steps we follow are not going to start with PCI. We are going to do the following:
- Load WMS data into QGIS to see our area
- Load the vector data from a zip file Scott Downloaded from GeoBC
- Load a data layer for clipping the vector data we are going to work with
- create a set of vectors for loading into PCI for DEM creation
QGIS – free and easy!
Setting up your canvas to look like this:
You need to go to firefox and go to:
Follow the links until you end up with the “Get Capabilities” link to the BC Albers Base Cache
BC Albers Base Cache –> right click on getCapabilities and copy link location.
ASIDE: You can also get a long (HUGE) list of WMS layers for BC at:
Now open QGIS and:
- Load WMS data into QGIS –> Layers load WMS layer
- Click New (adding a new WMS service)
- set the name as BC Gov Albers base map
- paste in the location of the URL you copied from firefox into the URL box
- Set on the fly projection
- Go to File
-> project properties
-> CRS tab
-> Enable on the fly transformations
-> filter 3005 and select BC Albers
- Load the vector data from a zip file Scott Downloaded from GeoBC
- grab the zip file from the lab3-dem directory in /home/labs/geog457 directory
- drag it into the TOC portion of QGIS
- pick the layers you would like to use (it is in Albers)
- Load a data layer for viewing/clipping the vector data we are going to work with
- Load the appropriate pgbound layer for clipping into the TOC
- Load in the LAS layer to see the smaller mapsheets we will be using for the Lidar data later on
- select CRS as UTM Zone North (NAD 83 or WGS 84)
You can now determine the are you would like to use for creating a DEM for PG using TRIM or LiDAR data. Scott actually used the LAS layer (dissolved and projected to BC Albers) as an extent for the TRIM data download for the lab.
As the data from LiDAR (later in the lab) is in UTM zone 10, perhaps you should consider projecting the data to UTM 10 before leaving QGIS for use in PCI by:
- right clicking on the layer
- save as – put the shape files in your directory
- set the CRS to UTM 10 in the same save as panel
DEM creation from TRIM data (1)
We can now create a DEM from the TRIM data. With PCI search out VDEMINT in the algorithm librarian. We are going to use it to create the dem with the layers we just clipped out.
To do this
- open VDEMINT and add the files to the input
- set the output to be greyscale in the view
- set the values in the input parameters
- we only need to set
- pixel size – 15 metres
- list of vector types – comma separated with the proper integer to describe the data type
- run the program and look at your results – check the breaklines to see if you made the right choice for type
- Create your own pgtrimdem.pix dataset
- How did you load the output into a dataset?
- Do we need 32 bit data – how could we put it in to the dataset with a proper bit number?
- What is that bit number?
If you have the TRIM Safe files
Organizing data for DEM creation using TRIM (.saf) data
If we had FME we would:…
We willl not do the same for DEM creation using TRIM data. We will be quickly looking at another piece of software you may have seen before called FME. We will use it to get the TRIM data in the format we want. Of course Scott has created the shape files from saf files prior to the lab.
To do this
- open a file browser in osmotar and navigate to L:\labs\geog457\dem_create_data\trim_from_saf
- doulble click on the shape2shape_clip.fmw file
- input the source files as all the shape files in N:\labs\geog457\dem_create_data\trim_from_saf
- output to a directory in your dem lab
- run the workbench file to get your results
- look the files you created to see what feature types they are – use the ogr command
- ogr -al shapefile | less
- ctrl c
- ogr name shapefile
- Why were there less shape files as output from FME than the number of input layers?
- Would this happen if we used ArcMap?
Free specialized version of FME
The reason there exists FME is funding from the provincial government to create a data type that can be used to unify all the TRIM data into format that can be easily extended and updated easily. This resulted in the beginnings of the software and a file format that unfortunately never took hold. We have all the data for the province in this format and use FME to convert it to other file types. Scott can use the free version to example how these data can be converted, but for the time being we will use files he already converted.
DEM from trim 1 data
The data for this exercise is in
Load it all into PCI for making a DEM with VDEMINT
DEMs from ASCII files
You can also grab elevation data from TRIM in 25 metre resolution in a text format. We have these files on our file server, but Scott has moved the relevant zip files into the lab3-dem directory.
Playing around with QGIS – optional (but you know Scott is going to show you it)
See if you can figure out how to load them into QGIS (HINT: the zip files work the same as the other zip file and the projection is UTM zone 10). I would also use a fresh QIS session and turn off the project on the fly option. You can use QGIS (under the raster –> translate) pulldown to convert the files to pix files from here if you are adventurous.
Using PCI to read the ascii files
You can unzip the files by opening them with the file archiver (double click) and extract them to your home directory. Once they are unzipped the files can be opened up directly in PCI.
Load the LAS shape file into focus and zone to it extents. Then:
- pick a favourite square that you would like to use for subsetting the ASCII Dem
- subset the ascii DEM based on the nearest round number (i.e 506550 – NOT 506534.78) coordinates
- if you pick an area that bridge across the ascii DEMs you need to mosiac them first (avoid this for now)
- also save a vector layer in the PCI file your create – pgasciidem
Creating a DEM from LiDAR ASCII data
We are only going to use one section for the dem building for this section of the lab. It took Scott several hours to figure out the best way to do the DEM build from LiDAR data, and In the end it was really quite simple. The best thing is that we get to use a bunch of Linux commands again!! You must be excited.
Looking at the raw data
We should understand the data we are about to use for the DEM, and be able to manipulate it easily and without waiting forever for things to load. By following the next steps we should be able to create a shape file for use in building the LiDAR DEM.
- pick a favourite square for the data from the LAS shape file in QGIS – note the name of the file
- open up QGIS and add the file you want by adding a delimited file
- navigate to /home/data/pg_city2010/projectAOI/BareEarth/ASCII
- grab the file (with xyz extension) that matches your square
- Open a terminal and navigate to the directory where you have this file
- type; more yourfile.xyz (use the numerical name of the file instead of yourfile)
- once you look at the data you can type the letter “q” to stop the read of the file
- Lets reduce the size of the xyz file in QGIS.
- select out and area of points
- save the file as a csv file
- open the file in gedit – remove the header and replace the commas with a space and save as a .txt file
- Open this text file in PCI and use it to make a DEM from it
What have we done just now?
Does this type of a file ring a bell for anyone (especially those that have suffered with a class with Scott before)?
Creating a DEM from LiDAR LAS data
We are going to look at the new data for 2014 we have for Prince George. The data has not been converted to ASCII yet so …
One of the best set of tools to deal with LAS data is LASTools. Scott has installed the files to our file system for us to use.
The LiDAR and ortho photos are all in /home/data/pg_lidar_rtho_2014. Make up a raster index shape file to see which file you would like to use. You can do this in QGIS as well as loading the OSM data into the project to see which one does the trick. Once you have selected the LAS file you want execute the following command example:
las2las -i /home/data/pg_lidar_orhto_2014/lidar2014/093G096421.las -o /home/emmons/geog457/dem/ground_96421.las –keep-classes 2 8
We now have a las file that is smaller than the original las file (only the soundings for the bare groud). We can use this LAS file in PCI, but of course we need to do some more work. You can run las2txt to convert it to a ASCII file and bring it into PCI as we did with the 2010 data above.
EASI commands to bring in LAS data
We are going to use the EASI command called VDEMINGEST. There is no algorithm for it, so we get to learn something new today! Once we create a vector segment from the command, we can use the data to create a DEM with VDEMINT.