Geospatial Modelling Environment

file.append (Append Text Files)

Appends text files located in the same folder


This tool appends text files located in the same folder. It is likely that you will want to use both of the optional settings for this tool. The first option, 'match', is a search string filter: the tool will only process files with names that match the specified search string. This is useful because it provides a means of preventing other files that may be in the specified folder from being erroneously included in the appended output file. The second option, 'skip', is a boolean setting that controls whether the first line is skipped in all appended files except the first one. This is useful because many data files have a header line that contains the column headings, but you would only with to include a header line once in the appended file.

Note that the file extension does not matter. Text files can have many different extensions (txt, dat, csv, etc.) so the tool does not automatically filter files based on the extension unless you use the 'match' expression (e.g. match=*.txt"). Nor does the tool automatically ignore binary files. You must either ensure that only text files are in the specified folder, or use the match option to ensure that only the appropriate text files are processed. If a binary file is processed the output will contain strange characters and will be useless.

The standard search pattern rules for Windows apply to the match option. Some examples are: *.txt returns all files with a txt extension, rain*.dat returns all files that begin with 'rain' and have a dat extension, and *data* return all files with word 'data' somewhere in the filename.

An example serves to illustrate the use of this tool. Many datasets are delivered in simple text file format (because it is platform and software independent) and can be downloaded from a website. These data are usually delivered in manageable chunks: divided into regions or time periods or both. Importing them into a useful format would take a great deal of time if they were processed individually. Given that we only want to run the import procedure once, this tool provides a means of appending all of the files into a single input file that we can then import.

A number of tools have been developed to manipulate text files. They were developed to make it more efficient to import datasets delivered in text file format (e.g. weather datasets that are delivered as separate files for each year, or XML files containing spatial data), but are likely to be useful for a wide range of tasks. See the commands beginning with 'file' for further information.


file.append(folder, out, [match], [skip]);

folderthe path of the folder containing the text files to append
outthe full path to the new output text file
[match]a search string filter: only files matching this search string will be processed (use the wildcard character * in the expression - see the help documentation for further details)
[skip](TRUE/FALSE) if TRUE skips the first line of all files apart from the first one (default=FALSE)


file.append(folder="C:\data", out="C:\data\appended.txt", match="RAIN", skip=TRUE);


Please consider making a purchase to support the continued development of these tools  Read more...

Tips on how to use this interface efficiently 


Open Source GIS

Copyright © 2001-2014 Hawthorne L. Beyer, Ph.D., Spatial Ecology LLC    Connect on LinkedIn