You are here: Home > Articles

Accessing raster cell values programmatically: the problem of mismatched data types.

This article relates to an automation error message that is sometimes generated when using raster datasets with scripts or extensions written in Visual Basic. That error message is: "Variable uses an automation type not supported in Visual Basic"

In order to understand why this error arises it is necessary to delve into the technical aspects of how raster data is stored. As we know, raster data is essentially a collection of numbers, arranged in a grided fashion, that represents some aspect of the world. There are a number of raster data formats that can be used to store these number: the three common ones used by ArcGIS are Grids, GeoTiffs, and ERDAS Imagine files. What is not at all apparent is that within these files, there are many different data types that can be used to store the numbers themselves. ArcGIS employs the following 14 data types:

Constant Value Description
PT_U1 0 Pixel values are 1 bit.
PT_U2 1 Pixel values are 2 bits.
PT_U4 2 Pixel values are 4 bits.
PT_UCHAR 3 Pixel values are unsigned 8 bit integers.
PT_CHAR 4 Pixel values are 8 bit integers.
PT_USHORT 5 Pixel values are unsigned 16 bit integers.
PT_SHORT 6 Pixel values are 16 bit integers.
PT_ULONG 7 Pixel values are unsigned 32 bit integers.
PT_LONG 8 Pixel values are 32 bit integers.
PT_FLOAT 9 Pixel values are single precision floating point.
PT_DOUBLE 10 Pixel values are double precision floating point.
PT_COMPLEX 11 Pixel values are complex.
PT_DCOMPLEX 12 Pixel values are double precision complex.

Table 1. Data types used to store numbers in raster datasets.

The reason there are a number of data types to choose from is because the data type affects file size and speed of processing. For example, one bit can have one of two values: 0 or 1. If we are storing boolean data (True/False) in theory only one bit/cell is needed to store that information. If we were to use the PT_LONG data type to store True/False data, 31 of 32 bits available to us per cell would be unused: this would increase the file size and slow down any processing of that grid (it takes longer to read a 32 bit value than a 1 bit value).

For a most operations on raster datasets (e.g. using ArcObjects), we can ignore the data type because ArcObjects deals with it automatically. However, in cases where we might want to directly access the raster values in order to perform a custom analysis we face the problem that not all of these data types are available to us in the Visual Basic development environment (this problem applies to other languages too). In Visual Basic 6 we have the following data types to work with:

Data type Storage size Range
Byte 1 byte 0 to 255
Boolean 2 bytes True or False
Integer 2 bytes -32,768 to 32,767
Long (long integer) 4 bytes -2,147,483,648 to 2,147,483,647
Single (single-precision floating-point) 4 bytes -3.402823E38 to -1.401298E-45 for negative values; 1.401298E-45 to 3.402823E38 for positive values
Double (double-precision floating-point) 8 bytes -1.79769313486232E308 to -4.94065645841247E-324 for negative values; 4.94065645841247E-324 to 1.79769313486232E308 for positive values
Currency (scaled integer) 8 bytes -922,337,203,685,477.5808 to 922,337,203,685,477.5807
Decimal 14 bytes +/-79,228,162,514,264,337,593,543,950,335 with no decimal point; +/-7.9228162514264337593543950335 with 28 places to the right of the decimal; smallest non-zero number is +/-0.0000000000000000000000000001
Date 8 bytes January 1, 100 to December 31, 9999
Object 4 bytes Any Object reference
String (variable-length) 10 bytes + string length 0 to approximately 2 billion
String (fixed-length) Length of string 1 to approximately 65,400
Variant (with numbers) 16 bytes Any numeric value up to the range of a Double
Variant (with characters) 22 bytes + string length Same range as for variable-length String

Table 2. Data types used to store numbers in Visual Basic 6

In order to compare this table to Table 1 it is necessary to know that there are 8 bits in 1 byte. It is also important to realize that data types come in a signed and unsigned variety. Taking Visual Basics 2 byte integer as an example (2 bytes = 16 bits), we can deduce that the range of values it could be used to represent is 0 to 2^16 (2 to the power of 16) which is 0 to 65536. But if we use the first bit to record whether that number is positive or negative, the range of values then becomes +/- 2^15, or -32768 to 32767.

If you compare Table 1 to Table 2 you will notice that there are some raster data types in Table 1 that do not have an equivalent data type in Table 2. The following are the raster data types that have an equivalent in Visual Basic 6:

PT_UCHAR = Byte (unsigned 1 byte)
PT_SHORT = Integer (signed 2 bytes)
PT_LONG = Long Integer (signed 4 bytes)
PT_DOUBLE = Double (signed 8 bytes)

Thus, if you were using a method that attempted to set a variable in VB to hold a raster value (e.g. using the IPixelBlock.GetVal method) you would have to ensure that the VB variable matched the raster data type, and that the raster data type was one of the above four types. If you do not, you will either generate the error message shown at the top of the screen, or you could incorrectly assign a data type (i.e. you are programmatically able to set a PT_CHAR cell value to a Byte variable, but you get a bizarre result if you do – do not do this!).

The situation is slightly improved in Visual Basic .NET, which has a greater number of data types as shown below:

Visual Basic type Abbrev Common language runtime type structure Nominal storage allocation Value range
Boolean bln System.Boolean 2 bytes True or False.
Byte byt System.Byte 1 byte 0 through 255 (unsigned).
Char cha System.Char 2 bytes 0 through 65535 (unsigned).
Date dtm System.DateTime 8 bytes 0:00:00 on January 1, 0001 through 11:59:59 PM on December 31, 9999.
Decimal dec System.Decimal 16 bytes 0 through +/-79,228,162,514,264,337,593,543,950,335 with no decimal point; 0 through +/-7.9228162514264337593543950335 with 28 places to the right of the decimal; smallest nonzero number is +/-0.0000000000000000000000000001 (+/-1E-28).
Double (double-precision floating-point) dbl System.Double 8 bytes -1.79769313486231570E+308 through -4.94065645841246544E-324 for negative values; 4.94065645841246544E-324 through 1.79769313486231570E+308 for positive values.
Integer or Int32 int System.Int32 4 bytes -2,147,483,648 through 2,147,483,647.
Long or Int64 (long integer) lng System.Int64 8 bytes -9,223,372,036,854,775,808 through 9,223,372,036,854,775,807.
Object obj System.Object (class) 4 bytes Any type can be stored in a variable of type Object.
Short or Int16 shr System.Int16 2 bytes -32,768 through 32,767.
Single (single-precision floating-point) sng System.Single 4 bytes -3.4028235E+38 through -1.401298E-45 for negative values; 1.401298E-45 through 3.4028235E+38 for positive values.
String (variable-length) str System.String (class) Depends on implementing platform 0 to approximately 2 billion Unicode characters.
User-Defined Type (structure)   (inherits from System.ValueType) Depends on implementing platform Each member of the structure has a range determined by its data type and independent of the ranges of the other members.

Table 3. Data types used to store numbers in Visual Basic .NET

In this case the following raster data types can be directly accessed:

PT_UCHAR = Byte (unsigned 1 byte)
PT_USHORT = Char (unsigned 2 byte) ***
PT_SHORT = Int16 (signed 2 bytes)
PT_DOUBLE = Double (signed 8 bytes)
PT_LONG = Int32 (signed 4 bytes)

What is the solution to this problem?

The solution I developed was to write a program that converts raster data types into one of the four data types that can be directly accessed by VB6. The tool is available for free in HawthsTools2 and is called Convert Raster Data Type.

Author: Hawthorne Beyer ([email protected])

  Home   |   Hawth's Tools   |   Articles   |   Services