Downloadable Free GIS Data Sets are provided in 2 formats: ESRI shapefiles and Geodatabase. The appropriate directory structures are compressed into one file using WinZip. Connect to to access all files within that directory. Vector data are stored as ESRI shapefiles Grid (raster) data are stored as DIVA gridfiles Each 'shapefile' consist of at least three actual files. This is a commonly used format that can be directly used in Arc-anything, DIVA-GIS, and many other programs.
| Filename extensions | .shp, .shx, .dbf |
|---|---|
| Developed by | Esri |
| Type of format | GIS |
| Standard | Shapefile Technical Description |
The shapefile format is a popular geospatial vector data format for geographic information system (GIS) software. It is developed and regulated by Esri as a (mostly) open specification for data interoperability among Esri and other GIS software products.[1] The shapefile format can spatially describe vector features: points, lines, and polygons, representing, for example, water wells, rivers, and lakes. Each item usually has attributes that describe it, such as name or temperature.
- 1Overview
- 2Limitations
Overview[edit]
The shapefile format is a digital vector storage format for storing geometric location and associated attribute information. This format lacks the capacity to store topological information. The shapefile format was introduced with ArcView GIS version 2 in the early 1990s. It is now possible to read and write geographical datasets using the shapefile format with a wide variety of software.
The shapefile format is simple. It stores the data as primitive geometric shapes like points, lines, and polygons. These shapes, together with data attributes that are linked to each shape, create the representation of the geographic data.
The term 'shapefile' is quite common, but is misleading since the format consists of a collection of files with a common filename prefix, stored in the same directory. The three mandatory files have filename extensions.shp, .shx, and .dbf. The actual shapefile relates specifically to the .shp file, but alone is incomplete for distribution as the other supporting files are required. Legacy GIS software may expect that the filename prefix be limited to eight characters to conform to the DOS 8.3 filename convention, though modern software applications accept files with longer names.
- Mandatory files
.shp— shape format; the feature geometry itself.shx— shape index format; a positional index of the feature geometry to allow seeking forwards and backwards quickly.dbf— attribute format; columnar attributes for each shape, in dBase IV format
- Other files
.prj— projection description, using a well-known text representation of coordinate reference systems.sbnand.sbx— a spatial index of the features.fbnand.fbx— a spatial index of the features that are read-only.ainand.aih— an attribute index of the active fields in a table.ixs— a geocoding index for read-write datasets.mxs— a geocoding index for read-write datasets (ODB format).atx— an attribute index for the.dbffile in the form of shapefile.columnname.atx(ArcGIS 8 and later).shp.xml— geospatial metadata in XML format, such as ISO 19115 or other XML schema.cpg— used to specify the code page (only for.dbf) for identifying the character encoding to be used.qix— an alternative quadtree spatial index used by MapServer and GDAL/OGR software
In each of the .shp, .shx, and .dbf files, the shapes in each file correspond to each other in sequence (i.e., the first record in the .shp file corresponds to the first record in the .shx and .dbf files, etc.). The .shp and .shx files have various fields with different endianness, so an implementer of the file formats must be very careful to respect the endianness of each field and treat it properly.
Shapefile shape format (.shp)[edit]
The main file (.shp) contains the geometry data. The binary file consists of a single fixed-length header followed by one or more variable-length records. Each of the variable-length records includes a record-header component and a record-contents component. A detailed description of the file format is given in the ESRI Shapefile Technical Description.[1] This format should not be confused with the AutoCAD shape font source format, which shares the .shp extension.
The 2D axis ordering of coordinate data assumes a Cartesian coordinate system, using the order (X Y) or (Easting Northing). This axis order is consistent for Geographic coordinate systems, where the order is similarly (longitude latitude). Geometries may also support 3- or 4-dimensional Z and M coordinates, for elevation and measure, respectively. A Z-dimension stores the elevation of each coordinate in 3D space, which can be used for analysis or for visualisation of geometries using 3D computer graphics. The user-defined M dimension can be used for one of many functions, such as storing linear referencing measures or relative time of a feature in 4D space.
The main file header is fixed at 100 bytes in length and contains 17 fields; nine 4-byte (32-bit signed integer or int32) integer fields followed by eight 8-byte (double) signed floating point fields:
| Bytes | Type | Endianness | Usage |
|---|---|---|---|
| 0–3 | int32 | big | File code (always hex value 0x0000270a) |
| 4–23 | int32 | big | Unused; five uint32 |
| 24–27 | int32 | big | File length (in 16-bit words, including the header) |
| 28–31 | int32 | little | Version |
| 32–35 | int32 | little | Shape type (see reference below) |
| 36–67 | double | little | Minimum bounding rectangle (MBR) of all shapes contained within the dataset; four doubles in the following order: min X, min Y, max X, max Y |
| 68–83 | double | little | Range of Z; two doubles in the following order: min Z, max Z |
| 84–99 | double | little | Range of M; two doubles in the following order: min M, max M |
The file then contains any number of variable-length records. Each record is prefixed with a record header of 8 bytes:
| Bytes | Type | Endianness | Usage |
|---|---|---|---|
| 0–3 | int32 | big | Record number (1-based) |
| 4–7 | int32 | big | Record length (in 16-bit words) |
Following the record header is the actual record:
| Bytes | Type | Endianness | Usage |
|---|---|---|---|
| 0–3 | int32 | little | Shape type (see reference below) |
| 4– | – | – | Shape content |
The variable-length record contents depend on the shape type, which must be either the shape type given in the file header or Null. The following are the possible shape types:
| Value | Shape type | Fields |
|---|---|---|
| 0 | Null shape | None |
| 1 | Point | X, Y |
| 3 | Polyline | MBR, Number of parts, Number of points, Parts, Points |
| 5 | Polygon | MBR, Number of parts, Number of points, Parts, Points |
| 8 | MultiPoint | MBR, Number of points, Points |
| 11 | PointZ | X, Y, Z Optional: M |
| 13 | PolylineZ | Mandatory: MBR, Number of parts, Number of points, Parts, Points, Z range, Z array Optional: M range, M array |
| 15 | PolygonZ | Mandatory: MBR, Number of parts, Number of points, Parts, Points, Z range, Z array Optional: M range, M array |
| 18 | MultiPointZ | Mandatory: MBR, Number of points, Points, Z range, Z array Optional: M range, M array |
| 21 | PointM | X, Y, M |
| 23 | PolylineM | Mandatory: MBR, Number of parts, Number of points, Parts, Points Optional: M range, M array |
| 25 | PolygonM | Mandatory: MBR, Number of parts, Number of points, Parts, Points Optional: M range, M array |
| 28 | MultiPointM | Mandatory: MBR, Number of points, Points Optional Fields: M range, M array |
| 31 | MultiPatch | Mandatory: MBR, Number of parts, Number of points, Parts, Part types, Points, Z range, Z array Optional: M range, M array |
Shapefile shape index format (.shx)[edit]
The index contains the same 100-byte header as the .shp file, followed by any number of 8-byte fixed-length records which consist of the following two fields:
| Bytes | Type | Endianness | Usage |
|---|---|---|---|
| 0–3 | int32 | big | Record offset (in 16-bit words) |
| 4–7 | int32 | big | Record length (in 16-bit words) |
Using this index, it is possible to seek backwards in the shapefile by, first, seeking backwards in the shape index (which is possible because it uses fixed-length records), then reading the record offset, and using that offset to seek to the correct position in the .shp file. It is also possible to seek forwards an arbitrary number of records using the same method.
Shapefile attribute format (.dbf)[edit]
This file stores the attributes for each shape; it uses the dBase IV format. An alternative format that can also be used is the xBase format, which has an open specification, and is used in open source shapefile libraries, such as the Shapefile C library.[2]
The names and values of attributes are not standardized, and will be different depending on the source of the shapefile.
Shapefile spatial index format (.sbn)[edit]
This is a binary spatial index file, which is used only by Esri software. The format is not documented by Esri. However it has been reverse-engineered and documented [3] by the open source community. It is not currently implemented by other vendors. The .sbn file is not strictly necessary, since the .shp file contains all of the information necessary to successfully parse the spatial data.
Limitations[edit]
Topology and the shapefile format[edit]
The shapefile format does not have the ability to store topological information. The ESRI ArcInfo coverages and personal/file/enterprise geodatabases do have the ability to store feature topology.
Spatial representation[edit]
The edges of a polyline or polygon are composed of points. The spacing of the points implicitly determines the scale at which the feature is useful visually. Exceeding that scale results in jagged representation. Additional points would be required to achieve smooth shapes at greater scales. For features better represented by smooth curves, the polygon representation requires much more data storage than, for example, splines, which can capture smoothly varying shapes efficiently. None of the shapefile format types supports splines.
Data storage[edit]
The size of both .shp and .dbf component files cannot exceed 2 GB (or 231 bytes) — around 70 million point features at best.[4] The maximum number of feature for other geometry types varies depending on the number of vertices used.
The attribute database format for the .dbf component file is based on an older dBase standard. This database format inherently has a number of limitations:[4]
- While the current dBase standard, and GDAL/OGR (the main open source software library for reading and writing shapefile format datasets) support null values, ESRI software represents these values as zeros — a very serious issue for analyzing quantitative data, as it may skew representation and statistics if null quantities are represented as zero
- Poor support for Unicode field names or field storage
- Maximum length of field names is 10 characters
- Maximum number of fields is 255
- Supported field types are: floating point (13 character storage), integer (4 or 9 character storage), date (no time storage; 8 character storage), and text (maximum 254 character storage)
- Floating point numbers may contain rounding errors since they are stored as text
Mixing shape types[edit]
Because the shape type precedes each geometry record, a shapefile is physically capable of storing a mixture of different shape types. However, the specification states, 'All the non-Null shapes in a shapefile are required to be of the same shape type.' Therefore, this ability to mix shape types must be limited to interspersing null shapes with the single shape type declared in the file's header. A shapefile must not contain both polyline and polygon data, for example, the descriptions for a well (point), a river (polyline), and a lake (polygon) would be stored in three separate datasets.
See also[edit]
- Open Source Geospatial Foundation (OSGeo)
External links[edit]
- Shapefile file extensions – Esri Webhelp docs for ArcGIS 10.0 (2010)
- shapelib.maptools.org – Free c library for reading/writing shapefiles
- Python Shapefile Library – Open Source (MIT License) Python library for reading/writing shapefiles
- Shapefile Projection Finder - Detect unknown projection of a shapefile automatically[5][6]
References[edit]
- ^ abESRI (July 1998). 'ESRI Shapefile Technical Description'(PDF). Retrieved 2007-07-04.Cite journal requires
|journal=(help) - ^'Shapefile C Library V1.2'.
- ^'Wayback Machine'(PDF). 13 August 2016. Archived from the original on 13 August 2016.Cite uses deprecated parameter
|deadurl=(help)CS1 maint: BOT: original-url status unknown (link) - ^ ab'ArcGIS Desktop 9.3 Help – Geoprocessing considerations for shapefile output'. Esri. April 24, 2009.
- ^Egger, Manfred. 'Shapefile Projectionfinder'(PDF). www.egger-gis.at.
- ^'Shapefile Projectionfinder'.
I want to create and edit Shapefiles for my application, and I'm looking for a free shape (ESRI) file editor - free for creating and editing.
nmtoken5 Answers
There are lots of free GIS tools available. One of the imho best is QuantumGIS: http://www.qgis.org. It is available for win/mac and linux.
But maybe you prefer openjump a tool written in java, which has special strengths in editing, topology-checking: http://www.openjump.org/
Also take a look at: http://freegis.org/
underdark♦Also, if you do not mind some scripting in Python, there is Shapefile module for this.
Small example, create .shp with point features from coords in .xls:
del f, Path, wb,
If you just want to view shapefiles quickly, http://www.mapdoc.co.uk does quick shapefile reading in the browser (dont need to download anything or upload file). You need modern browser though
EverlyMap is A Free Shapefile Editor you might like. Does the whole job. Windows Desktop App. from http://www.everlymap.com