GeoServer and OpenStreetMap

OpenStreetMap is a free and editable map of the world.  Founded in 2004 in the United Kingdom in response to the need for a free geospatial data source, it is a community-driven project, allowing for anyone to edit and contribute information.   It has since grown to include data from almost all countries in the world.  The map generated from their data can be used as an alternative to commercial map layers such as those found in Google Maps.  The OpenStreetMap base layer is rendered using Mapnik, however since the data is free to download and distribute, it is possible to serve it using GeoServer.   I recently went through the process of rendering OpenStreetMap’s data with GeoServer.

The data

OpenStreetMap provides instructions on downloading their data.  The full data set is a large file, currently about 4GB when compressed and about 100GB when uncompressed.  (You can download sections of the data set as well.)  This data is an XML-based data format that GeoServer cannot read natively.  PostGIS is the optimal way of storing such a large volume of data when using GeoServer.  Luckily, there is a converter that allows you to load the OpenStreetMap data into PostGIS called osm2pgsql.  This program is a package available on Debian-based distributions (such as Ubuntu), and is also available as a binary on Windows.

Run the following command:

osm2psql -E 900913 -d osm planet.osm

This will process the XML information and load the data into a PostGIS database called “osm”. The -E defines the projection of the source data, which in this case is 900913, the projection used in Google Maps, and the default for OpenStreetMap.

If successful, the database will contain the following tables:

planet_osm_line
planet_osm_point
planet_osm_polygon
planet_osm_roads

Making sense of the data

OpenStreetMap’s data can be a bit confusing.  For a first time user like myself, just trying to figure out their naming convention was challenging.  They do, however, have a wonderful map key hidden away in their wiki.  This page is a lifesaver.

There are two different tables that contain line data, planet_osm_line and planet_osm_roads. The former includes railroads, subways, and other linear information.  The latter is made up exclusively of roads.  The planet_osm_point table has a range of data: subway stations, shopping centers, universities, and even brothels. Lastly the planet_osm_polygon table has, but is not limited to, parks, bodies of water, and even buildings in certain urban areas.

Styling

On my map I grouped the roads into three major groups; residential, secondary, and limited-access (highways).  To achieve this grouping I used the following SLD filters.

Road class PropertyName Values
Residential highway residential OR unclassified
Secondary highway primary OR secondary
Limited-access highway motorway OR trunk

I then styled the map to distinctly display the three groups of roads, and also varied their widths depending on the zoom level.  This was the most time-consuming part of the process, as I needed to create and evaluate styles for each zoom level.  The final SLD is very large, but is available for those interested.

Final touches

Since the purpose of this map was to create a viable base layer that anyone can incorporate into their mapping projects, performance was a concern.  To address this, I used GeoWebCache, a tile cache mechanism built into GeoServer.  The GeoWebCache documentation has details on this process.

The finished (?) map

The finished product can be found at http://demo.opengeo.org/openstreetmap.

Feel free to link to this map.  Use this OpenLayers code to get started.  This map is a work in progress.  Future enhancements will include adding public transportation systems, railroads, buildings, and more. If you have any feedback on this map please email me at iwillig at opengeo dot org.

With GeoServer, PostGIS, OpenLayers and the OpenStreetMap data set, it is possible to build high-quality, professional-looking maps, allowing you to take control of what data you present, define your own mapping aesthetic, and free yourself from having to use commercial map layers.

29 Comments

  1. Posted 2009/01/30 at 2:17 pm | Permalink

    Great looking SLD, Ivan!

  2. Posted 2009/01/30 at 2:52 pm | Permalink

    Hi Ivan
    A great post! If I have the time I’ll try repeat your steps in my geoserver application, and then I will make a blog post.

    Thank you,

    a

  3. Marcel
    Posted 2009/02/02 at 1:16 pm | Permalink

    Hello Ivan,

    Great article. However the link to the windows version of osm2pgsql points to a version that does not recognize the -E argument.
    I found a version that recognizes it: http://tile.openstreetmap.org/direct/osm2pgsql.zip

    Cheers,

    Marcel

  4. Mike Pumphrey
    Posted 2009/02/02 at 5:36 pm | Permalink

    Thanks for the heads up, Marcel! I have changed the osm2pgsql link in the main text.

  5. Martin
    Posted 2009/02/20 at 5:01 pm | Permalink

    the osm2pgsql.zip isn’t there:

    The requested URL /direct/osm2pgsql.zip was not found on this server.

  6. Gbemisola Adefuye
    Posted 2009/02/26 at 12:21 pm | Permalink

    I really need your help. I’ve been learning Geoserver for a while now and i already know how to use it to an extent but i seem to be hanging. I don’t understand how to move further. I can’t create a complete webmap site yet. Please get back to me.

  7. amir
    Posted 2009/02/28 at 2:01 pm | Permalink

    please i can not use the file with extension .osm
    can you told me how to uncompress this file and if there is a program used to that send the link to my email .
    than you
    sorry my email (amir_1988_ghazaly@hotmail.com)

  8. facundo
    Posted 2009/03/03 at 9:02 pm | Permalink

    Now there is no need to do the convertion…

    here you can download the osm data in shapefile, is updated daily
    http://download.geofabrik.de/osm/

    hope this help.

  9. Erfan
    Posted 2009/03/11 at 3:23 pm | Permalink

    Very good, interesting, valuable and useful post.

  10. milosh
    Posted 2009/03/12 at 11:10 am | Permalink

    link to osm2pgsql utility is broken.
    Is there another link that can be used.

    Thanks

  11. Martin
    Posted 2009/03/25 at 4:57 am | Permalink

    How is progress on SLD’s for other tables than planet_osm_line ?

  12. Posted 2009/04/21 at 3:30 am | Permalink

    Nice tutorial, buth not tottaly complete. Please write somewhere, which table should be visualized? lines or roads? Probably result is the same, buth Iam not sure.

    After importing all like described here, I have an offset of open street map layer. About 25 kilometers. What to do now? :D Iam just na gis beginer. Please kick me to right way. What to check? (data in postgre postgis db is in 900913, geoserver feature is reading automaticaly this srs, srid, or what it is. Projection to quantum gis is done in 4326 (gps, wgs84) and it doesnt fit any other wms data. Offset is realy big)

  13. Dicky
    Posted 2009/08/11 at 10:19 pm | Permalink

    http://tile.openstreetmap.org/osm2pgsql.zip file is invalid or corrupted. I can’t unzip this file after downloading it.

  14. Doug
    Posted 2009/09/09 at 9:44 am | Permalink

    I’ve run into a snag with this, and I was wondering if you’ve noticed anything similar. The issue is that, on my map, regardless of what I put in the SLD, any ways that are marked with highway=residential are not displayed unless they also have admin_level=*. That is, GeoServer will not render any residential streets unless they are also administrative boundaries. Is this something you’ve noticed?

  15. Doug
    Posted 2009/09/09 at 11:28 am | Permalink

    Found that my earlier issue was caused by using the planet_osm_roads table instead of the planet_osm_line table. It’s worth noting that osm2pgsql only adds major roads (highway=tertiary or higher) to the planet_osm_roads table. Lower classified roads are included only if the way also constitutes part of something else (such as an administrative boundary).

  16. Jovie
    Posted 2009/09/24 at 3:41 pm | Permalink

    When I try and add the sld to geoserver, i get the following when I validate the xml:

    org.xml.sax.SAXParseException: cvc-complex-type.2.4.a: Invalid content was found starting with element ‘Name’. One of ‘{“http://www.opengis.net/sld”:Symbolizer}’ is expected.

    anyone know what this means? Thanks

  17. Roman
    Posted 2009/10/08 at 5:05 am | Permalink

    Hi, Ur sld is really cool, How can I be informed if you have improvements on the sld for open street map.

  18. Wai Chung Hon
    Posted 2009/10/13 at 3:29 pm | Permalink

    Hi, I’m fairly new to GeoServer and I’m really stuck on how to import multiple tables from one database. Can you please provide some information or a URL? Thanks in advance.

  19. Donna Mills
    Posted 2009/10/20 at 11:11 am | Permalink

    Thanks for the SLD Ivan. After a bit of playing around I was able to successfully get it working.

    I have downloaded a shapefile of the OSM data from the Geofabrik website as suggested by Facundo above. I then imported this shapefile into a PostGIS database.

    Initially when I loaded the SLD into Geoserver I received the same parsing error as Jovie above i.e.

    org.xml.sax.SAXParseException: cvc-complex-type.2.4.a: Invalid content was found starting with element ‘Name’. One of ‘{”http://www.opengis.net/sld”:Symbolizer}’ is expected.

    The issue was the location of the lines containing the and tags which I moved to after the tag in each instance. The lines beginning <VendorOption… also had to be moved to just before the tag in each instance.

    I removed the section from the SLD relating to runways as these are not present in my OSM dataset and replaced all occurances of ‘highway’ with ‘type’ to reference the correct field name in the OSM roads database table that I have in PostGIS.

    The SLD then successfully parsed and I was able to assign it to my OSM roads feature type in Geoserver. The SLD now symbolises my roads dataset according to the different zoom levels.

    I would make my version of the SLD available but there is no attachment facility. Hopefully I have explained my amendments well enough for others to follow.

  20. John J. Mitchell
    Posted 2009/10/20 at 1:34 pm | Permalink

    Hi,

    I downloaded the iwillig-openstreetmap-sld sld’s and I don’t know what feature to assign to the various styles and what order to layer the features.

    First I assigned the osm_roads.sld and osm_roads1.sld styles from the default directory to the planet_osm_roads feature.

    Then I replaced the osm_roads.sld and osm_roads1.sld styles with the global_roads.sld (Note: within this sld I had to do I a global replace of type with highway) from the bright directory and assigned to the planet_osm_roads feature.

    The global_roads.sld style is definitely a brighter color compared to the default directory styles.

    Since the bright directory styles look better than what is in the default directory what styles will these 5 bright styles replace within the default directory and what style will be assigned to what feature (planet_osm_line, planet_osm_point, planet_osm_polygon, planet_osm_roads)?

    Summary: In addition to all the Bright styles what additional Default styles do I use and for each style what feature do I assign them.

    Thanks,

    John

  21. Roshan Shrestha
    Posted 2009/11/17 at 9:51 am | Permalink

    How do I get the look and feel that you are getting in the “finished product”: “http://demo.opengeo.org/openstreetmap” ? Is the SLD you have provided enough, or are there other files that I need?

    Thanks
    Roshan

  22. Roshan Shrestha
    Posted 2009/11/17 at 9:53 am | Permalink

    Googling for OpenStreetMap SLD, I found this site:

    http://github.com/iwillig/openstreetmap-sld

    I will try it later – hope this will work

  23. Ben
    Posted 2010/01/27 at 11:00 am | Permalink

    it realy looks good!
    How did you realised the search funktion? Is this based on the WFS served from Geoserver as well?

    great work!

  24. Posted 2010/04/15 at 12:09 pm | Permalink

    Great job Ivan! II wonder if we can access to the other SLD (for polygon, line and point tables).
    Thank you very much!

  25. Posted 2010/04/15 at 4:30 pm | Permalink

    Ivan Great Work!
    Donna Mills, can you explain more detailed what you said about the expect?

    org.xml.sax.SAXParseException: cvc-complex-type.2.4.a: Invalid content was found starting with element ‘Name’. One of ‘{”http://www.opengis.net/sld”:Symbolizer}’ is expected.

    The issue was the location of the lines containing the and tags which I moved to after the tag in each instance. The lines beginning <VendorOption… also had to be moved to just before the tag in each instance.

    I removed the section from the SLD relating to runways as these are not present in my OSM dataset and replaced all occurances of ‘highway’ with ‘type’ to reference the correct field name in the OSM roads database table that I have in PostGIS.

    Thank!

  26. Posted 2010/06/30 at 4:33 am | Permalink

    Hi Ivan,

    Thanks for the useful article, and gorgeous cartography!

    As Marcel says, osm2pgsql seems to have difficulty with the -E option, only it’s not limited to the Windows version. I’ve encountered a buffer overflow in the latest version on Ubuntu 10.4, and the when compiling latest copy from svn (think they’re both 0.66)

    Removing the -E option from:
    osm2psql -E 900913 -d osm planet.osm

    to:
    osm2pgsql -d osm planet.osm

    seems to resolve the issue. There’s a missing ‘g’ in the osm2psql as well ;-)

    Cheers,
    Leigh.

  27. Matthew
    Posted 2010/06/30 at 11:09 am | Permalink

    Yes, can someone please explain how to fix “org.xml.sax.SAXParseException: cvc-complex-type.2.4.a: Invalid content was found starting with element ‘Name’. One of ‘{”http://www.opengis.net/sld”:Symbolizer}’ is expected.” in more detail. I don’t understand Donna’s explanation. Move what where? What tags?
    Please Help!
    Thanks,
    Matt

  28. Thomasbr
    Posted 2010/07/21 at 8:22 am | Permalink

    to validate the sld file featured, you should move <maxscale… and <minscale… tags after <ogc:filter and all <vendor tags just before the end of the <Text… tag

  29. Posted 2010/07/27 at 2:25 am | Permalink

    Regarding the following comment:

    “As Marcel says, osm2pgsql seems to have difficulty with the -E option, only it’s not limited to the Windows version. I’ve encountered a buffer overflow in the latest version on Ubuntu 10.4, and the when compiling latest copy from svn (think they’re both 0.66)”

    It seems to also be a problem on SLES11, so probably also in Redhat5 then too.

Download GeoServer