GeoServer Code Sprint 2016 Ready

Follow up to our GeoServer Code Sprint 2016 post – we are all ready to go for next week!

  • Monday: Initial migration, Wicket 7 bootcamp / Java 8 bootcamp
  • Tuesday: Migration/testing workspace and datastore configuration
  • Wednesday: Migration/testing global, service and layer configuration
  • Thursday: Migration/testing tile cache, security and demos
  • Friday: demos, extensions and select community modules

Mike Pumphrey will be on hand to update the User Guide reference section and ask awkward questions about how things work. We expect to corner a few new volunteers for the scheduled release of GeoServer 2.8.2.

See the OSGeo wiki page for details, sponsors, participants and schedule. For participants (local and remote) there is a pre-flight wiki page with background reading for the upgrade.

Thank you Sponsors

A big thanks to organizations sponsoring the sprint – you help make this gathering possible. Special thanks to the “In-Kind” sponsors sending staff to participate.

Gold Sponsors:

OSGeo    Boundless    Vivid Solutions

Silver Sponsors:

How 2 Map

Bronze Sponsors:

San Jose Water Company     Transient     Geobeyond

In-Kind Sponsors:

scitus development     GeoSolutions    CCRi

Astun Technology   Voyager

Welcome to Victoria

Thanks to Boundless Victoria for making their offices available (see prep below). We look forward to taking over next week and thanks to everyone for their hospitality.

IMG_1104

DSC01574

IMG_1106

GeoServer Explorer Plugin for QGIS

Victor Olaya has just announced the QGIS GeoServer Explorer plugin which uses the REST API to configure GeoServer.  This plugin serves as a QGIS-based tool to configure and manage GeoServer catalogs, acting as a GUI for GeoServer.

Browsing GeoServer Catalog

The GeoServer Explorer plugin also wraps up some QGIS functionality making it possible to perform operations that go beyond the capabilities of the REST API and easily perform more complex workflows:

  • For instance, uploading a layer in a format not supported by the REST API is done in the same way as uploading a simple shapefile. The plugin will take care of converting its format before uploading, without the user noticing it. Pre-upload hooks can be set up, to process all layers before they are published (for instance, running some topology check or a geometry simplification routine.)
  • Styling can be defined in QGIS using its symbology tools and will be converted to SLD when publishing a layer. This allows a layer to be published with the same style as it has when rendered as part of a QGIS project.

The GeoServer Explorer plugin integrates with other QGIS plugins, like the Processing framework. This allows easy automation of tasks such as publishing a set of layers or seeding a GWC layer, and these tasks can also be integrated in workflows using the QGIS Processing Graphical modeler.

QGIS GeoServer Explorer REST API Integration

QGIS GeoServer Explorer REST API Integration

QGIS GeoExplorer Plugin

The GeoServer Explorer plugin is available in the QGIS plugins server, and can now be downloaded from within QGIS using the plugin manager.

GeoServer Explorer Installation

GeoServer Explorer Installation

For developers the source code is available on github (https://github.com/boundlessgeo/qgis-geoserver-plugin), GPL with details on how to contribute.

We would like to thank Victor for his hard work, the community members who encouraged this release, and Boundless for making this plugin available for wider use. We look forward to seeing continued GeoServer and QGIS community collaboration.

GeoServer 2.7.5 released

The GeoServer team is happy to announce the release of GeoServer 2.7.5. Download bundles are provided (zip, war, dmg and exe) along with documentation and extensions.

GeoServer 2.7.5 is a maintenance release of GeoServer recommended for production deployment. Thanks to everyone taking part, submitting fixes including:

Bug
[GEOS-4179] – Importer and monitoring REST resources are not thread-safe
[GEOS-7116] – Applying a CRS in Importer clears other found CRSes
[GEOS-7306] – Stored Queries don’t work on App-Schema layers backed by database
[GEOS-7346] – WPS cancelling output stream ends up writing a single byte at a time

For a full list, see the release notes.

The next release, 2.7.6, will mark the planned end of life for this release series. You and your organisation should now consider looking into to the 2.8 release and start testing it now.

Thanks to Ian Turton (Astun Technology) for this release.

This release is made in conjunction with GeoTools 13.5.

GeoServer Code Sprint 2016

The GeoServer web administration console is built on top of the Wicket 1.4.x series, which is pretty old and unmaintaned. The purpose of the sprint is to update it to Wicket 7.x, the current stable version.

Upgrade from Wicket 1.3 to Wicket 7

Due to the large distance between the two releases and the number of backwards incompatible changes accumulated by Wicket in the years this will require the concerted effort of a handful of developers over a few days, including the changes to upgrade the code, and the thorough hand testing of the resulting modified interface.

Victoria British Columbia

The code sprint is planned for the week of January 18th in sunny Victoria British Columbia. Thanks to Boundless for providing a venue (either the boundless offices or Fort Techtoria depending on numbers).

A note on the timing: We were unable to join the Paris Code Sprint 2016 as it is scheduled too close to the GeoServer 2.9 code freeze. This location was selected to reduce travel costs allowing us to run the event with minimal sponsorship.

Participation and Sponsorship

We have the following sponsorship levels available:

  • Gold: $1000
  • Silver: $500
  • Bronze: $250

We are reaching out to international and local sponsors. Contributions will be put towards travel costs for overseas sprinters who would be otherwise unable to attend. Any surplus at the end of the event will be turned over to OSGeo or used for a future code sprints.

For more details on participation, sponsorship or budget for the event please see the GeoServer Code Sprint 2016 wiki page.

 

GeoServer 2.8.1 Released

The GeoServer team is pleased to announce the release of GeoServer 2.8.1. Download bundles are provided (binwardmg and exe) along with documentation and extensions.

GeoServer 2.8.1 is the latest stable release of GeoServer and is recommended for production deployment. This release is made in conjunction with GeoTools 14.1. Thanks to all contributors. Fixes and new functionality include:

  • Multidimensional GRIB / NetCDF / NetCDF Output modules promoted to extension
  • Fixed query parameters in SLD external graphic
  • Fixed legend preview with SLD external graphic
  • Fixed multiline labels in PDF WMS request with translation
  • Fixed layer preview GML links for app-schema layers
  • Fixed JMS clustering to use qualified names for layers, layer groups, and styles
  • Avoid catalog linear scans in GWC integration listeners
  • Fixed OpenLayers preview with the authkey module enabled
  • For a full list, see the release notes

Thanks to Ben Caradoc-Davies (Transient) for this release.

About GeoServer 2.8

Articles, blog posts and presentations:

 

GeoServer 2.7.4 released

The GeoServer team is happy to announce the release of GeoServer 2.7.4. Download bundles are provided (zipwardmg and exe)  along with documentation and extensions.

GeoServer 2.7.3 is a maintenance release of GeoServer recommended for production deployment. Thanks to everyone taking part, submitting fixes and new functionality including:

Bug

  • [GEOS-3228] – Empty filter causes IndexOutOfBoundsException
  • [GEOS-3432] – RESTConfig “styles” list does not get generated if a style is missing its associated sld file
  • [GEOS-4986] – Creating SQL Views via RESTConfig as JSON fails
  • [GEOS-6768] – externalGraphic with relative path and query parameters problem
  • [GEOS-7045] – Layer Security – Catalog Mode
  • [GEOS-7243] – Render (or transform) fails on Multipolygon but not on polygon
  • [GEOS-7256] – Maven Cobertura plugin does not work
  • [GEOS-7259] – JMS based cluster should use qualified names for Layers and Layergroups
  • [GEOS-7267] – JMS Clustering should prefix Styles names with workspace
  • [GEOS-7295] – OpenLayers preview does not work if authkey community module is enabled
  • [GEOS-7302] – Using on the fly meta tiling in WMS request may result in rendered images not being disposed of
  • [GEOS-7312] – RawDataPPIO does not close InputStreams it opens
  • [GEOS-7314] – GeoTiffPPIO can return the source file of a processed coverage

Improvement

  • [GEOS-4762] – WCS should force usage of imageread
  • [GEOS-7150] – Features counted twice for WFS queries with GeoJSON responses

For a full list, see the release notes.

Also, as a heads up for Oracle users, the Oracle store does not ship anymore with the JDBC driver (due to redistribution limitations imposed by Oracle). For details see the updated the oracle installation instructions here.

Thanks to Alessio Fabiani (GeoSolutions) for this release.

This release is made in conjunction with GeoTools 13.4 and GeoNode 2.4.

Remote Execution Vulnerability

GeoServer has encountered an remote execution vulnerability in the REST API (used for remote administration).

This vulnerability GEOS-7124 is addressed in the following scheduled releases:

Thanks to Andrea Aime (GeoSolutions) and Kevin Smith (Boundless) for both fixing this issue and back porting to the stable and maintenance series.

Users are encouraged to upgrade, keeping in mind exposure to this issue is limited to scripts using administrator credentials to access the REST API. Accounts making use of gsconfig (Python Library) also make use of these facilities.

About Remote Execution

For more information see redhat security article on remote code execution via serialized data.

Responsible Disclosure

Thanks to Matthias Kaiser for reporting this issue.

If you encounter a security vulnerability in GeoServer (or any other open source software) please take care to report the issue in a responsible fashion:

  • Keep exploit details out of issue report (send to developer/PSC privately – just like you would do for sensitive sample data)
  • Be prepared to work with Project Steering Committee (PSC) members on a solution
  • Keep in mind PSC members are volunteers and an extensive fix may require fundraising / resources

If you are not in position to communicate in public (or make use of the issue tracker) please consider commercial support, contacting a PSC member privately or contacting us via the Open Source Geospatial Foundation at info@osgeo.org.

GeoServer 2.7.3 released

The GeoServer team is happy to announce the release of GeoServer 2.7.3. Download bundles are provided (zipwardmg and exe)  along with documentation and extensions.

GeoServer 2.7.3 is a maintenance release of GeoServer recommended for production deployment. Thanks to everyone taking part, submitting fixes and new functionality including:

  • Further fixes for the XXE vulnerability, along with a fix for a remote code execution vulnerability in the REST API (requires admin credentials to trigger it)
  • Some WCS 1.1 and 2.0 fixes
  • Some improvements in the management of style specific workspaces when modifying layer groups with the REST API
  • Optimized the size of DBF in the SHAPE-ZIP output format
  • A few improvements in the importer, including speeding up import setup by delaying layer bounds computation, and allowing to harvest granules in an empty mosaic previously setup via the REST API
  • For a full list, see the release notes.

Also, as a heads up for Oracle users, the Oracle store does not ship anymore with the JDBC driver (due to redistribution limitations imposed by Oracle). For details see the updated the oracle installation instructions here.

Thanks to Andrea (GeoSolutions) and Kevin (Boundless) for this release.

 

GeoServer 2.8.0 released

We are happy to announce the release of GeoServer 2.8.0. Downloads are available (zipwardmg and exe) along with docs and extensions.

What is new

GeoServer 2.8.0 is the culmination of our latest six month development cycle and contains several new features, along with fixes and security updates.

This blog post provides a breakdown by functional area, for more detail see the 2.8.02.8-beta and 2.8-M0 release notes.

Data access and configuration

PostGIS curves and Oracle speedups

GeoServer 2.6.0, released one year ago, added read only support for Oracle curved geometries, along with all the necessary machinery to represent them in memory, draw them and encode them in WMS.

This release adds read and write support for curves in PostGIS instead, bringing our support for the famous open source spatial database to match and surpass (with write support) the Oracle one. PostGIS curves are supported in all OGC protocols, either via native support (e.g., GML) or on the fly linearization (e.g. shapefile output).

Sample curve data for PostGIS - Details

On the Oracle side, we improved the startup times for installation that are serving of Oracle layers by optimizing the table geometry type and metadata access.

Note: Due to license restrictions the oracle extension no longer includes an Oracle JDBC driver, see the user guide for manual install instructions.

Filtering layers during configuration

For all those that want to publish only a subset of the original data to the public we are now offering the configuration of a simple CQL filter that will be applied on data access, no matter what protocol is used. Think of it as a mini “sql view” that can be applied at ease against any data source, not just databases.

../../_images/data_layers_cql.png

Layer CQL Filter

This is of course not meant to limit feature access for security reasons, for that use case you should really look into GeoFence.

Raster NODATA with JAI-Ext Library

You may now optionally use the JAI-Ext image processing operations when working with raster data. These operations directly support raster NODATA and footprints (reducing the amount of processing required when working with these datasets).

JAI-Ext Operations

JAI-Ext Operations

This feature is available in GeoServer 2.8 but is off by default – to enable start up with:

-Dorg.geotools.coverage.jaiext.enabled=true

REST API for Image Moasic Granule Management

Structured coverages have recently been added to GeoServer, you can now use the REST API to manage and update individual granules in an image mosaic.

Process raster data during import

Vector import has supported limited data processing during import for some time. GeoServer 2.8.0 provides the same functionality (allowing raster files to be processed using GDAL command line tools during import).

To reproject a raster:

{
  "type": "GdalWarpTransform",

  "options": [
  "-t_srs", "EPSG:4326”
  ]
}

To transform the raster into a GeoTIFF:

{
  “type”: “GdalTranslateTransform”,
  “options”: [
    “-co”, “TILED=YES”,
    “-co”, “BLOCKXSIZE=512”,
    “-co”, “BLOCKYSIZE=512”
  ]
}

To introduce GeoTIFF overviews:

{
  "type": "GdalAddoTransform",
  "options": [ "-r", "average"],
  "levels" : [2, 4, 8, 16]
}

Mapping improvements

This release is full of big and small map rendering improvements for all. Here is an organized list.

Z ordering support

This new features extends SLD and CSS with vendor options allowing the style writer to control the painting order of features, either inside a single layer, or across layers: this allows proper map rendering of areas where a number of objects have below/above relationsthips, like this area in Germany, where a lot of roads and rails are crossing each other in a maze of underpasses, overpasses, and bridges:

../../../_images/roads-group.png

z-order example

This is achieved by specifying a “sortBy” vendor option at the FeatureTypeStyle level, with one or more sorting attributes, and in case multiple layers or FeatureTypeStyles are involved, by grouping them into a single “sortByGroup”. You can find more information, along with examples in CSS and SLD, in your user guide.

We would like to thank DLR for sponsoring this improvement.

Constrast enhancement improved

GeoServer has been supporting contrast enhancement for a while, within the limits of the SLD specification. Version 2.8.0 steps beyond the limits of the standard by adding vendor parameters to control the normalization sub-algorithm (stretch to min/max, clip to min/max, clip to zero), as well as its parameters. Here is an example of the syntax:

<ContrastEnhancement>
  <Normalize>
   <VendorOption name="algorithm">StretchToMinimumMaximum</VendorOption>
   <VendorOption name="minValue">50</VendorOption>
   <VendorOption name="maxValue">100</VendorOption>
  </Normalize>
</ContrastEnhancement>

along with a visual example, before and after the contrast enhancement:

contrast enhancement before

contrast enhancement after

contrast enhancement after

New arrow mark

Lots of map needs arrows… but every time is the same story, yes, the arrow is almost fine, but it should be longer, or thicker, or with a bigger head, and so on. Instead of having to re-invent a new arrow symbol each time, we created one whose proportions can be altered by changing parameters in its name.

Here is the general syntax of this new “well known mark”:

<WellKnownName>extshape://arrow?hr=[hrValue]&t=[tValue]&ab=[abValue]</WellKnownName>

and some examples varying its t (thickness) value between 0 and 1:

arrow1

or changing the witdh the height ratio (hr):

arrow2

So next time they ask you for a customized arrow, you can whip up your arrow mark, and give them something like this:

<WellKnownName>extshape://arrow?hr=4&amp;ab=0.8</WellKnownName>

arrow4

Multi-script maps made easier

GeoServer 2.8.0 improves its support for maps in multiple scripts, which can be a source of headaches. While it’s often easy to find support for most scripts in fonts, it’s hard to get one that would support, for example, western languages, arabic, corean, indi and simplified chinese in a single package. Especially for scripts like simplified chinese you have to resort to custom fonts.

Now, what happens if you are labelling a map that contains them all, and sometimes, contains more than one of them in a single label? Before GeoServer 2.8.0 we did not have a great answer to that, but now, you can simply specify multiple fonts in a TextSymbolizer, and the most suitable one will be chosen on the fly, eventually using multiple fonts in a single label in case there is no one able to handle the whole of it. Here is an example with mixed script labels:

Selezione_072

Multi-script Support

We would like to thank DLR for sponsoring this improvement.

Improved labeling density

Before GeoServer 2.8.0 labelling dense road networks with lots of diagonal and curved labels might have left the impression that more labels could have fit the map… and that was not just an impression! Indeed, the previous label algorithm was reserving  a busy area for the bounding box containing the label, which as you may see, is a lot more space than the actual label occupancy:

Letter reservation - conflict

The French National Institute for geographic information provided a patch that makes the single chars of diagonal or curved labels be reserved instead, resulting in maps with quite a bit more labelled items per square inch:

tiger-tiger_roads (1)

Improved Label Density

WMS/WMTS protocol and configuration improvements

Creating new styles from templates

It’s now possible to create new styles starting from the built-in templates, and the style will be encoded in the desired style language (SLD, or CSS, or even something else, if you created your own styling language extension point):

../../_images/data_style_editor_generate.png

Generate Default Style

 

GeoWebCache filter parameters GUI improved

It’s now possible to configure integer parameters in the caching section of a layer configuration.

Parameter Filter

Parameter Filter

GeoWebCache Storage

GeoWebCache can now store cached tiles on a perlayer basis – including Amazon S3.

 

tile storage

Request parameter support in Freemaker templates

Freemarker GetFeatureInfo templates can now access to the request parameter, as well as the Java process environment variables, in order to customize their response. For example, it’s now possible to expand the following variables in the template:

${request.LAYERS}
${request.ENV.PROPERTY}
${environment.GEOSERVER_DATA_DIR}
${environment.WEB_SITE_URL}

Controlling interpolation on a layer by layer basis

You can now control layer interpolation via GetMap, and specify a different interpolation policy on different layers. This is great if you are serving multiple raster maps, and maybe you want to have your classified raster use nearest neighbor, while showing the ozone density layer with bilinear interpolation.

Inspire configuration improved

Security

REST API for access control

Their is now a REST API for configuring security access control – see the user guide for details.

About GeoServer 2.8

Articles, blog posts and presentations:

For additional details see the 2.8.02.8-beta and 2.8-M0 release notes.

GeoServer Community

GeoServer Community modules provide an area for ideas and experimentation:

  • WCS and WPS output formats based on gdal_translate to provide a greater range of output formats
  • Gabriel has created a community module for vector tiles experimentation
  • Embedded GeoFence server, REST API and GUI is the result of a productive collaboration between GeoSolutions and Boundless offering greater rule-based control of GeoServer security
  • MongoDB DataStore enabling GeoServer to publish from this popular JSON based document database (no zip packaging, needs volunteer)

Community modules should be considered a work-in-progress and are subject to quality assurance, documentation IP checks and a maintainer before being considered ready for release.

New repository and release delay

A quick message for all those who have been asking – I have started the GeoServer 2.8 release process but have run into a snag. The repo.boundlessgeo.com maven repository has been slowed down due to increased network traffic. We are setting up a replacement (cloud hosted which will allow allow more developers to manage).

What I would like to ask is for developers to try it out by adding the following to maven settings.xml:

  <mirrors>
    <mirror>
      <id>boundlessgeo</id>
      <name>Boundless Cloud Repository</name>
      <url>https://boundless.artifactoryonline.com/boundless/main</url>
      <mirrorOf>boundless</mirrorOf>
    </mirror>
  </mirrors>

Please try the above and report back to geoserver-devel, and we can cut over to the new repository tomorrow.

Thank you for your assistance, and please accept our apologies for the delay in releasing 2.8.0.

Update: We have now migrated to the new repository.

Maven Developers

For developers using maven to depend on geoserver jars (for those running a custom geoserver build) please note that we have now migrated to the new repository.

The repository details have not changed:

  <repository>
   <id>boundless</id>
   <name>Boundless Maven Repository</name>
   <url>http://repo.boundlessgeo.com/main/</url>
  </repository>

The exception is for projects (such as GeoWebCache, GeoFence, GeoScript) that deploy artifacts. We ask you to change your distributionManagement section to the following:

<distributionManagement>
  <repository>
   <id>boundless</id>
   <name>Boundless Release Repository</name>
   <url>https://boundless.artifactoryonline.com/boundless/release/</url>
   <uniqueVersion>false</uniqueVersion>
  </repository>
  <snapshotRepository>
   <id>boundless</id>
   <uniqueVersion>false</uniqueVersion>
   <name>Boundless Snapshot Repository</name>
   <url>https://boundless.artifactoryonline.com/boundless/snapshot/</url>
  </snapshotRepository>
 </distributionManagement>

Contact geoserver-devel if you have any questions.

Download GeoServer