Open data for global multimodal land use classification
The Contest: Goals and Organization
The 2017 IEEE GRSS Data Fusion Contest, organized by the Image Analysis and Data Fusion Technical Committee, aims at promoting progress on fusion and analysis methodologies for multisource remote sensing data.
The 2017 Data Fusion Contest will consist in a classification benchmark. The task to perform is classification of land use (more precisely, Local Climate Zones, LCZ, Stewart and Oke, 2012) in various urban environments. Several cities have been selected to test the ability of LCZ prediction at generalizing all over the world. Input data are multi-temporal, multi-source and multi-modal (image and semantic layers).
Local climate zones are a generic, climate-based typology of urban and natural landscapes, which delivers information on basic physical properties of an area that can be used by land use planners or climate modelers [Bechtel et al., 2015]. LCZ are used as first order discretization of urban areas by the World Urban Database and Access Portal Tools initiative (WUDAPT, http://www.wudapt.org), which aims to collect, store and disseminate data on the form and function of cities around the world.
The LCZ classes in this study correspond to those of [Stewart & Oke, 2012]:
- 10 urban LCZs corresponding to various built types:
- Compact high-rise (class code in the ground truth: 1);
- Compact midrise (class code in the ground truth: 2);
- Compact low-rise (class code in the ground truth: 3);
- Open high-rise (class code in the ground truth: 4);
- Open midrise (class code in the ground truth: 5);
- Open low-rise (class code in the ground truth: 6);
- Lightweight low-rise (class code in the ground truth: 7);
- Large low-rise (class code in the ground truth: 8);
- Sparsely built (class code in the ground truth: 9);
- Heavy industry (class code in the ground truth: 10).
- 7 rural LCZs corresponding to various land cover types:
- Dense trees (class code in the ground truth: 11);
- Scattered trees (class code in the ground truth: 12);
- Bush and scrub (class code in the ground truth: 13);
- Low plants (class code in the ground truth: 14);
- Bare rock or paved (class code in the ground truth: 15);
- Bare soil or sand (class code in the ground truth: 16);
- Water (class code in the ground truth: 17).
An example for the city of Bologna (Italy) can be seen below:
The contest aims to promote innovation in classification algorithms, as well as to provide objective and fair comparisons among methods. Ranking is based on quantitative accuracy parameters computed with respect to undisclosed test samples from cities unseen during training. Participants will be given a limited time to submit their classification maps after the competition is started. The contest will consist of two steps:
Step 1 – training: Participants are provided with five training cities (Berlin, Rome, Paris, Sao Paulo, Hong Kong), including ground truth to train their algorithms.
Step 2 – testing on new cities: Participants will receive the data of the test cities and will submit their classification maps by three weeks from the release of this second part of the data set. In parallel, they will submit a short description of the approach used. After evaluation of the results, 4 winners will be announced.
Calendar:
January 9th Contest opening: release of training cities (Step 1) March 13 Release of test cities (Step 2): evaluation server is open. April 1st Submission of classification maps deadline: the submission server is closed April 5 Winners announcement
References:
[Stewart & Oke, 2012] I.D. Stewart and T.R. Oke, Local climate zones for urban temperature studies, Bulletin of the American Meteorological Society, 93(12):1879-1900, December 2012.
[Bechtel et al., 2015] B. Bechtel et al., “Mapping Local Climate Zones for a Worldwide Database of the Form and Function of Cities,” ISPRS International Journal of Geo-Information, vol. 4, no. 1, pp. 199–219, Feb. 2015.
The Data:
The dataset comprises several city sites. For each city, we provide:
Landsat data, in the form of images with 8 multispectral bands (i.e. visible, short and long infrared wavelengths) resampled at 100m resolution (courtesy of the U.S. Geological Survey);
Sentinel2 images, with 9 multispectral bands (i.e. visible, vegetation red edges and short infrared wavelengths) resampled at 100m resolution (Contains modified Copernicus Data 2016); participants are encouraged to use the full resolution data, for which a direct link is provided in the data package.
Ancillary data: Open Street Map (OSM) layers with land use information: building, natural, roads and land-use areas (Data © OpenStreetMap contributors, available under the Open Database Licence – http://www.openstreetmap.org/copyright). We also provide rasterized versions of OSM layers at 20m resolution for building and land-use areas, superimposable with the satellite images.
Moreover, for the training cities only, we also provide ground-truth of the various LCZ classes on several areas of the city (defined as polygons using the class codes above). They are provided as raster layers at 100m resolution, superimposable to the satellite images. The ground-truth for the test set will remain undisclosed and will be used for evaluation of the results.
Results, Awards, and Prizes:
The authors of the 4 best ranking classification maps will:
Be awarded IEEE Certificates of Recognition. The award ceremony will take place during the Technical Committees and Chapter Chairs Dinner at IGARSS 2017, Forth Worth, TX, in July 2017.
Will be invited to prepare a 4-page manuscript after the closing of the Contest. Upon acceptance of these manuscript by the Organizing Committee of the Contest, they will also be invited present them in an oral Invited Session dedicated to the Contest at IGARSS 2017 and to publish them in the Proceedings of IGARSS 2017.
Be invited to contribute to WUDAPT and support the implementation of their methods as open tools.
The first and second ranking teams will co-author a journal paper (in a limit of 3 co-authors per team), which will summarize the outcome of the Contest and will be submitted to IEEE JSTARS. To maximize impact and promote the potential of current multisource remote sensing technologies, the open-access option will be used for this journal paper. The method proposed for the Contest will be published in this paper only.
The authors of the winning classification map (1st ranking) will receive as a prize an NVIDA GPU graphic card 
The costs for open-access publication, for the winners’ participations to the Technical Committees and Chapter Chairs Dinner at IGARSS 2017 will be supported by the GRSS.
The rules of the game:
Data can be requested by registering for the Contest on the IEEE GRSS DASE website: http://dase.grss-ieee.org/. Participants must read and accept the Contest Terms and Conditions.
Participants of the contest are intended to submit classification maps (in raster format, similar to the .tif file included in the training data) for all test cities. These results will be submitted to the IEEE GRSS DASE website for evaluation:
http://dase.grss-ieee.org/. Ranking between the participants will be based on the scores of the confusion matrix and accuracy parameters (Overall Accuracy, etc.).
Deadline for classification maps submission is April, 1st, 2017, 23:59 UTC – 12 hours (e.g., April, 2nd, 2017, 7:59 in New York City, 13:59 in Paris, or 19:59 in Beijing). Submission server will be opened from March 13, 2017.
Each classification map submission will be authored by one or more co-authors (team of participants). One and only one submission originating from each team will be allowed to the Contest. Should multiple entries from the same team be received, then exclusively the first-ranked submission received will be considered. A 48-hour delay between 2 consecutive submissions is imposed and will be checked carefully.
While submitting a classification result, each team will acknowledge that, should the result be among the 4 best ranking ones, at least one team member will participate to the Data Fusion invited session at IGARSS 2017.
Failure to follow any of these rules will automatically make the submission invalid, resulting in the submitted map not being evaluated.
Participants to the Contest are not requested to submit an extended abstract to IGARSS 2017 by the corresponding conference deadline in January 2017. Only contest winners (participants corresponding to the 4 best-ranking results) will submit a 4-page paper describing their approach to the Contest by May 1st, 2017. The received manuscripts will be reviewed by the Organizing Committee of the Contest (D. Tuia, G. Moser, B. Le Saux, B. Bechtel, L. See). Then winners will submit the final version of their 4 full-papers to the IGARSS Data Fusion Contest Invited Session by May 26, 2017, for inclusion in the IGARSS Technical Program and Proceedings.
Any scientific publication using the data shall refer to the following paper: N. Yokoya, P. Ghamisi, J. Xia, S. Sukhanov, R. Heremans, I. Tankoyeu, B. Bechtel, B. Le Saux, G. Moser, D. Tuia, “Open Data for Global Multimodal Land Use Classification: Outcome of the 2017 IEEE GRSS Data Fusion Contest,” in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 11, no. 5, pp. 1363-1377, May 2018. http://doi.org/10.1109/JSTARS.
Acknowledgements
Landsat 8 data available from the U.S. Geological Survey (https://www.usgs.gov/).
OpenStreetMap Data © OpenStreetMap contributors, available under the Open Database Licence – http://www.openstreetmap.org/copyright.
Original Copernicus Sentinel Data 2016 available from the European Space Agency (https://sentinel.esa.int).
The Contest is being organized in collaboration with the WUDAPT (http://www.wudapt.org/) and GeoWIKI (http://geo-wiki.org/) initiatives. The IADF TC chairs would like to thank the organizers and the IEEE GRSS for continuously supporting the annual Data Fusion Contest through funding and resources.