Data Organisation
The data organization step manages the aspect of the adoption of field observation data, lab data, analyzed data and metadata into a central system. It prevents data loss, assesses the quality and harmonization of the data and preparation of the data for reuse. Data organization follows the soil archiving step in the development of the SIS and when the smaller definition of a SIS as a digital infrastructure for soil data is used, can be the first step after the user needs assessment.
Digital soil data, other than reports, maps, and documents, are stored in two main digital representations : attribute tables and raster formats.
The attribute table, if it includes a geometry, also referred to as vector data, typically contains the results of a soil data collection campaign or subsequent lab analysis. Raster format is a result of earth observation (aerial imagery) or a modelling effort. For every pixel on a grid, a numeric value(s) for the given variable is observed or predicted. The tools and procedures to work with these two types of data are quite distinct.
Raster data are often stored as flat files, two common formats being Geo Tagged Image File Format (GeoTIFF) and Network Common Data Form (NetCDF). These files are organized into catalogues (a folder structure). Rasters can also be stored in a database. This scenario is relevant if you use the raster analyses functions of the database to filter on or calculate cell values. Raster files can result in large files if they extend to a large area or are detailed. Large files led to long loading times. Various scenarios are common to optimize performance. Most common is adoption of an image pyramid, in which the data is subsampled at various resolutions. The client then reads data from the most relevant resolution only. Attribute table data can be stored as spreadsheets, or in databases. Storing in databases has two main advantages. 1) Data are accessible for reading and writing by multiple users, and 2) the integrity and security of the data are better maintained:
- Integrity: databases provide options to set rules to maintain integrity. For example, a parent record cannot be removed if there is a related child record.
- Security: databases enable fine-grained configuration of access on table level, for Read, Update and Delete privileges.
In either format, metadata of data is typically registered as a file close to or embedded in the data file, or in a registry, catalogue and/or Document Management System.
This workflow step introduces various standards and tools available to facilitate data organization.
Statistical learning refers to a vast set of tools for understanding data. These tools can be classified as supervised or unsupervised. Broadly speaking, supervised statistical learning involves building a statistical model for pre- dicting, or …
ANZSoilML is an information model that specifies the set of classes, properties, relationships and supporting vocabularies needed to structure digital information used in the Australian and New Zealand soil sciences. It is primarily concerned with …
This document was prepared in the framework of ISRIC’s Community of Practice for soil information providers. It outlines basic principles for compiling a profile dataset for consideration in WoSIS (World Soil Information Service). Templates for data …
The impact of soil science to the society has probably never been as high as it is currently. The importance of soils in the Earth system must make us, the soil scientists, more noticeable at the EGU. The aim of the Division on Soil System Sciences …
There is an urgent need to improve the infrastructure supporting the reuse of scholarly data. A diverse set of stakeholders—representing academia, industry, funding agencies, and scholarly publishers—have come together to design and jointly endorse a …
WoSIS™ stands for ‘World Soil Information Service’, a large database based on PostgreSQL + API’s, workflows, dashboards etc., developed and maintained by ISRIC, WDC-Soils. It provides a growing range of quality-assessed and …
The standardized SOil and TERrain SOTER methodology has been proposed by the International Union of Soil Sciences (IUSS) as a method to make soils and terrain information available to a wide spectrum of land users. The 1995 version of the Procedures …
The Global Soil Partnership (GSP) is a globally recognized mechanism established in 2012 with the mission to position soils in the Global Agenda and to promote sustainable soil management. The Partnership, hosted by FAO, works hard to improve soil …
Get an overview of the INSPIRE Directive, its founding principles and the maintenance and implementation framework. Find guidance on how to implement and how to benefit from its reuse in different contexts. For more information, visit the link below. …
The International Union of Soil Sciences (IUSS) is the global union of soil scientists. The objectives of the IUSS are to promote all branches of soil science, and to support all soil scientists across the world in the pursuit of their activities.
World Reference Base for Soil Resources (WRB) is an international soil classification system for naming soils and creating legends for soil maps The mission of the Working Group WRB is to test and further develop the WRB classification. The Working …
This engineering report describes the results of the Soil Data Interoperability Experiment (the IE) conducted under the auspices of the OGC Agriculture Domain Working Group in 2015. Soil data exchange and analysis is compromised by the lack of a …
This document was prepared in the framework of ISRIC’s Community of Practice for soil information providers. It provides a compilation of resources required to compile a Soil and Terrain (SOTER) database at (sub)national scale using existing …
This Soil Data Assimiliation Guidance wiki is a collaborative effort to collect and describe hands-on good practices on data assimilation in the soil domain, with a focus on Europe. The term Soil data assimilation has been chosen by the autors as an …
This page attempts to catalogue all freely-downloadable primary soils information usable in a GIS either as ‘points’, lines, polygons or grids (‘rasters’). This compilation has been maintained first at University of Twente/ITC …
A journal article through which the Findable, Accessible, Interoperable and Re-usable (FAIR) principles for data publishing were launched.
The World Soil Information Service (WoSIS) aims to serve the user with a selection of standardised and ultimately harmonised soil profile data. The quality-assessed data may be used to underpin digital soil mapping and a range of global assessments. …