With the development of geospatial services, Web GIS (Geospatial Information System) has progressed towards a service-oriented paradigm. Nowadays, geospatial services are widely used to support effectively the most common needs of geospatial information users, such as discovery, access, process, and visualization of geospatial datasets. The most commonly used services are Catalogue Service for Web (CSW), Web Feature Service (WFS), Web Processing Service (WPS), and Web Mapping Service (WMS). While expected and desirable, use of geospatial resources in mission critical applications and business processes will raise pertinent quality concerns – users are likely to be concerned about the performance, reliability and security of geospatial services in addition to quality of geospatial data delivered by such services. In order to achieve and sustain user-satisfaction in the marketplace, there is a need to continuously guarantee acceptable levels of quality of geospatial resources for users.
In geospatial domain, there are two main testing activities and test-beds for geospatial resources usually focus on compliance and interoperability testing. Test-beds for compliance checking are mainly dedicated to industry requirements and provide testing tools and formalized procedures to determine to which degree a candidate product or implementation fulfills the specifications of a standard or directive. The OGC CITE (Open Geospatial Consortium Compliance and Interoperability Testing Initiative) (http://cite.opengeospatial.org/) and the GDI-DE (Spatial Data Infrastructure Germany) test suite (http://testsuite.gdi-de.org/gdi/) initiatives are two compliance testing platforms which allow compliance checking of services and applications implementing OGC standards and INSPIRE (Infrastructure for Spatial Information in Europe) specifications.
However, existing testing activities and test-beds for geospatial resources have some limitations. They are in initial steps of development and available for some but not all geospatial standards such as geoprocessing services (e.g., OGC WPS). Moreover, they have just focused on compliance and interoperability testing and do not support to test functional and non-functional properties of geospatial services. For example, they are using the following test criteria for evaluating WMS which has two mandatory operations:
- GetCapabilities operation: for various combinations of valid mandatory and optional request parameters validate that the WMS returns a valid XML (Extensible Markup Language) file, checked against the WMS GetCapabilities response schema.
- GetMap operation: for various combinations of valid mandatory and optional request parameters validate that the WMS returns a valid map file.
Several parameters such as VERSION, REQUEST, LAYERS, STYLES, SRS (Spatial Reference System), BBOX (Bounding Box), FORMAT, WIDTH, HEIGHT, and EXCEPTIONS are evaluated during the compliance and interoperability test process. However, it is necessary to add some test criteria to make the test more applicable.
Several solutions can be applied in order to address such limitations. The first solution is to develop existing, open source geospatial testing platforms to support more functional and non-functional test cases. Another solution is to utilize open source testing tools such as soapUI (http://www.soapui.org/) and loadUI (http://www.loadui.org/) to implement a new, comprehensive testing platform. Integrating common geospatial test-beds (e.g., OGC CITE) with open source testing tools (e.g., soapUI) can be considered as a third solution. Moreover, to make geospatial resources testing more comprehensive, there is a need to implement new test criteria for geospatial services and data models.
Recently, ESDIN (European Spatial Data Infrastructure Network) consortium (http://www.esdin.eu/) has developed a comprehensive test environment to address such limitations. The ESDIN test environment has been developed based on soapUI open source and provides a systematic approach for testing INSPIRE Network Services namely View Service, Download Service, and Coordinate Transformation Service as well as INSPIRE datasets. For example, the following criteria have been considered in ESDIN test environment for evaluating WMS (aka View Service in INSPIRE terminology):
- Availability Test: during a period of one week continuously sending valid GetCapabilities requests. The frequency of the requests is 1 request per hour. The percentage of requests returned properly should not be less than 99%.
- Performance Test: during a period of 6 hours, valid GetMap requests are continuously sent. The frequency of the requests is 10 requests per second. The mandatory request parameters are selected randomly, except for the image size which is set to 800 x 600 pixels. The number of responses faster than 5 seconds should be greater than 90%.
- Performance Scaling Test: criterion 2 is repeated but for image size of 400 x 300 pixels, 1200 x 900 pixels and 1600 x 1200 pixels. The number of responses faster than 5 seconds should be greater than 90%.
- Capacity Test: criterion 2 is repeated but the expected frequency is 20 requests per second. The number of responses faster than 5 seconds should be greater than 90%.
- Capacity Scaling Test: criterion 2 is repeated but the expected frequency is set to 50 and 100 requests per second. The number of responses faster than 5 seconds should be greater than 90%.
By using such approach, a customized testing platform can be developed which not only supports compliance and interoperability testing but also any arbitrary functional and non-functional tests.
In conclusion, although geospatial services make it easy for users to access desired geospatial information, the quality of geospatial resources will greatly affect the willingness of users in access of such services. Therefore, in order to improve the use of geospatial SOA for distributed geospatial data sharing, proper testing of such geospatial resources is highly valuable.