An Evaluation Platform for Semantic Web Technology

The vision of the Semantic Web aims at enhancing today's Web in order to provide a more efficient and reliable environment for both providers and consumers of Web resources (i.e. information and services). To deploy the Semantic Web, various technologies have been developed, such as machine understandable description languages, language parsers, goal matchers, and resource composition algorithms. Since the Semantic Web is just emerging, each technology tends to make assumptions about different aspects of the Semantic Web's architecture and use, such as the kind of applications that will be deployed, the resource descriptions, the consumers' and providers' requirements, and the existence and capabilities of other technologies. In order to ensure the deployment of a robust and useful Semantic Web and the applications that will rely on it, several aspects of the technologies must be investigated, such as whether the assumptions made are reasonable, whether the existing technologies allow construction of a usable Semantic Web, and the systematic identification of which technology to use when designing new applications. In this thesis we provide a means of investigating these aspects for service discovery, which is a critical task in the context of the Semantic Web. We propose a simulation and evaluation platform for evaluating current and future Semantic Web technology with different resource sets and consumer and provider requirements. For this purpose we provide a model to represent the Semantic Web, a model of the evaluation platform, an implementation of the evaluation platform as a multi-agent system, and an illustrative use of the platform to evaluate some service discovery technology in a travel scenario. The implementation of the platform shows the feasibility of our evaluation approach. We show how the platform provides a controlled setting to support the systematic identification of bottlenecks and other challenges for new Semantic Web applications. Finally, the evaluation shows that the platform can be used to assess technology with respect to both hardware issues such as the kind and number of computers involved in a discovery scenario, and other issues such as the evaluation of the quality of the service discovery result