| Mission |
|
Evaluation of pervasive and ubiquitous systems has been the focus of many workshops in the recent years1. Analysing and comparing solutions proposed by researchers in this field is paramount for the assessment of the research results in this area. To this end, we promote an international contest called EvAAL: Evaluating AAL Systems Through Competitive Benchmarking, aimed at raising interest of the research and developer communities in the multidisciplinary research fields enabling Ambient Assisted Living2(AAL), and at creating benchmarks for the evaluation and comparison of AAL systems. We are now forming the EvAAL Contest, and invite you to join in our efforts. The EvAAL competition is developed as a project under the AALOA association EvAAL (Evaluating AAL Systems Through Competitive Benchmarking) is an international contest aimed at raising interest in the research and developer communities in the multidisciplinary research fields enabling Ambient Assisted Living (AAL), and at creating benchmarks for the evaluation and comparison of AAL systems. The main objective of the competitions organized by EvAAL is to enable the comparison of different AAL solutions, by establishing suitable benchmarks and evaluation metrics that will be progressively refined and improved in the years. In particular, EvAAL will focus not only on comparison of algorithms, but also of cost, deployment effort, time and costs, etc. EvAAL’s objective relies on offering to researchers an arena where to try, test and experiment not only AAL solutions but also benchmarks and evaluation methods. To this purpose, EvAAL will be open to all issues related to the test environment (living laboratories vs. into the wild), to the benchmarking (automatic vs. based on users’ evaluations), to the tools supporting the competition etc. The outcome would be a toolkit of techniques from which system builders can draw. Making these techniques open, available, and easy to use will enable comparative evaluation between similar components across systems and, in the end, of whole AAL systems. The EvAAL competition aims at contributing to the AAL disciplines in the same way as other competitions have contributed to their respective areas. Under this respect EvAAL tracts inspirations from successful competitions such as the Trading Agent Competition3 (TAC) and DARPA Grand Challenge4. We invite everyone to participate in the activities of EvAAL, to propose new competition ideas, and to contribute actively to the organization of the competition.
1 K. Connelly, K.A. Siek, I. Mulder, S. Neely, G. Stevenson, C. Kray “Evaluating Pervasive and Ubiquitous Systems” , 2008 |
