This document refines the evaluation methods for the competition that were introduced in the technical annex of the call .

1.    Organization and Test Procedures

A few notes about the organization of the competition:

• The organizers give an appointment to each competitor so that each competitor may stay at the living lab for the time necessary for the evaluation of his/her system.
• Only one competitor at a time is admitted in the living lab (no other competitors will be admitted during the evaluation of the other competitors).
• We will give the opportunity to each competitor to repeat the test one time and to opt for the best result between the first try and the second.
• The organizers will not provide any tools for bricolage, hence competitors should bring all the tools they need. If necessary, there are several mall near the living lab where the competitors may buy some tools. In any case, if competitors need to hang devices on the walls and they plan to use glue, nails or any other method that may damage the living lab they are requested to contact the organizers as soon as possible.

Installation of activity recognition systems at the living lab:

•  Each competitor will be given up to 2.5 hours to: install the system, execute the tests, uninstall everything. Specifically, we plan to give at most 1 hour for the installation. If any competitor feels that he/she cannot meet this requirement, he/she is invited to communicate this as soon as possible to the organizers.
• After the installation the organizers will make a preliminary test to ensure that the connection between the competing system and the evaluation tools is correctly established. The time required to solve any issue related to this does NOT count for the Installation Complexity criteria.
• The porch will be part of the activity recognition area of the LivingLab.

In all the tests the competing systems will recognize the activities of an actor (a person of the organization trained in moving along pre-defined paths).

2. Integration with the benchmarking system

Competitors are kindly requested to perform a simple SW integration with our benchmarking system.

Our server will have a static IP and will use Sockets to receive the information from competitors. The format used to send the information through sockets will be a string with the NTP time data (miliseconds from 1970) and the activity separated by a coma:

1338974503713,4

1338974550858,1

…

The integer for each activity:

STAND = 0,

WALK = 1,

SIT = 2,

BEND = 3,

CYCLE = 4,

FALL = 5,

LIEDONW = 6,

NULLACTIVITY = 7;

We will release our evaluation software next week and we will prepare a server to perform some test with each team to avoid last minute problems.

3.    Time Synchronization

Since the evaluation tool will rely competitors’ machine for timestamping the localization packets, the two computers (the one with the REAL position/timestamps and the competitor's one) must be synchronized.

The competitors MUST use NTP to send their time (We used the server pool.ntp.org).