[evaal contest] Evaluation / Time synchronisation

[Dante Tapia]

Dear Juan Pablo,

We would like to clarify that due to the randomly duplicated samples, we
have checked our log (we have provided it to you) and it does not match with
the input file you have provided to us (i.e. our log file does not have
duplicated samples). I hope you can find where the error is and solve it.
Please let us know if you need some help.

Thank you for your efforts!

_________________________
*Dante I. Tapia*
*Grupo de Investigación BISITE / BISITE Research Group
*Universidad de Salamanca (España) / *University of Salamanca (Spain)*
http://bisite.usal.es



2011/9/22 Juan Pablo Lázaro Ramos <jplazaro at tsbtecnologias.es>

> Dear Dante and Stefan and other members of Competition,****
>
> ** **
>
> We have received Dante’s and Stefan’s concerns, identified them, and we are
> very near (a couple of hours) to provide you with an “official” answer to
> both of them.****
>
> Both concerns are not connected. ****
>
> **-          **For the case of Stefan it is related to synchronization
> aspects.****
>
> **-          **For the case of Dante it is related to a random pattern of
> series of two identical lines in the input file.****
>
> ** **
>
> We wanted to publish everything and applying the same rules for everybody
> and considering the samples “as is”, “as they arrive to us”. So then, when
> you check the published algorithms (java project) and samples we verify
> together that everything is ok.****
>
> ** **
>
> We would should insist in the fact that we want to be as fair as possible
> according to the rules and procedures we established. Any change to the way
> to proceed with one competitor, may affect the others, and should be
> analyzed very carefully.****
>
> ** **
>
> @Dante, @Stefan: Thanks for your effort in checking and reporting!****
>
> ** **
>
> Juan Pablo****
>
> ** **
>
> ** **
>
> *De:* contest-bounces at evaal.aaloa.org [mailto:
> contest-bounces at evaal.aaloa.org] *En nombre de *Dante Tapia
> *Enviado el:* jueves, 22 de septiembre de 2011 12:44
> *Para:* Stefan Knauth
> *CC:* Stefano Chessa; contest at evaal.aaloa.org
> *Asunto:* Re: [evaal contest] Evaluation / Time synchronisation****
>
> ** **
>
> Dear EvAAL organizers,****
>
> ** **
>
> We have also found important inconsistencies on the results provided. The
> most important problems we have found are related to randomly duplicated
> samples in your input file for each test which lead to truncated test
> samples on our side.****
>
> ** **
>
> For this reason, we are recalculating the results and we will provide you
> the report asap.****
>
> ** **
>
> Best regards,****
>
>
> _________________________
> *Dante I. Tapia*
> *Grupo de Investigación BISITE / BISITE Research Group
> *Universidad de Salamanca (España) / *University of Salamanca (Spain)*
> http://bisite.usal.es****
>
>
>
> ****
>
> On Thu, Sep 22, 2011 at 11:45 AM, Stefan Knauth <
> stefan.knauth at hft-stuttgart.de> wrote:****
>
> Dear EvAAL competition evaluators,
> Dear Competitors
>
> (via Contest mailing list)
>
> Yesterday we (the iLoc team) received the data used for evaluation,
> together with the evaluation code.
>
> When running the evaluation using the timestamps given in the files
> provided by the email, obtained scores were more or less zero for all
> tests.
>
> The reason is a time mismatch between our system and the evaal system of 50
> seconds.
> This can be easily seen for example in the AOI data, where the iLoc
> reported AOIs appear 50 secs later than the expected AOIs.
>
> We request to take into account a time shift of 50 seconds,
> with respect to the data received in the mail from Sept 21, i.e. to rerun
> the evaluation and add 50 seconds (i.e. 50000 msec) to the "Start timestamp"
> and "Stop timestamp" values.
>
> I don't know how the obtained time mismatch was induced.
>
> The system is installed for 2 years now at the iHomeLab and runs there with
> a typical delay of less than 0.5 seconds. Also during the setup in Valencia,
> we tested the system and the AOI detection, and a delay of 50 seconds would
> obviously have been realized by us.
>
> Best regards,
>
> Stefan Knauth / iLoc Team
>
> -----------------------------------
> Prof. Dr. Stefan Knauth
> Stuttgart University of Applied Sciences - HFT Stuttgart
> Faculty of Geomatics, Computer Science and Mathematics
> Schellingstr. 24
> D-70174 Stuttgart
> Germany
>
> mail  : stefan.knauth at hft-stuttgart.de
> phone : +49 711 8926 2966
> cell  : +49 179 2932 009
>
>
> Am 21.09.11 15:15, schrieb Stefano Chessa:
> ...
> >
> >
> > I send you the logfiles and the software used to perform the evaluation.
> > With these data you can, if you wish, evaluate your own score regarding
> > availability and accuracy.
> >
> >
> >
> > I apologize for sending this material so late, our initial plan was
> deliver
> > all of this material much earlier than the EvAAL workshop, so that each
> > competitor would have had time to check his own score. However many
> issues
> > (included even software bugs) delayed us too much.
> >
> >
> >
> > Please note that each competitor is, in this phase, granted access to its
> > own data, but we will put all the logs of everybody on the EvAAL website
> > soon after the workshop and the prize awarding.
> >
> >
> >
> > If you test the program and you find bugs or other issue I ask you to let
> us
> > know as soon as possible.
> >
> ...
>
>
>
>
> _______________________________________________
> Contest mailing list
> Contest at evaal.aaloa.org
> http://evaal.aaloa.org/mailman/listinfo/contest****
>
> ** **
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://evaal.aaloa.org/pipermail/contest/attachments/20110922/d6212427/attachment-0001.html>