Dear Juan Pablo,<div><br></div><div>We would like to clarify that due to the randomly duplicated samples, we have checked our log (we have provided it to you) and it does not match with the input file you have provided to us (i.e. our log file does not have duplicated samples). I hope you can find where the error is and solve it. Please let us know if you need some help.</div>
<div><br></div><div>Thank you for your efforts!</div><div><br>_________________________<br><b>Dante I. Tapia</b><br><font color="#666666"><b>Grupo de Investigación BISITE / <i>BISITE Research Group</i><br></b>Universidad de Salamanca (España) / <i>University of Salamanca (Spain)</i><br>
</font><a href="http://bisite.usal.es" target="_blank">http://bisite.usal.es</a><br><div style="display:inline"></div><div style="display:inline"></div><div style="display:inline"></div><div style="display:inline"></div>
<br>
<br><br><div class="gmail_quote">2011/9/22 Juan Pablo Lázaro Ramos <span dir="ltr"><<a href="mailto:jplazaro@tsbtecnologias.es">jplazaro@tsbtecnologias.es</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div lang="ES" link="blue" vlink="purple"><div><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;color:#1F497D">Dear Dante and Stefan and other members of Competition,<u></u><u></u></span></p><p class="MsoNormal">
<span lang="EN-US" style="font-size:11.0pt;color:#1F497D"><u></u> <u></u></span></p><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;color:#1F497D">We have received Dante’s and Stefan’s concerns, identified them, and we are very near (a couple of hours) to provide you with an “official” answer to both of them.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;color:#1F497D">Both concerns are not connected. <u></u><u></u></span></p><p><u></u><span lang="EN-US" style="font-size:11.0pt;color:#1F497D"><span>-<span style="font:7.0pt "Times New Roman""> </span></span></span><u></u><span lang="EN-US" style="font-size:11.0pt;color:#1F497D">For the case of Stefan it is related to synchronization aspects.<u></u><u></u></span></p>
<p><u></u><span lang="EN-US" style="font-size:11.0pt;color:#1F497D"><span>-<span style="font:7.0pt "Times New Roman""> </span></span></span><u></u><span lang="EN-US" style="font-size:11.0pt;color:#1F497D">For the case of Dante it is related to a random pattern of series of two identical lines in the input file.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;color:#1F497D"><u></u> <u></u></span></p><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;color:#1F497D">We wanted to publish everything and applying the same rules for everybody and considering the samples “as is”, “as they arrive to us”. So then, when you check the published algorithms (java project) and samples we verify together that everything is ok.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;color:#1F497D"><u></u> <u></u></span></p><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;color:#1F497D">We would should insist in the fact that we want to be as fair as possible according to the rules and procedures we established. Any change to the way to proceed with one competitor, may affect the others, and should be analyzed very carefully.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;color:#1F497D"><u></u> <u></u></span></p><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;color:#1F497D">@Dante, @Stefan: Thanks for your effort in checking and reporting!<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;color:#1F497D"><u></u> <u></u></span></p><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;color:#1F497D">Juan Pablo<u></u><u></u></span></p><p class="MsoNormal">
<span lang="EN-US" style="font-size:11.0pt;color:#1F497D"><u></u> <u></u></span></p><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;color:#1F497D"><u></u> <u></u></span></p><div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:10.0pt">De:</span></b><span style="font-size:10.0pt"> <a href="mailto:contest-bounces@evaal.aaloa.org" target="_blank">contest-bounces@evaal.aaloa.org</a> [mailto:<a href="mailto:contest-bounces@evaal.aaloa.org" target="_blank">contest-bounces@evaal.aaloa.org</a>] <b>En nombre de </b>Dante Tapia<br>
<b>Enviado el:</b> jueves, 22 de septiembre de 2011 12:44<br><b>Para:</b> Stefan Knauth<br><b>CC:</b> Stefano Chessa; <a href="mailto:contest@evaal.aaloa.org" target="_blank">contest@evaal.aaloa.org</a><br><b>Asunto:</b> Re: [evaal contest] Evaluation / Time synchronisation<u></u><u></u></span></p>
</div><div><div></div><div class="h5"><p class="MsoNormal"><u></u> <u></u></p><p class="MsoNormal">Dear EvAAL organizers,<u></u><u></u></p><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">We have also found important inconsistencies on the results provided. The most important problems we have found are related to randomly duplicated samples in your input file for each test which lead to truncated test samples on our side.<u></u><u></u></p>
</div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">For this reason, we are recalculating the results and we will provide you the report asap.<u></u><u></u></p></div><div><p class="MsoNormal">
<u></u> <u></u></p></div><div><p class="MsoNormal">Best regards,<u></u><u></u></p></div><div><p class="MsoNormal"><br>_________________________<br><b>Dante I. Tapia</b><br><b><span style="color:#666666">Grupo de Investigación BISITE / <i>BISITE Research Group</i><br>
</span></b><span style="color:#666666">Universidad de Salamanca (España) / <i>University of Salamanca (Spain)</i><br></span><a href="http://bisite.usal.es" target="_blank">http://bisite.usal.es</a><u></u><u></u></p><p class="MsoNormal" style="margin-bottom:12.0pt">
<br><br><u></u><u></u></p><div><p class="MsoNormal">On Thu, Sep 22, 2011 at 11:45 AM, Stefan Knauth <<a href="mailto:stefan.knauth@hft-stuttgart.de" target="_blank">stefan.knauth@hft-stuttgart.de</a>> wrote:<u></u><u></u></p>
<p class="MsoNormal">Dear EvAAL competition evaluators,<br>Dear Competitors<br><br>(via Contest mailing list)<br><br>Yesterday we (the iLoc team) received the data used for evaluation, together with the evaluation code.<br>
<br>When running the evaluation using the timestamps given in the files provided by the email, obtained scores were more or less zero for all<br>tests.<br><br>The reason is a time mismatch between our system and the evaal system of 50 seconds.<br>
This can be easily seen for example in the AOI data, where the iLoc reported AOIs appear 50 secs later than the expected AOIs.<br><br>We request to take into account a time shift of 50 seconds,<br>with respect to the data received in the mail from Sept 21, i.e. to rerun the evaluation and add 50 seconds (i.e. 50000 msec) to the "Start timestamp" and "Stop timestamp" values.<br>
<br>I don't know how the obtained time mismatch was induced.<br><br>The system is installed for 2 years now at the iHomeLab and runs there with a typical delay of less than 0.5 seconds. Also during the setup in Valencia, we tested the system and the AOI detection, and a delay of 50 seconds would obviously have been realized by us.<br>
<br>Best regards,<br><br>Stefan Knauth / iLoc Team<br><br>-----------------------------------<br>Prof. Dr. Stefan Knauth<br>Stuttgart University of Applied Sciences - HFT Stuttgart<br>Faculty of Geomatics, Computer Science and Mathematics<br>
Schellingstr. 24<br>D-70174 Stuttgart<br>Germany<br><br>mail : <a href="mailto:stefan.knauth@hft-stuttgart.de" target="_blank">stefan.knauth@hft-stuttgart.de</a><br>phone : <a href="tel:%2B49%20711%208926%202966" target="_blank">+49 711 8926 2966</a><br>
cell : <a href="tel:%2B49%20179%202932%20009" target="_blank">+49 179 2932 009</a><br><br><br>Am 21.09.11 15:15, schrieb Stefano Chessa:<br>...<br>><br>><br>> I send you the logfiles and the software used to perform the evaluation.<br>
> With these data you can, if you wish, evaluate your own score regarding<br>> availability and accuracy.<br>><br>><br>><br>> I apologize for sending this material so late, our initial plan was deliver<br>
> all of this material much earlier than the EvAAL workshop, so that each<br>> competitor would have had time to check his own score. However many issues<br>> (included even software bugs) delayed us too much.<br>
><br>><br>><br>> Please note that each competitor is, in this phase, granted access to its<br>> own data, but we will put all the logs of everybody on the EvAAL website<br>> soon after the workshop and the prize awarding.<br>
><br>><br>><br>> If you test the program and you find bugs or other issue I ask you to let us<br>> know as soon as possible.<br>><br>...<br><br><br><br><br>_______________________________________________<br>
Contest mailing list<br><a href="mailto:Contest@evaal.aaloa.org" target="_blank">Contest@evaal.aaloa.org</a><br><a href="http://evaal.aaloa.org/mailman/listinfo/contest" target="_blank">http://evaal.aaloa.org/mailman/listinfo/contest</a><u></u><u></u></p>
</div><p class="MsoNormal"><u></u> <u></u></p></div></div></div></div></div></blockquote></div><br></div>