<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Dear Jarek Karciarz,<div class=""><br class=""></div><div class="">After some deliberation we have decided to publish the ground truth data for the previous competitions.</div><div class="">Next week we will publicly release this data in the EvAAL website as well as other repositories.</div><div class=""><br class=""></div><div class="">Best regards,<br class=""><div class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div dir="auto" style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">-- <br class=""><b class="">Joaquín Torres-Sospedra</b><br class="">Postdoc @ <a href="http://geotec.uji.es/" class="">GEOTEC</a><br class="">Universitat Jaume I, Castellón, Spain<br class="">Social media: <a href="https://twitter.com/jtorres_phd" class="">@jtorres_phd</a> - <a href="https://www.researchgate.net/profile/Joaquin_Torres-Sospedra" class="">LinkedIn</a> - <a href="https://www.researchgate.net/profile/Joaquin_Torres-Sospedra" class="">ResearchGate</a><br class="">Web:<span class="Apple-converted-space"> </span><a href="http://jtorr.es" class="">http://jtorr.es</a></div></div></div></div></div></div></div>
</div>
<div><br class=""><blockquote type="cite" class=""><div class="">El 10 may 2019, a las 8:52, Jarek (Jerry) Karciarz <<a href="mailto:j.karciarz@samsung.com" class="">j.karciarz@samsung.com</a>> escribió:</div><br class="Apple-interchange-newline"><div class=""><div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Dear Joaquin,<o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""> </o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Thank you so much for explanations and considering the release of ground truth evaluation data.<o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""> </o:p></span></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="PL" style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Best Regards<o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="PL" style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""> </o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="PL" style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Jarek (Jerry) Karciarz<o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="PL" style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""> </o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><span id="cid:image001.png@01D5070D.BD6B41F0"><image001.png></span></span><span lang="PL" style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-family: Arial, sans-serif; color: rgb(125, 158, 192); background-color: white; background-position: initial initial; background-repeat: initial initial;" class=""><o:p class=""> </o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-family: Arial, sans-serif; color: rgb(125, 158, 192); background-color: white; background-position: initial initial; background-repeat: initial initial;" class="">Senior Professional</span><span style="color: rgb(31, 73, 125);" class=""><o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-family: Arial, sans-serif; color: rgb(125, 158, 192); background-color: white; background-position: initial initial; background-repeat: initial initial;" class="">Head of<span class="Apple-converted-space"> </span><b class="">Network Analytics</b></span><span style="color: rgb(31, 73, 125);" class=""><o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><b class=""><span style="font-family: Arial, sans-serif; color: rgb(125, 158, 192);" class="">Samsung R&D Institute Poland</span></b><span style="font-family: Arial, sans-serif; color: rgb(125, 158, 192);" class=""></span><span style="color: rgb(31, 73, 125);" class=""><o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-family: Arial, sans-serif; color: rgb(125, 158, 192);" class="">Samsung Electronics<span class="Apple-converted-space"> </span></span><span style="color: rgb(31, 73, 125);" class=""><o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-family: Arial, sans-serif; color: rgb(125, 158, 192);" class="">Office: +48 22 377 8784</span><span style="color: rgb(31, 73, 125);" class=""><o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-family: Arial, sans-serif; color: rgb(125, 158, 192);" class="">Cell: +48 600 098 433<span class="Apple-converted-space"> </span></span><span style="color: rgb(31, 73, 125);" class=""><o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-family: Arial, sans-serif; color: rgb(117, 120, 123);" class=""><a href="mailto:j.karciarz@samsung.com" style="color: purple; text-decoration: underline;" class="">j.karciarz@samsung.com</a><o:p class=""></o:p></span></div></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""> </o:p></span></div><div class=""><div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(181, 196, 223); padding: 3pt 0cm 0cm;" class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><b class=""><span style="font-size: 10pt; font-family: Tahoma, sans-serif;" class="">From:</span></b><span style="font-size: 10pt; font-family: Tahoma, sans-serif;" class=""><span class="Apple-converted-space"> </span>Joaquín Torres Sospedra [<a href="mailto:jtorres@uji.es" style="color: purple; text-decoration: underline;" class="">mailto:jtorres@uji.es</a>]<span class="Apple-converted-space"> </span><br class=""><b class="">Sent:</b><span class="Apple-converted-space"> </span>Thursday, May 09, 2019 11:08 PM<br class=""><b class="">To:</b><span class="Apple-converted-space"> </span>Jarek (Jerry) Karciarz<br class=""><b class="">Cc:</b><span class="Apple-converted-space"> </span>Francesco Potortì;<span class="Apple-converted-space"> </span><a href="mailto:contest@evaal.aaloa.org" style="color: purple; text-decoration: underline;" class="">contest@evaal.aaloa.org</a>; Yuqin Wang<br class=""><b class="">Subject:</b><span class="Apple-converted-space"> </span>Re: [evaal contest] Question about competition<o:p class=""></o:p></span></div></div></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class="">Dear Jarek Karciarz,<o:p class=""></o:p></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class="">Sorry for my late reply, I have been out of the office the last two days. <o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class="">The datasets collected for the Track 3 (Smartphone-based off-site) can be reached from the EvAAL website (except the ones of the IPIN 2018 conference). Although we are working to create a common repository, you can find all datasets in the links below:<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><a href="http://indoorloc.uji.es/ipin2016track3/" style="color: purple; text-decoration: underline;" class="">http://indoorloc.uji.es/ipin2016track3/</a><br class=""><a href="http://indoorloc.uji.es/ipin2017track3/" style="color: purple; text-decoration: underline;" class="">http://indoorloc.uji.es/ipin2017track3/</a><br class=""><a href="http://ipin-conference.org/2018/ipincompetition" style="color: purple; text-decoration: underline;" class="">http://ipin-conference.org/2018/ipincompetition</a><br class=""><a href="http://indoorloc.uji.es/ipin2019track3/" style="color: purple; text-decoration: underline;" class="">http://indoorloc.uji.es/ipin2019track3/</a> <o:p class=""></o:p></div></div><div class=""><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class="">We are organizing this track in the current format since 2016. In 2016, we provided training and evaluation data. Since 2017, we provide training, validation and evaluation data. The validation data sets contain the location of some landmarks and they can be used to evaluate your model based built on the training data sets.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class="">It is true that we provided blind evaluation data for the competition and the ground truth is not available. We are now discussing about this matter and the decision is not easy to take. We will answer your request about releasing the Ground Truth of the evaluation sets (2016, 2017 and 2018) tomorrow.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class="">In the meanwhile, if you have any question about track 3 or the datasets format, please let me know. <o:p class=""></o:p></div></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div class=""><div class=""><div class=""><div class=""><div class=""><div class=""><div class=""><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class="">-- <br class=""><b class="">Joaquín Torres-Sospedra</b><br class="">Postdoc @ <a href="http://geotec.uji.es/" style="color: purple; text-decoration: underline;" class="">GEOTEC</a><br class="">Universitat Jaume I, Castellón, Spain<br class="">Social media: <a href="https://twitter.com/jtorres_phd" style="color: purple; text-decoration: underline;" class="">@jtorres_phd</a> - <a href="https://www.researchgate.net/profile/Joaquin_Torres-Sospedra" style="color: purple; text-decoration: underline;" class="">LinkedIn</a> - <a href="https://www.researchgate.net/profile/Joaquin_Torres-Sospedra" style="color: purple; text-decoration: underline;" class="">ResearchGate</a><br class="">Web:<span class="apple-converted-space"> </span><a href="http://jtorr.es/" style="color: purple; text-decoration: underline;" class="">http://jtorr.es</a><o:p class=""></o:p></span></div></div></div></div></div></div></div></div></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><br class=""><br class=""><o:p class=""></o:p></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class="">El 9 may 2019, a las 11:01, Jarek (Jerry) Karciarz <<a href="mailto:j.karciarz@samsung.com" style="color: purple; text-decoration: underline;" class="">j.karciarz@samsung.com</a>> escribió:<o:p class=""></o:p></div></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div><div class=""><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class="">Dear Joaquin, Francesco,<br class=""><br class="">Thanks for your response. Sharing data from 2018 competition would be highly appreciated.<br class="">However our question is a little bit wider. The data shared from 2017 and before editions are so called training and evaluation data. Training data contain Wi-Fi and other sensor readings plus true locations. That’s fine. The evaluation data don’t contain locations because the location should be estimated by the competing algorithm. That’s obvious. Our question was, could you please share the true locations for the evaluation data for 2018 and before editions?. This would help us to real test the performance of our algorithm on data from past competition editions and compare to other participants, especially to the winners.<br class="">Thank you in advance for understanding.<br class=""><br class="">Best Regards<br class=""><br class="">Jarek (Jerry) Karciarz<br class=""><br class=""><br class=""><br class="">Senior Professional<br class="">Head of Network Analytics<br class="">Samsung R&D Institute Poland<span class="Apple-converted-space"> </span><br class="">Samsung Electronics<span class="Apple-converted-space"> </span><br class="">Office: +48 22 377 8784<br class="">Cell: +48 600 098 433<span class="Apple-converted-space"> </span><br class=""><a href="mailto:j.karciarz@samsung.com" style="color: purple; text-decoration: underline;" class="">j.karciarz@samsung.com</a><span class="Apple-converted-space"> </span><br class=""><br class="">-----Original Message-----<br class="">From: Francesco Potortì [<a href="mailto:Potorti@isti.cnr.it" style="color: purple; text-decoration: underline;" class="">mailto:Potorti@isti.cnr.it</a>]<span class="Apple-converted-space"> </span><br class="">Sent: Wednesday, May 08, 2019 6:14 PM<br class="">To: Jarek (Jerry) Karciarz<br class="">Cc:<span class="Apple-converted-space"> </span><a href="mailto:contest@evaal.aaloa.org" style="color: purple; text-decoration: underline;" class="">contest@evaal.aaloa.org</a>; 'Yuqin Wang'<br class="">Subject: Re: [evaal contest] Question about competition<br class=""><br class=""><br class=""><o:p class=""></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class="">We have the question regarding off-line, smartphone based track. Is it<span class="Apple-converted-space"> </span><br class="">possible you to share ground true locations for previous contests<span class="Apple-converted-space"> </span><br class="">editions? Say for years 2015-2018?<br class=""><br class="">This would greatly let us challenge our solution against corresponding<span class="Apple-converted-space"> </span><br class="">winners.<o:p class=""></o:p></div><p class="MsoNormal" style="margin: 0cm 0cm 12pt; font-size: 12pt; font-family: "Times New Roman", serif;"><br class="">Jarek,<br class=""><br class="">on<span class="Apple-converted-space"> </span><a href="http://evaal.aaloa.org/" style="color: purple; text-decoration: underline;" class="">http://evaal.aaloa.org</a><span class="Apple-converted-space"> </span>you find data for past editions until 2017.<br class="">We are still working on publishing data for 2018, sorry about that.<br class=""><br class="">Since you are specifically interested in smartphone off-site, you may want to ask for details to the Track 3 chair Joaquín Torres Sospedra <<a href="mailto:jtorres@uji.es" style="color: purple; text-decoration: underline;" class="">jtorres@uji.es</a>>.<br class=""><br class="">Sincerely<br class=""><br class="">--<span class="Apple-converted-space"> </span><br class="">IPIN'19<span class="Apple-converted-space"> </span><a href="http://ipin2019.isti.cnr.it/" style="color: purple; text-decoration: underline;" class="">http://ipin2019.isti.cnr.it</a><span class="Apple-converted-space"> </span> Voice: +39.050.621.3058<br class="">Francesco Potortì (ricercatore) Mobile: +39.348.8283.107<br class="">ISTI - Area della ricerca CNR Skype: wnlabisti<br class="">via G. Moruzzi 1, I-56124 Pisa Web: <a href="http://fly.isti.cnr.it/" style="color: purple; text-decoration: underline;" class="">http://fly.isti.cnr.it</a><br class=""><br class=""><o:p class=""></o:p></p></div></div></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div></div></div></div><table id="bannersignimg" data-cui-lock="true" namo_lock="" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; letter-spacing: normal; orphans: auto; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><tbody class=""><tr class=""><td class=""><div class=""> <br class="webkit-block-placeholder"></div></td></tr></tbody></table><table id="confidentialsignimg" data-cui-lock="true" namo_lock="" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; letter-spacing: normal; orphans: auto; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><tbody class=""><tr class=""><td class=""><div class=""> <img unselectable="on" data-cui-image="true" src="cid:cafe_image_0@s-core.co.kr" style="border: 0px solid currentcolor; border-image-source: none; width: 520px; height: 144px; display: inline-block;" class=""></div></td></tr></tbody></table></div></blockquote></div><br class=""></div></body></html>