[evaal contest] Question about competition

Fri May 10 17:24:43 CEST 2019

Dear Joaquin Torres,

Thank you very much for this decision.

Best Regards

Jarek (Jerry) Karciarz

samsungLogo2016

Senior Professional

Head of Network Analytics

Samsung R&D Institute Poland 

Samsung Electronics 

Office:  +48 22 377 8784

Cell:     +48 600 098 433 

j.karciarz at samsung.com 

From: Joaquín Torres Sospedra [mailto:jtorres at uji.es] 
Sent: Friday, May 10, 2019 4:55 PM
To: Jarek (Jerry) Karciarz
Cc: Francesco Potortì; contest at evaal.aaloa.org; Yuqin Wang
Subject: Re: [evaal contest] Question about competition

Dear Jarek Karciarz,

After some deliberation we have decided to publish the ground truth data for the previous competitions.

Next week we will publicly release this data in the EvAAL website as well as other repositories.

Best regards,

-- 
Joaquín Torres-Sospedra
Postdoc @ GEOTEC <http://geotec.uji.es/> 
Universitat Jaume I, Castellón, Spain
Social media: @jtorres_phd <https://twitter.com/jtorres_phd>  - LinkedIn <https://www.researchgate.net/profile/Joaquin_Torres-Sospedra>  - ResearchGate <https://www.researchgate.net/profile/Joaquin_Torres-Sospedra> 
Web: http://jtorr.es

El 10 may 2019, a las 8:52, Jarek (Jerry) Karciarz <j.karciarz at samsung.com> escribió:

Dear Joaquin,

Thank you so much for explanations and considering the release of ground truth evaluation data.

Best Regards

Jarek (Jerry) Karciarz

<image001.png>

Senior Professional

Head of Network Analytics

Samsung R&D Institute Poland

Samsung Electronics 

Office:  +48 22 377 8784

Cell:     +48 600 098 433 

 <mailto:j.karciarz at samsung.com> j.karciarz at samsung.com

From: Joaquín Torres Sospedra [ <mailto:jtorres at uji.es> mailto:jtorres at uji.es] 
Sent: Thursday, May 09, 2019 11:08 PM
To: Jarek (Jerry) Karciarz
Cc: Francesco Potortì;  <mailto:contest at evaal.aaloa.org> contest at evaal.aaloa.org; Yuqin Wang
Subject: Re: [evaal contest] Question about competition

Dear Jarek Karciarz,

Sorry for my late reply, I have been out of the office the last two days. 

The datasets collected for the Track 3 (Smartphone-based off-site) can be reached from the EvAAL website (except the ones of the IPIN 2018 conference). Although we are working to create a common repository, you can find all datasets in the links below:

 <http://indoorloc.uji.es/ipin2016track3/> http://indoorloc.uji.es/ipin2016track3/
 <http://indoorloc.uji.es/ipin2017track3/> http://indoorloc.uji.es/ipin2017track3/
 <http://ipin-conference.org/2018/ipincompetition> http://ipin-conference.org/2018/ipincompetition
 <http://indoorloc.uji.es/ipin2019track3/> http://indoorloc.uji.es/ipin2019track3/ 

We are organizing this track in the current format since 2016. In 2016, we provided training and evaluation data. Since 2017, we provide training, validation and evaluation data. The validation data sets contain the location of some landmarks and they can be used to evaluate your model based built on the training data sets.

It is true that we provided blind evaluation data for the competition and the ground truth is not available. We are now discussing about this matter and the decision is not easy to take. We will answer your request about releasing the Ground Truth of the evaluation sets (2016, 2017 and  2018) tomorrow.

In the meanwhile, if you have any question about track 3 or the datasets format, please let me know. 

-- 
Joaquín Torres-Sospedra
Postdoc @  <http://geotec.uji.es/> GEOTEC
Universitat Jaume I, Castellón, Spain
Social media:  <https://twitter.com/jtorres_phd> @jtorres_phd -  <https://www.researchgate.net/profile/Joaquin_Torres-Sospedra> LinkedIn -  <https://www.researchgate.net/profile/Joaquin_Torres-Sospedra> ResearchGate
Web:  <http://jtorr.es/> http://jtorr.es

El 9 may 2019, a las 11:01, Jarek (Jerry) Karciarz < <mailto:j.karciarz at samsung.com> j.karciarz at samsung.com> escribió:

Dear Joaquin, Francesco,

Thanks for your response. Sharing data from 2018 competition would be highly appreciated.
However our question is a little bit wider. The data shared from 2017 and before editions are so called training and evaluation data. Training data contain Wi-Fi and other sensor readings plus true locations. That’s fine. The evaluation data don’t contain locations because the location should be estimated by the competing algorithm. That’s obvious. Our question was, could you please share the true locations for the evaluation data for 2018 and before editions?. This would help us to real test the performance of our algorithm on data from past competition editions and compare to other participants, especially to the winners.
Thank you in advance for understanding.

Best Regards

Jarek (Jerry) Karciarz

Senior Professional
Head of Network Analytics
Samsung R&D Institute Poland 
Samsung Electronics 
Office:  +48 22 377 8784
Cell:     +48 600 098 433 
 <mailto:j.karciarz at samsung.com> j.karciarz at samsung.com 

-----Original Message-----
From: Francesco Potortì [ <mailto:Potorti at isti.cnr.it> mailto:Potorti at isti.cnr.it] 
Sent: Wednesday, May 08, 2019 6:14 PM
To: Jarek (Jerry) Karciarz
Cc:  <mailto:contest at evaal.aaloa.org> contest at evaal.aaloa.org; 'Yuqin Wang'
Subject: Re: [evaal contest] Question about competition

We have the question regarding off-line, smartphone based track. Is it 
possible you to share ground true locations for previous contests 
editions? Say for years 2015-2018?

This would greatly let us challenge our solution against corresponding 
winners.

Jarek,

on  <http://evaal.aaloa.org/> http://evaal.aaloa.org you find data for past editions until 2017.
We are still working on publishing data for 2018, sorry about that.

Since you are specifically interested in smartphone off-site, you may want to ask for details to the Track 3 chair Joaquín Torres Sospedra < <mailto:jtorres at uji.es> jtorres at uji.es>.

Sincerely

-- 
IPIN'19  <http://ipin2019.isti.cnr.it/> http://ipin2019.isti.cnr.it    Voice:  +39.050.621.3058
Francesco Potortì (ricercatore)        Mobile: +39.348.8283.107
ISTI - Area della ricerca CNR          Skype:  wnlabisti
via G. Moruzzi 1, I-56124 Pisa         Web:     <http://fly.isti.cnr.it/> http://fly.isti.cnr.it

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://evaal.aaloa.org/pipermail/contest/attachments/20190510/6cd1a2e8/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 13168 bytes
Desc: not available
URL: <http://evaal.aaloa.org/pipermail/contest/attachments/20190510/6cd1a2e8/attachment-0001.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 5564 bytes
Desc: not available
URL: <http://evaal.aaloa.org/pipermail/contest/attachments/20190510/6cd1a2e8/attachment-0001.png>