如何評價FAIR的最新工作Data Distillation？

論文傳送門： https://arxiv.org/pdf/1712.04440.pdf

Data Distillation: Towards Omni-Supervised Learning

We investigate omni-supervised learning, a special regime of semi-supervised learning in which the learner exploits all available labeled data plus internet-scale sources of unlabeled data. Omni-supervised learning is lower bounded by performance on existing labeled datasets, offering the potential to surpass state-of-the-art fully supervised methods. To exploit the omni-supervised setting, we propose data distillation, a method that ensembles predictions from multiple transformations of unlabeled data, using a single model, to automatically generate new training annotations. We argue that visual recognition models have recently become accurate enough that it is now possible to apply classic ideas about self-training to challenging real world data. Our experimental results show that in the cases of human keypoint detection and general object detection, state-of-the-art models trained with data distillation surpass the performance of using labeled data from the COCO dataset alone.

(修改了一下問題方式，請各位見諒，來自知乎小透明)

上面那位，讀對paper再來噴

這篇文章想說的是distillation，大聲說出來這個詞什麼意思

怎麼hinton在知乎就有文章出來就是厲害的名譽呢…

kaiming文章被提一個問題就要噴…

然後11月掛完文章，12月就掛很奇怪么。。

看起來應該就是cvpr的投稿吧。。

2. arXiv:1711.10370 [pdf, other]

Learning to Segment Every Thing

Ronghang Hu, Piotr Dollár, Kaiming He, Trevor Darrell, Ross Girshick

Subjects: Computer Vision and Pattern Recognition (cs.CV)

3. arXiv:1711.07971 [pdf, other]

Non-local Neural Networks

Xiaolong Wang, Ross Girshick, Abhinav Gupta, Kaiming He

Comments: tech report

Subjects: Computer Vision and Pattern Recognition (cs.CV)

你咋不說人家一個禮拜就出來learning to segment every thing...

一個人手裡有幾個項目一起做很奇怪么？

也一直在想怎樣去給domain的data貼標籤做。。

最後貼兩個最近一直在思考的related work吧。。

https://arxiv.org/pdf/1710.09412

ftp://ftp.math.ucla.edu/pub/camreport/cam17-66.pdf

以後再打COCO這種比賽，估計得專門招個寫爬蟲的夥計給全網爬圖了...

奉勸樓下的跪舔的姿態優雅一些，我說的是只這篇文章，如果你覺得論好，請用一百字一千字一萬字總結一下精華，別只說觀點不講證據。還在教別人單詞，語不驚人死不休，飽醉豚？

jian sun自從當了Face++研究院院長，文章多了去了，有更多跪舔的機會。

--------------------華麗的分割線------------------------------------------------------------

1、首先第一作者不是kaiming He，第三作者是Ross Girshick，是否學生搞的掛kaiming He的大旗？11月28日arxiv才掛了learning to segment every thing，12月13日就Data Distillation,神速！

2、在看到模型圖的時候，我嚇到了，transform又是什麼新的辭彙？看到後面才知道是geometric

transformations ，也就是scaling and horizontal flipping ，縮放和水平翻轉。不就是數據增廣嘛！

3、論文講到Data distillation 4個步驟:

(1) training a model on manually labeled data (just as in normal supervised learning);

(2) applying the trained model to multiple transformations of unlabeled data;

(3) converting the predictions on the unlabeled data into labels by ensembling the

multiple predictions; and

(4) retraining the model on the union of the manually labeled data and automatically labeled data.

(1) 用手動標註的數據訓練模型A， (2)用模型A去訓練數據增廣的未標註數據，（3）

（3）將未標註數據的預測結果通過ensembling多個預測結果，轉化為labels。

4、Data Distillation(DD) 在COCO Keypoint Detection ，Object Detection 驗證，可以看到有提升

歡迎關注個人專欄

推薦一個：Automatic Dataset Augmentation

講道理，我覺得這個idea的創新點很小吧...