Deep Learning 101 (深度學習 101 ) - Tutorial 新手村 入門 教學


2018/05 公司投資添購(開箱) NVIDIA TITAN V + NVIDIA TITAN XP
2023/08 公司投資添購新設備 RTX 6000 Ada 48 GB * 2 和 A 100 80GB * 4


嗯 ... 因為陸續會有 Deep Learning 101 (Tutorial) 深度學習 新手村 入門 教學 的需求 ...

2017/04/29 科技部科國司 / 國立成功大學:智慧聯網融入跨領域工程人才培育計畫
機器學習技術與應用介紹 (Deep Learning 101)
http://blog.twman.org/2017/04/ncku-engineering-education.html

所以我想好像是該像當初的 " 惡意程式分析 / 惡意程式行為分析 "  http://blog.twman.org/2014/07/Malware-Analysis.html

好好的把整個心酸血淚史做個記錄;畢竟自己也是多虧幾位前輩不吝指點才勉強略懂的 ! 
我會努力不定時的持續更新的 !


如果有路過的神人,發現有寫錯的地方也請不吝指正啊 ! 跪 ... 本文會有大量的文字,慎入 !

首先 ! 還是得先廣告一下過去的幾篇文章 ... 可能會感覺有點混亂,不如就先大概瞄一下吧 !

好了,那麼接著要分成兩個部份來講 ... 一個是偏學術,一個是偏業界
其實是剛好我自己就是卡在這兩種的中間,所以勉強算的上是都有點心得感想 ! 我會盡量從中間來思考給點建議 ! 首先,你要先決定是玩真的還是玩假的,玩真的那請你先存點零用錢或是跟你的主管申請 ... 買張 GPU 來體驗 (就是傳說中的想搞大數據,請先準備個 10TB 硬碟空間的概念一樣);可以參考上方快速建置 TensorFlow 那篇 ... 如果你真的沒錢 ... 我想你還是先再考慮一下,不要像我硬是自己挖洞給自己跳 !

開始之前先快速瞭解一下到底要幹麼以及現在有那些工具可以使用吧 !

3 分鐘搞懂深度學習到底在深什麼:https://panx.asia/archives/53209

Google. (2015). TensorFlow is an Open Source Software Library for Machine Intelligence. vailable: https://www.tensorflow.org
FACEBOOK. (2015). Torch: A Scienitific Computing Framework for LUAJIT. Available: http://torch.ch
Microsoft Cognitive Toolkit (CNTK): http://cntk.ai
Amazon. (2016). Deep Scalable Sparse Tensor Network Engine (DSSTNE). Available: https://github.com/amznlabs/amazon-dsstne
Theano Development Team. Theano. Available: http://deeplearning.net/software/theano/
Yangqing Jia, Caffe Deep learning framework by the BVLC. Available: http://caffe.berkeleyvision.org
François Chollet, Deep Learning library for Python. Available: https://github.com/fchollet/keras
Baidu, Paddle: PArallel Distributed Deep Learning. Available: https://github.com/baidu/Paddle
TensorLayer: Deep Learning and Reinforcement Learning Library for TensorFlow: http://tensorlayer.org
A Powerful, Flexible, and Intuitive Framework for Neural Networks: http://chainer.org

 ==================== 學術界 ====================
如果你決定要從學術界觀點出發,你得要先有心理準備 ... 這會是非常難的一件事
因為套句咱家 CM 哥 (國文鼎淵獵豹清流哥) 最常對我講的:學術界要的是一個創新的 MODEL ,不是新的領域應用;所以,你就先認命吧 ! 不懂 ? 那看看 DeepMind 的論文吧

https://deepmind.com/research/publications/

 ==================== 業界 ====================
至於業界,可能會比較舒服一點,因為你得先要有數據,還要有某特定領域的經驗,那麼會比學術界還快一點上手 ! 到這邊 ... 接著就可以開始往下講了 ! 繼續聊聊怎樣進入這新手村吧 !
而兩者最大的差別,我想就是到底做得那個東西 (系統) 能不能 " 變現 " 了

首先 Youtube 上 田神的 機器學習基石 跟 機器學習技法 看完沒 ? 講中文的你還不看 ?

另外就是這個 " 一天搞懂深度學習 ":

認命吧 ... 再怎樣看不懂都非常建議你花時間把全套影片至少看過一遍 ... 這只是讓你先對整個所謂的機器學習跟深度學習有點概念而已 ! 真心不騙的建議你一定要有耐心 !

好吧 ! 感覺寫到這邊一直在打擊人的自信心 ... 先來個最簡單的 " 遷移學習 ( Transfer Learning) " 吧 ... 然後你得先知道什麼是 Inception V3 ... 這是一個 Google 去參加影像辨識比賽 ImageNet Large Scale VisualRecognition Challenge (ILSVRC) 的成果 ,算的上目前最強大的影像辨識的模型;當然建議你也瞭解一下 Fei-Fei Li Ph.D. (Associate Professor, Stanford University) ....

那麼關於這個模型呢,這邊有2個說明提供給大家看看 !

Image Recognition:https://www.tensorflow.org/versions/master/tutorials/image_recognition/index.html
TensorFlow在图像识别中的应用:http://www.csdn.net/article/2015-12-16/2826496

然後提供幾個網路上整理蠻清楚的深度學習文章,想入門就乖乖讀一下吧

相關文章或訊息:
機器之心:http://www.jiqizhixin.com | AI科技評論:http://www.leiphone.com/category/ai

中文教學:
Unsupervised Feature Learning and Deep Learning (UFLDL):http://deeplearning.stanford.edu/wiki/index.php/UFLDL教程
Deep Learning速成教程:http://www.leiphone.com/news/201608/7lwVZCXnScbQb6cJ.html
Keras中文文档:http://keras-cn.readthedocs.io/en/latest/getting_started/FAQ/
TensorFlow中文社區:http://www.tensorfly.cn
Prof. Hsuan-Tien Lin:http://www.csie.ntu.edu.tw/~htlin/mooc/
Prof. Hung-Yi Lee:http://speech.ee.ntu.edu.tw/~tlkagk/courses_MLSD15_2.html

Tutorial:
http://deeplearning.net/tutorial/
CS231n Convolutional Neural Networks for Visual Recognition (CS231 一定要讀):http://cs231n.github.io/convolutional-networks/
CS224d: TensorFlow Tutorial:https://cs224d.stanford.edu/lectures/CS224d-Lecture7.pdf
大師解釋給你看:http://www.cs.nyu.edu/~yann/talks/lecun-ranzato-icml2013.pdf
http://www.bioinfo.org.cn/~casp/temp/DeepLearning.pdf

Github
Tensorflow:https://github.com/tensorflow/tensorflow
Keras:https://github.com/fchollet/keras
Inception v3@keras:https://gist.github.com/neggert/f8b86d001a367aa7dde1ab6b587246b5
DeepLearning Tutorials:https://github.com/wepe/MachineLearning
Deep-Learning-TensorFlow:https://github.com/blackecho/Deep-Learning-TensorFlow

下面這幾個更是大師解釋經典中的經典了啊
http://www.cs.nyu.edu/~yann/talks/lecun-ranzato-icml2013.pdf
http://www.bioinfo.org.cn/~casp/temp/DeepLearning.pdf

最後剩的就是 1. 讀 PAPER;2. 快讀 PAPER;3. 還是要讀 PAPER ... 讀不懂 ? 沒關係 ... 我也一直不懂很多地方的數學式 ... 但就是先有一個最基本的準則 ... 很多地方不要揪結太久,因為深度學習很多都是 " 試 " 出來的,而這也就是上面提到學術界跟業界最大的不一樣的地方 ... 那麼這邊就先給個覺得一定要搞懂的感受一下吧 ! 這 20 多篇估計讀懂了,應該就無敵了吧 ! XD



最後是投影片裡會提到的相關指令和網址 !!! 謝謝 !!!



pp. 19: http://www.mlyearning.org

pp. 20: https://www.deeplearningbook.org


pp. 21: http://cs231n.stanford.edu | http://cs20si.Stanford.edu | http://cs20si.Stanford.edu

pp. 22: https://www.youtube.com/c/DeepLearning101

pp. 25: https://github.com/exacity/deeplearningbook-chinese

pp. 26: http://www.leiphone.com/category/ai | http://www.jiqizhixin.com

pp. 29: International Conference on Learning Representations: http://www.iclr.cc
Advances in Neural Information Processing Systems: https://nips.cc
Computer Vision and Pattern Recognition: http://cvpr2017.thecvf.com
Association for the Advancement of Artificial Intelligence: http://www.aaai.org/Conferences/AAAI/aaai17.php

pp. 31: https://trends.google.com/trends/explore?q=deep%20learning,machine%20learning,artifical%20intelligence

pp. 37: https://arxiv.org/abs/1704.01568

pp. 39: https://arxiv.org/abs/1611.00847


pp. 41: Source: https://www.datanami.com/2017/04/11/four-common-mistakes-machine-learning-projects/

pp. 46: https://arxiv.org/abs/1608.07249


pp. 63: https://developer.nvidia.com/cuda-downloads

pp. 64: https://developer.nvidia.com/cudnn

pp. 66: 
wget http://developer.download.nvidia.com/compute/cuda/7.5/Prod/local_installers/cuda_7.5.18_linux.run
./cuda_7.5.18_linux.run -extract=/tensorflow/cuda
./cuda-linux64-rel-7.5.18-19867135.run
./cuda-samples-linux-7.5.18-19867135.run

pp. 67:
cp include/cudnn.h /usr/local/cuda-7.5/include/  
cp lib64/libcudnn* /usr/local/cuda-7.5/lib64/

pp. 68: 
pip install https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.8.0-cp27-none-linux_x86_64.whl

pp. 71: https://github.com/NVIDIA/nvidia-docker

pp. 72: https://docs.docker.com/engine/installation/linux/ubuntu/#install-using-the-repository

pp. 73: apt-get install python-pip python-dev build-essential 

pp. 74:
pip install --upgrade pip
pip list | awk '{print $1}' | xargs -n 1 pip install --upgrade

pp. 75:
add-apt-repository ppa:webupd8team/java
apt-get update apt-get install oracle-java8-installer

pp. 76:
echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list 
curl https://storage.googleapis.com/bazel-apt/doc/apt-key.pub.gpg | sudo apt-key add -
apt-get update
apt-get install bazel 
apt-get upgrade bazel

pp. 77
git clone --recurse-submodules https://github.com/tensorflow/tensorflow
cd /tensorflow/tensorflow
./configure

pp. 78
sudo apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
sudo apt-add-repository 'deb https://apt.dockerproject.org/repo ubuntu-xenial main'
sudo apt update
sudo apt install linux-image-generic linux-image-extra-virtual
sudo apt install docker-engine

pp. 79
docker run -it gcr.io/tensorflow/tensorflow:1.0.0-devel





pp. 123
https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks

pp. 125
https://arxiv.org/abs/1409.1556

pp. 127
https://arxiv.org/abs/1512.00567

pp. 130
https://arxiv.org/abs/1611.05431

pp. 134
apt-get update apt-get install wget cd /tensorflow/ wget https://storage.googleapis.com/download.tensorflow.org/models/inception_dec_2015.zip -O tensorflow/examples/label_image/data/inception_dec_2015.zip unzip tensorflow/examples/label_image/data/inception_dec_2015.zip -d tensorflow/examples/label_image/data/ bazel build -c opt --copt=-mavx tensorflow/examples/image_retraining:retrain

pp.135
bazel-bin/tensorflow/examples/image_retraining/retrain \ --bottleneck_dir=/tf_files/bottlenecks \ --model_dir=/tf_files/inception \ --output_graph=/tf_files/retrained_graph.pb \ --output_labels=/tf_files/retrained_labels.txt \ --image_dir /tf_files/flower_photos

bazel build tensorflow/examples/label_image:label_image && \ bazel-bin/tensorflow/examples/label_image/label_image \ --graph=/tf_files/retrained_graph.pb \ --labels=/tf_files/retrained_labels.txt \ --output_layer=final_result \ --image=/tf_files/flower_photos/XXX.jpg

pp. 146
https://github.com/tensorflow/models