From Notebook to Production with Amazon SageMaker Put Machine Learning in the hands of every developer and data scientist Our mission Application Services Platform Services Frameworks & Infrastructure API-driven services: Vision, Language & Speech Services, Chatbots Deploy machine learning models with high-performance machine learning algorithms, broad framework support, and one-click training, tuning, and inference. Develop sophisticated models with any framework, create managed, auto-scaling clusters of GPUs for large scale training, or run prediction on trained models. AWS ML Stack h t t p s : / / m l . a w s h t t p s : / / m e d i u m . c o m / @ j u l s i m o n / a - m a p - f o r - m a c h i n e - l e a r n i n g - o n - a w s - a 2 8 5 f c d 8 d 9 3 2 Data Visualization & Analysis Business Problem ML problem framing Data Collection Data Integration Data Preparation & Cleaning Feature Engineering Model Training & Parameter Tuning Model Evaluation Are Business Goals met? Model Deployment Monitoring & Debugging Yes No Data Augmentation Feature Augmentation The Machine Learning Process Re-training Predictions Amazon SageMaker Pre-built notebooks for com m on problem s K-Means Clustering Principal Component Analysis Neural Topic Modelling Factorization Machines Linear Learner XGBoost Latent Dirichlet Allocation Image Classification Seq2Seq, And more! ALGORITHMS Apache MXNet, Chainer TensorFlow, PyTorch, scikit-learn FRAMEWORKS S e t u p a n d m a n a g e e n v i r o n m e n t s f o r t r a i n i n g T r a i n a n d t u n e m o d e l ( t r i a l a n d e r r o r ) D e p l o y m o d e l i n p r o d u c t i o n S c a l e a n d m a n a g e t h e p r o d u c t i o n e n v i r o n m e n t Built-in, high- perform ance algorithm s Build Git integration Elastic inference Git integration Elastic Inference https://aws.amazon.com/blogs/aws/amazon-elastic-inference-gpu-powered-deep-learning-inference-acceleration/ Amazon SageMaker Pre-built notebooks for com m on problem s K-Means Clustering Principal Component Analysis Neural Topic Modelling Factorization Machines Linear Learner XGBoost Latent Dirichlet Allocation Image Classification Seq2Seq, And more! ALGORITHMS Apache MXNet, Chainer TensorFlow, PyTorch, scikit-learn FRAMEWORKS S e t u p a n d m a n a g e e n v i r o n m e n t s f o r t r a i n i n g T r a i n a n d t u n e m o d e l ( t r i a l a n d e r r o r ) D e p l o y m o d e l i n p r o d u c t i o n S c a l e a n d m a n a g e t h e p r o d u c t i o n e n v i r o n m e n t Built-in, high- perform ance algorithm s Build New built-in algorithms scikit-learn environment Model marketplace Search Search training jobs Amazon SageMaker Pre-built notebooks for com m on problem s Built-in, high- perform ance algorithm s One-click training Hyperparam eter optim ization Train Deploy model in production Scale and manage the production environment P3DN, C5N TensorFlow on 256 GPUs Resume HPO tuning job Build Amazon SageMaker Fully m anaged hosting with auto- scaling One-click deploym ent Pre-built notebooks for com m on problem s Built-in, high- perform ance algorithm s One-click training Hyperparam eter optim ization Deploy Model compilation Elastic inference Inference pipelines Train Build Amazon SageMaker Fully m anaged hosting with auto- scaling One-click deploym ent Pre-built notebooks for com m on problem s Built-in, high- perform ance algorithm s One-click training Hyperparam eter optim ization Deploy Model compilation Elastic inference Inference pipelines Train Build P3DN, C5N TensorFlow on 256 GPUs Resume HPO tuning job New built-in algorithms scikit-learn environment Model marketplace Search Git integration Elastic inference Working with Amazon SageMaker The Amazon SageMaker API • Python SDK orchestrating all Amazon SageMaker activity • High-level objects for algorithm selection, training, deploying, automatic model tuning, etc. • Spark SDK (Python & Scala) • AWS CLI: ‘ aws sagemaker ’ • AWS SDK: boto3, etc. Model Training (on EC2) Model Hosting (on EC2) Training data Model artifacts Training code Helper code Helper code Inference code Ground Truth Client application Inference code Training code Inference request Inference response Inference Endpoint Training code Factorization Machines Linear Learner Principal Component Analysis K-Means Clustering XGBoost And more Built-in Algorithms Bring Your Own Container Bring Your Own Script Model options Built-in algorithms orange: supervised, yellow: unsupervised Linear Learner : regression, classification Image Classification : Deep Learning (ResNet) Factorization Machines : regression, classification, recommendation Object Detection (SSD) : Deep Learning (VGG or ResNet) K -Nearest Neighbors: non-parametric regression and classification Neural Topic Model : topic modeling XGBoost : regression, classification, ranking https://github.com/dmlc/xgboost Latent Dirichlet Allocation : topic modeling (mostly) K -Means: clustering Blazing Text : GPU-based Word2Vec, and text classification Principal Component Analysis : dimensionality reduction Sequence to Sequence : machine translation, speech to text and more Random Cut Forest : anomaly detection DeepAR : time-series forecasting (RNN) Object2Vec : general-purpose embedding IP Insights : usage patterns for IP addresses Semantic Segmentation : Deep Learning XGBoost • Open Source project • Popular tree-based algorithm for regression, classification and ranking • Builds a collection of trees. • Handles missing values and sparse data • Supports distributed training • Can work with data sets larger than RAM https://github.com/dmlc/xgboost https://xgboost.readthedocs.io/en/latest/ https://arxiv.org/abs/1603.02754 Optimizing TensorFlow https://aws.amazon.com/blogs/machine-learning/faster-training- with-optimized-tensorflow-1-6-on-amazon-ec2-c5-and-p3- instances/ (March 2018) Training a ResNet-50 benchmark with the synthetic ImageNet dataset using our optimized build of TensorFlow 1.11 on a c5.18xlarge instance type is 11x faster than training on the stock binaries. https://aws.amazon.com/about-aws/whats- new/2018/10/chainer4-4_theano_1-0- 2_launch_deep_learning_ami/ (October 2018) Automatic Model Tuning