Published inTowards Data Science·Nov 22Managing Your Cloud-Based Data Storage with RcloneHow to optimize data transfer across multiple object storage systems — As companies become more and more dependent on cloud-based storage solutions, it is imperative that they have the appropriate tools and techniques for effective management of their big data. In previous posts (e.g., here and here) we have explored several different methods for retrieving data from cloud storage and demonstrated…Data7 min readData7 min read
Published inTowards Data Science·Nov 15Accelerating PyTorch Training Workloads with FP8How to make the most of your modern-day GPU — The past few years have seen revolutionary advancements in the field of AI, perhaps best exemplified by the recent popularity and proliferation of LLM-based applications such as ChatGPT. These breakthroughs have been powered by equally exciting developments in the machinery used to train AI models. New and innovative architectures, sophisticated…AWS8 min readAWS8 min read
Published inTowards Data Science·Sep 20PyTorch Model Performance Analysis and Optimization — Part 6How to Identify and Analyze Performance Issues in the Backward Pass with PyTorch Profiler, PyTorch Hooks, and TensorBoard — This is the sixth part in our series of posts on the topic of analyzing and optimizing PyTorch models using PyTorch Profiler and TensorBoard. In this post we will tackle one of the more complicated types of performance issues to analyze — a bottleneck in the backward-propagation pass of a…Pytorch8 min readPytorch8 min read
Published inTowards Data Science·Sep 4Effective Load Balancing with Ray on Amazon SageMakerA method for increasing DNN training efficiency and reducing training costs — In previous posts (e.g., here) we expanded on the importance of profiling and optimizing the performance of your DNN training workloads. Training deep learning models — especially large ones — can be an expensive undertaking. Your ability to maximize the utilization of your training resources in a manner that both…Sagemaker10 min readSagemaker10 min read
Published inTowards Data Science·Aug 31How to Optimize Your DL Data-Input Pipeline with a Custom PyTorch OperatorPyTorch Model Performance Analysis and Optimization — Part 5 — This post is the fifth in a series of posts on the topic of performance analysis and optimization of GPU-based PyTorch workloads and a direct sequel to part four. In part four, we demonstrated how PyTorch Profiler and TensorBoard can be used to identify, analyze, and address performance bottlenecks in…Pytorch6 min readPytorch6 min read
Published inTowards Data Science·Aug 30Deep Learning Training on AWS InferentiaYet another money-saving AI-model training hack — The topic of this post is AWS’s home-grown AI chip, AWS Inferentia — more specifically, the second-generation AWS Inferentia2. This is a sequel to our post from last year on AWS Trainium and joins a series of posts on the topic of dedicated AI accelerators. Contrary to the chips we…Cloud Ml Engine5 min readCloud Ml Engine5 min read
Published inTowards Data Science·Aug 26Solving Bottlenecks on the Data Input Pipeline with PyTorch Profiler and TensorBoardPyTorch Model Performance Analysis and Optimization — Part 4 — This is the fourth post in our series of posts on the topic of performance analysis and optimization of GPU-based PyTorch workloads. Our focus in this post will be on the training data input pipeline. In a typical training application, the host’s CPUs load, pre-process, and collate data before feeding…Pytorch11 min readPytorch11 min read
Published inTowards Data Science·Aug 10PyTorch Model Performance Analysis and Optimization — Part 3How to reduce “Cuda Memcpy Async” events and why you should beware of boolean mask operations — This is the third part of a series of posts on the topic of analyzing and optimizing PyTorch models using PyTorch Profiler and TensorBoard. Our intention has been to highlight the benefits of performance profiling and optimization of GPU-based training workloads and their potential impact on the speed and cost…Performance11 min readPerformance11 min read
Published inTowards Data Science·Jun 26Managing the Cloud Storage Costs of Big-Data ApplicationsTips for Reducing the Expense of Using Cloud-Based Storage — With the growing reliance on ever-increasing amounts of data, modern day companies are more dependent than ever on high-capacity and highly scalable data-storage solutions. For many companies this solution comes in the form of cloud-based storage service, such as Amazon S3, Google Cloud Storage, and Azure Blob Storage, each of…Optimization11 min readOptimization11 min read
Published inTowards Data Science·Jun 19PyTorch Model Performance Analysis and Optimization — Part 2How to Identify and Reduce CPU Computation In Your Training Step with PyTorch Profiler and TensorBoard — This is the second part of a series of posts on the topic of analyzing and optimizing a PyTorch model running on a GPU. In our first post we demonstrated the process — and the significant potential — of iteratively analyzing and optimizing a PyTorch model using PyTorch Profiler and…Optimization15 min readOptimization15 min read
