For this post, we will use pyvnml, a Python wrapper for the library.Īmazon CloudWatch is great for monitoring your workload on your EC2 instances without needing to set up, manage, or scale the systems and infrastructure around it. For more information on this library, go to the reference manual. With nvidia-smi, users query information about the GPU utilization, memory consumption, fan usage, power consumption, and temperature of their NVIDIA GPU devices.īecause nvidia-smi is based on the NVIDIA Management Library ( NVML), we can use this C-based API library to capture the same data points to send to Amazon CloudWatch as a custom metric.

The current common practice to help with monitoring and management of GPU-enabled instances is to use NVIDIA System Management Interface ( nvidia-smi), a command line utility. For an Amazon Machine Image (AMI), we recommend that your instance uses the Amazon Deep Learning AMI. In this blog post, I’ll show you how you can monitor your GPU and memory usage using Amazon CloudWatch metrics. Amazon Web Services allows you to spin up P2 or P3 instances that are great for running Deep Learning frameworks such as MXNet, which emphasizes speeding up the deployment of large-scale deep neural networks.Īs data scientists and developers fine-tune their network, they want to optimize their GPU utilization so that they are using appropriate batch sizes.

Deep learning requires a large amount of matrix multiplications and vector operations that can be parallelized by GPUs (graphics processing units) because GPUs have thousands of cores.

Ubuntu monitor ram usage