Sign In Start Free Trial
Account

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying Accelerate Model Training with PyTorch 2.X
  • Table Of Contents Toc
  • Feedback & Rating feedback
Accelerate Model Training with PyTorch 2.X

Accelerate Model Training with PyTorch 2.X

By : Maicon Melo Alves
4.4 (10)
close
close
Accelerate Model Training with PyTorch 2.X

Accelerate Model Training with PyTorch 2.X

4.4 (10)
By: Maicon Melo Alves

Overview of this book

This book, written by an HPC expert with over 25 years of experience, guides you through enhancing model training performance using PyTorch. Here you’ll learn how model complexity impacts training time and discover performance tuning levels to expedite the process, as well as utilize PyTorch features, specialized libraries, and efficient data pipelines to optimize training on CPUs and accelerators. You’ll also reduce model complexity, adopt mixed precision, and harness the power of multicore systems and multi-GPU environments for distributed training. By the end, you'll be equipped with techniques and strategies to speed up training and focus on building stunning models.
Table of Contents (17 chapters)
close
close
Free Chapter
1
Part 1: Paving the Way
4
Part 2: Going Faster
10
Part 3: Going Distributed

Conventions used

There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: “The ipex.optimize function returns an optimized version of the model.”

A block of code is set as follows:

config_list = [{    'op_types': ['Linear'],
    'exclude_op_names': ['layer4'],
    'sparse_ratio': 0.3
}]

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

def forward(self, x):    out = self.layer1(x)
    out = self.layer2(out)
    out = out.reshape(out.size(0), -1)
    out = self.fc1(out)
    out = self.fc2(out)
    return out

Any command-line input or output is written as follows:

maicon@packt:~$ nvidia-smi topo -p -i 0,1Device 0 is connected to device 1 by way of multiple PCIe

Bold: Indicates a new term, an important word, or words that you see onscreen. For instance, words in menus or dialog boxes appear in bold. Here is an example: “OpenMP is a library used for parallelizing tasks by harnessing all the power of multicore processors by using the multithreading technique.”

Tips or important notes

Appear like this.

Unlock full access

Continue reading for free

A Packt free trial gives you instant online access to our library of over 7000 practical eBooks and videos, constantly updated with the latest in tech
bookmark search playlist download font-size

Change the font size

margin-width

Change margin width

day-mode

Change background colour

Close icon Search
Country selected

Close icon Your notes and bookmarks

Delete Bookmark

Modal Close icon
Are you sure you want to delete it?
Cancel
Yes, Delete

Confirmation

Modal Close icon
claim successful

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY