BMTrain
stable

Getting Started

  • Installation
  • Quick Start
  • Introduction to Core Technology

Package Reference

  • bmtrain
  • bmtrain.nccl
  • bmtrain.inspect
  • bmtrain.lr_scheduler
BMTrain
  • BMTrain’s Documentation!
  • Edit on GitHub

BMTrain’s Documentation!

BMTrain is an efficient large model training toolkit that can be used to train large models with tens of billions of parameters. It can train models in a distributed manner while keeping the code as simple as stand-alone training.


Getting Started

  • Installation
    • Install BMTrain
    • Compilation Options
    • Recommended Configuration
    • FAQ
  • Quick Start
    • Step 1: Initialize BMTrain
    • Step 2: Enable ZeRO-3 Optimization
    • Step 3: Enable Communication Optimization
    • Step 4: Launch Distributed Training
    • Other Notes
  • Introduction to Core Technology
    • ZeRO-3 Optimization
    • Overlap Communication and Computation
    • CPU Offload

Package Reference

  • bmtrain
    • Initialization
    • Distributed Parameters and Modules
    • Methods for Parameters
    • Utilities
  • bmtrain.nccl
  • bmtrain.inspect
  • bmtrain.lr_scheduler
    • LR Schedulers

API

  • Index

  • Module Index

  • Search Page

Next

© Copyright 2022, OpenBMB. Revision e8799fb0.

Built with Sphinx using a theme provided by Read the Docs.