Fine-Tuning LLaMA 2 with torchrun
Published:
Fine-tuning large models like LLaMA 2 is a big task, but with torchrun, you can scale it across multiple GPUs with ease. In this post, I’ll walk you through how to do it step by step.
Posts by Tags
AI Infrastructure
Fine-Tuning LLaMA 2 with
Algorithm
Tensor Train Decomposition and Training
Published:
High-dimensional data (tensors) appear in many fields such as scientific computing, quantum physics, and machine learning. However, storing and operating on these tensors is challenging due to the exponential growth of parameters with the number of dimensions (the so-called “curse of dimensionality”). Tensor Train (TT) decomposition is one way to represent high-dimensional tensors in a compact format by expressing them as a sequence of smaller 3D tensors (often called TT-cores).
Fine-tuning
Fine-Tuning LLaMA 2 with torchrun
Published:
Fine-tuning large models like LLaMA 2 is a big task, but with torchrun, you can scale it across multiple GPUs with ease. In this post, I’ll walk you through how to do it step by step.
Machine Learning
Tensor Train Decomposition and Training
Published:
High-dimensional data (tensors) appear in many fields such as scientific computing, quantum physics, and machine learning. However, storing and operating on these tensors is challenging due to the exponential growth of parameters with the number of dimensions (the so-called “curse of dimensionality”). Tensor Train (TT) decomposition is one way to represent high-dimensional tensors in a compact format by expressing them as a sequence of smaller 3D tensors (often called TT-cores).
Parallel Computing
Fine-Tuning LLaMA 2 with torchrun
Published:
Fine-tuning large models like LLaMA 2 is a big task, but with torchrun, you can scale it across multiple GPUs with ease. In this post, I’ll walk you through how to do it step by step.
Tensor
Tensor Train Decomposition and Training
Published:
High-dimensional data (tensors) appear in many fields such as scientific computing, quantum physics, and machine learning. However, storing and operating on these tensors is challenging due to the exponential growth of parameters with the number of dimensions (the so-called “curse of dimensionality”). Tensor Train (TT) decomposition is one way to represent high-dimensional tensors in a compact format by expressing them as a sequence of smaller 3D tensors (often called TT-cores).
conda
Install Miniconda on Server
Published:
Step 1: Download Miniconda Locally
Run the following command to download the latest Miniconda installer for Linux (adjust the link if using macOS):
Python Env for Scientific Experiment
Published:
As we usually need to work on various project, the environment for the experiment some times different from the local environment. Docker is one of the option, to run on the virtual machine. However, Conda or mamba is more like the standard for current academia.
docker
Docker Container Debug with VScode
Published:
When using the docker container, one may want to use the feature in the vscode to remotely work on the code debug. For personal server, this would guarantee a relatively clean environment together with the debug feature.
Utilize the Docker container
Published:
CUDA Docker Container Setup and Usage Guide This tutorial covers how to build, run, attach, and detach a CUDA-enabled Docker container supporting three NVIDIA A6000 GPUs.
env
Docker Container Debug with VScode
Published:
When using the docker container, one may want to use the feature in the vscode to remotely work on the code debug. For personal server, this would guarantee a relatively clean environment together with the debug feature.
Utilize the Docker container
Published:
CUDA Docker Container Setup and Usage Guide This tutorial covers how to build, run, attach, and detach a CUDA-enabled Docker container supporting three NVIDIA A6000 GPUs.
Install Miniconda on Server
Published:
Step 1: Download Miniconda Locally
Run the following command to download the latest Miniconda installer for Linux (adjust the link if using macOS):
Python Env for Scientific Experiment
Published:
As we usually need to work on various project, the environment for the experiment some times different from the local environment. Docker is one of the option, to run on the virtual machine. However, Conda or mamba is more like the standard for current academia.
pyTorch
Fine-Tuning LLaMA 2 with torchrun
Published:
Fine-tuning large models like LLaMA 2 is a big task, but with torchrun, you can scale it across multiple GPUs with ease. In this post, I’ll walk you through how to do it step by step.
python
Tensor Train Decomposition and Training
Published:
High-dimensional data (tensors) appear in many fields such as scientific computing, quantum physics, and machine learning. However, storing and operating on these tensors is challenging due to the exponential growth of parameters with the number of dimensions (the so-called “curse of dimensionality”). Tensor Train (TT) decomposition is one way to represent high-dimensional tensors in a compact format by expressing them as a sequence of smaller 3D tensors (often called TT-cores).
Docker Container Debug with VScode
Published:
When using the docker container, one may want to use the feature in the vscode to remotely work on the code debug. For personal server, this would guarantee a relatively clean environment together with the debug feature.
Utilize the Docker container
Published:
CUDA Docker Container Setup and Usage Guide This tutorial covers how to build, run, attach, and detach a CUDA-enabled Docker container supporting three NVIDIA A6000 GPUs.
Install Miniconda on Server
Published:
Step 1: Download Miniconda Locally
Run the following command to download the latest Miniconda installer for Linux (adjust the link if using macOS):
Python Env for Scientific Experiment
Published:
As we usually need to work on various project, the environment for the experiment some times different from the local environment. Docker is one of the option, to run on the virtual machine. However, Conda or mamba is more like the standard for current academia.
vscode
Docker Container Debug with VScode
Published:
When using the docker container, one may want to use the feature in the vscode to remotely work on the code debug. For personal server, this would guarantee a relatively clean environment together with the debug feature.