Adafactor
Author: Aolei Cao (ac3237), Ziyang Li (zl986), Junjia Liang (jl4439) (ChemE 6800 Fall 2024)
Stewards: Nathan Preuss, Wei-Han Chen, Tianqi Xiao, Guoqing Hu
Introduction
Problem Formulation
Objectives
Minimize the loss function Minimize the loss function \( f(x) \), where \( x \in \mathbb{R}^n \) and \( x \) is the weight vector to be optimized.