Does not compute full gradient of function, but optimizes one unknown at a time Holds all unknowns constant while performing line search on a single variable Rinse + repeat