WordNet (r) 3.0 (2006):

step by step
    adv 1: in a gradual manner; "the snake moved gradually toward
           its victim" [syn: gradually, bit by bit, step by
           step]
    2: proceeding in steps; "the voltage was increased stepwise"
       [syn: stepwise, step by step]

Moby Thesaurus II by Grady Ward, 1.0:

38 Moby Thesaurus words for "step by step":
   bit by bit, by degrees, by inches, by inchmeal, by slow degrees,
   by stages, consecutively, degreewise, drop by drop, gradatim,
   grade by grade, gradually, hand running, hierarchically,
   in Indian file, in a chain, in a line, in a row, in a series,
   in column, in file, in order, in sequence, in series,
   in single file, in succession, in turn, inch by inch, inchmeal,
   little by little, progressively, running, sequentially, serially,
   seriatim, slowly, successively, turn about