The Collaborative International Dictionary of English v.0.48:

Initially \In*i"tial*ly\, adv.
   In an initial or incipient manner or degree; at the
   beginning. --Barrow.
   [1913 Webster]

Moby Thesaurus II by Grady Ward, 1.0:

19 Moby Thesaurus words for "initially":
   ab initio, ab ovo, aborigine, at first, at the start,
   before everything, chiefly, first, first and foremost, first thing,
   firstly, from the beginning, in the beginning, mainly, originally,
   primarily, primitively, primo, principally