The Collaborative International Dictionary of English v.0.48:
Guiltily \Guilt"i*ly\ (g[i^]lt"[i^]*l[y^]), adv. In a guilty manner. [1913 Webster]WordNet (r) 3.0 (2006):
guiltily adv 1: in the manner of someone who has committed an offense; "she blushed guiltily as she spoke"