I remember having that problem a long while back, but that was using taskbar3. I've never noticed that in xtaskbar.