1) Yes and no. Increased memory usage after a recycle is a sign that a module isn't clearing out its memory upon recycle. However, sometimes it just happens...especially if you recycle over and over and over again.

2) No, your memory should not increase every time you open a popup. This would be a good place to start when looking for a bad module. Id check around to see if someone has also reported a bug with a xPopup setting.

3) jkey is loaded as part of OTS2. I thought that it was loaded by step.rc but I could be wrong. The j modules were released by jugg therefore they are the greatest modules ever written. :)

4) I would find out what module it is first. Once you do that disable any eye candy and see if that helps.