It sounds like you're saying that all other themes are still showing desktop icons? That's kind of unusual, since most themes don't have desktop icon support (you could be just getting those, though). You could try updating your build to RC3, which is also on this site. Just download it, and then right click your desktop >> Litestep >> utilities >> Update Build.

Oh, btw, you can get rid of the desktop icons in Austerity by opening the theme.rc file in the theme's root folder, and commenting out *NetLoadModule icondesk-0.66 by putting a ";" infront of it.