Reducing Background Noise
You usually want to reduce noise in your recordings, unless you are purposefully leaving some ambiance in to create a sense of space (for example, people talking in the background in a crowded restaurant, or fans at a sports game). But, for most voice recording, you'll want a crisp, dry sound that can easily be combined with music or other audio.
Reducing noise in your recordings is easiest done by not recording it in the first place. Make sure to turn off fans, air conditioners, furnaces, and anything else that is making noise in your recording space. If you have to have your computer in your room with you, try to use a microphone that is designed to reject noise from the rear, and keep the microphone faced away from the computer.
A second source of noise is your voice's reflection off of hard surfaces like walls, hard floors, and the surface of your desk. You can experiment with carpet, curtains over reflective glass windows, and hanging spare blankets on walls or in a corner where there is a troublesome echo.

Comments
Got my work cut out for me. I’ll be recording in my office, at my computer, mini-blinds but no curtains, hardwood floors… But I picked up some tips just now for counteracting all that so thanks. (you might guess from all the comments I’m leaving today that I’m new to the podcasting scene but I’m spending this Saturday afternoon digging into your site and bookmarking pages like crazy.)