The first place to go when starting out is indiepodder.org to learn the what? where? when? and how? and to download the free iPodder podcast receiver - or use iTunes, if you already have it.

Don't forget - you only need a computer to listen to podcasts, you don't HAVE to have an iPod (or any other portable mp3 player) - they are simply mp3 files delivered to you when they are released by the broadcaster.

More useful info available at Wikipedia