Your bitcoin wallet is where you keep your bitcoins. I will not go into every detail but I keep my bitcoins in what is called cold storage and I also have a hot wallet. For cold storage I download the VanityGen (https://bitcointalk.org/index.php?topic=25804.0). By running this I get a public key and private key. The public key is what you give to receive your bitcoins and can be seen on the Bitcoin ledger along with the balance. Your private key is like the password to access those funds. I will not disclose how I keep my private key safe but many people have different approaches on how to keep it safe including paper wallets, USB in a safety deposit box, etc.
Your hot wallet is like your wallet that you would carry in public with just enough money to spend. I downloaded the Blockchain Bitcoin wallet on my cell phone and put my spending money there. It is not completely secure because if someone hacks your cell phone they can eventually get your bitcoins. But it is convenient, PIN protected and allows you to send and receive bitcoins using a QR code.