I'm not an expert in this area, but I do all the bills at our house. In Money Magazine, I found out about mint.com
I love this website. It took me about an hour to download my bank info and my credit card info and set up my budget.
It emails you alerts. Yesterday it told me I was over my grocery budget for the month. Today it sent me a weekly summary of activity. It really shows you how much you are spending, which I kind of wish it didn't!
According to their website: With Mint, you can achieve better online financial management in less than five minutes. After that, revolutionary, patent–pending Mint money management software does the rest, with virtually no more work required. It automatically pulls together your bank, credit union and credit card data, and provides up–to–date and amazingly accurate views of your financial—life from the big picture to specific details, in a friendly and intuitive way.
I know lots of people worry about Internet security, and their website explains their security policies.