If you're going to read just one book on this list, make it Michael Paterson's A Brief History of Life in Victorian Britain. It's concise, well-written, and chock full of interesting details. You will definitely get a feel for the period from this book. If you want more along those lines, consider supplementing with Daniel Pool's What Jane Austen Ate and Charles Dickens Knew; however, be forewarned that Pool's book does contain a few inaccuracies!
One of the best ways to capture the language, attitude, hopes, and fears of the era is to absorb the literature of the Victorian period. You really can't go wrong with either Broadview's The Victorian Era or Longman's The Victorian Age. (The Norton anthology is okay, but a bit conservative in its approach to the canon.) Of course, since novels were of great importance throughout the 19th century, consider adding a longer work by Charles Dickens, William Makepeace Thackery, George Eliot, or Anthony Trollope to your reading list.
The reprint of the 1897 Sears, Roebuck & Co. Catalogue is a fantastic resource on the material culture of the era. Also, it's the most comprehensive equipment book you could every hope for. (If you're running a British-centric game the exchange rate was about five dollars to the British pound sterling.)
Sooner or later you will probably want to run a game in London; these are the books to use for that. Liza Picard's Victorian London is a great overview, while Drew Gray's London's Shadows will give you all the underworld stuff you want to trouble your players with.
If you're like me, eventually any game you run will take a turn for the macabre. To add a bit of the grotesque to any Victorian game I recommend Philip Sugden's The Complete History of Jack the Ripper (easily the most through of books on the Whitechapel murders) and Ronald Pearsall's The Table-Rappers (everything you might hope to learn about Victorian seances and spiritualism).
