Londonist has an article about London’s Lost Museums, including the Theatre Museum, the Cuming Museum, and the Bramah Tea and Coffee Museum. You could argue that not everything featured in the article is a museum – the London Planetarium, for example – but it’s a nice sampling of the diversity of now-lost venues.