Does anyone know the title of this book?

About a Victorian-era deaf gentleman who lived in London, a proficient lip-reader and keen observer of society figures. He frequented the Cafe Royal to sit and write, and said cafe also happened to be the hang-out of society figures like Oscar Wilde – which meant he got the scoop on scandals and events just by watching people. No idea if it’s fiction or an account by a real person.

I read it many years ago but can neither recall its title nor locate it anymore. I’ve Googled extensively, asked many librarians and posted my question on various book- and reading-related sites to no avail. Can anyone help?