In the 1740s, a house at 64 St. James was converted to a chocolate house and became, by the Regency era, a noted Tory Chocolate House. Accounts of high and even foul play by the 1780s battled with reputation for hosting fashionable gentlemen (Timbs); the Cocoa Tree “had a double-edged reputation as a fashionable hell” (Thevoz, 2018). Lord Byron, for example, was a member of the Cocoa Tree (Timbs, 1906).
Journal of the Society Arts, 1890
And another description:
Memorials of St. James Street, 1922
More about Chocolate Houses: https://www.atlasobscura.com/articles/history-of-gentlemens-clubs
Thevoz, S. (2018). Club Government: How the Early Victorian World Was Ruled from London Clubs. I.B. Tauris.
Timbs, J. (1906). Clubs and Club Life in London.