Frequently, in Regency recounts of dances and balls, the expression “trip the light fantastic toe” was used. Based on a line in the 15th century poem L’Allegro written by John Milton, “Come, and trip it as you go / On the light fantastic toe”, this was not an expression unique to the Regency but nonetheless was a popular turn of phrase.
Trip means to dance, and fantastic in this instance is related to fancy, or extravagant.
According to Ngram viewer, the expression had its peak usage around the end of the 19th century.