From George Washington’s American foxhound “Sweetlips” to the Obamas’ pair of Portuguese water dogs, Bo and Sunny, the White House has a long history of presidential pets. In fact, of all the presidents in the more than 200-year history of the United States, only two have chosen not to take a pet into the White House: James K Polk in the 1840s and Donald Trump.

In that time, the White House has also been home to some somewhat peculiar animals. Although the tale may be apocryphal, John Quincy Adams reportedly kept a pet alligator in the East Room of the White House. Thomas Jefferson famously owned two brown bear cubs, which he was eventually compelled to donate to the Peale Museum in Philadelphia when they proved too dangerous to keep at home. Martin Van Buren was gifted two tiger cubs by the Sultan of Oman. Benjamin Harrison kept two opossums known as Mr. Reciprocity and Mr. Protection — names inspired by the 1896 Republican party slogan. Theodore Roosevelt owned a garter snake his daughter chose to name Emily Spinach. And Calvin Coolidge had a pygmy hippopotamus known as Billy — or, to give him his full name, “Mr. William Johnson Hippopotamus.”

Of all the presidents’ pets, however, one of the most memorable was a Congo African grey parrot called Poll, or Polly, that was owned by President Andrew Jackson. What made Polly such a memorable addition to the presidency, moreover, had less to do with President Jackson’s life, and more to do with what happened after his death.

Polly was reportedly originally President Jackson’s wife Rachel’s pet, but when she passed away of a heart attack very unexpectedly in December 1828—just days after her husband’s election victory and a matter of weeks before his inauguration—Jackson adopted Polly as his own.

As president, Jackson was known for his no-nonsense attitude—a reputation that he had originally earned during his military service when the soldiers under his command had given him the nickname “Old Hickory” for his toughness and straightforwardness. And it seems at least some of that plain-speaking candor rubbed off on Polly, as before long she had amassed quite a rich vocabulary of curse words.

Alas, as well as being capable of mimicking their owners’ somewhat fruity language, parrots are also known for their longevity. So when President Jackson himself passed away in Nashville 1845 at the age of 78, Polly was still going strong—having now outlived both of her former owners. As a result, Polly was brought along to her keeper’s funeral, along with more than 1,000 guests and mourners gathered to celebrate the life of their former Commander in Chief.

Unfortunately, Polly didn’t quite appreciate the reverence of the occasion, and shortly after President Jackson’s funeral began, the bird started swearing very loudly and very raucously. “Before the sermon and while the crowd was gathering,” recalled the funeral’s cleric, the Reverend William Menefee Norment, in a later account, “a wicked parrot that was a household pet got excited, and commenced swearing so loud and long as to disturb the people.” The bird, Norment explained, was apparently “excited by the multitude” of people in front of it and, as a result, “let loose perfect gusts of cuss words.” People were “horrified and awed at the bird’s lack of reverence,” and eventually Polly “had to be carried from the house.”

What happened to Polly after the shambolic funeral of President Jackson is unknown. But her indecent outbursts nevertheless earned her—and her equally uncompromising owner—a place in the history books.