I wasn't planning to cut back the butterfly bush, but it already had a significant amount of new growth, so I figured it was safe to trim it back to about 18". Amazing how some plants grow better by being cut back each year. When the weather warms a bit more I will do the same to the miniature rose bush.
The green growth of the peony, columbine, thyme, oregano, mums and mint naturally die off at the end of each season. I trimmed out all of the dead leaves of those plants. It is nice to finally have some established perennials in the garden.
On a related note the bulbs are starting to peak up through the dirt as well. The irises didn't bloom as well last year as they have in the past, so they got "cut back" as well. They are rhizomes so I unearthed several of the larger nodes, broke off the smaller rhizomes and put back a few of the big ones and a few of the small ones. The result of dividing the irises is... lots of little irises with no homes.
After adding a few each to the mum pots (I hope they provide a little interest in the back of the herb bed until the mums bloom) I still have my gathering basket full of baby irises. I think it is time for some guerrilla gardening; there is an ugly bare spot near the retaining wall above the the train tracks. Hopefully they will be as prolific there as they have been in my garden and my walk to the train will be brightened each morning for many summers to come.