I like So Luckys ideas about a bench (for watching sunsets) a birdbath, and grasses to soften the boulder over your pipe and add some airy movement. I love Northern Sea Oats myself, they have really attractive seed heads, and I don't think they self sow.
At least I haven't noticed them coming up all over my garden.
There are some really nice cranesbill geraniums for ground cover, some have small leaves and are short and more compact than others. They also have a nice fall color.
I like
Lancastriense a pastel pink shorty.
Threadleaf coreopsis is a nice airy perennial (Moonbeam is pale yellow) and looks great with echinaceas and nepeta, which I love for it's non stop bloom all summer.
There are short eupatoriums, Joe Pye weed, "Little Joe" is shorter than the regular guy, and I think they just came out with an even shorter one in the last few years?
Or maybe I just made that up.
Meadow sage is great for all summer bloom, although it's a deep blue, not pastel.
Centranthus ruber is a nice perennial, common names are Jupiters Beard and Keys of Heaven.
If you can find it Gillenia trifoliata is a wonderful perennial, grows up shrub like, but is actually herbaceous and dies down to the ground in winter. It's covered with star shaped pastel pink blooms right now, and then turns russety in the fall.
One of my favorites. I can't find it locally anymore, but bought it years ago as a tiny plant at a perennial nursery that specialized in unusual plants. It's about 3'x3' now.
I know this isn't an area you're asking advice on, but smokebush would do great on your walkway against the house.
You can keep it coppiced so it doesn't overwhelm that bed, and the foliage can't be beat IMO.
Here's a pic of one of mine as a little squirt, this one is "Grace", I love the color of it's leaves. I don't care for the blooms, so coppicing is good for avoiding that yearly.
