If you can’t wear kookie clothes when gardening, you just can’t period. I would start with something trendy but maybe something you wouldn’t have the guts to wear anytime else… patterned leggings.
Sure you may feel a little like you’re in Desperately Seeking Susan, but who cares?