Expertise for Athletics, & Sneak Attack for more damage.
I can definitely see a College of Swords: Bard (Xanathar's Guide) although a Lore Bard is also tempting its just too much of a squishy caster to rely this much on melee. Now if the College of Swords: Bard could get Unarmed Fighting as a fighting style that would tempt me to abandon the fighter entirely :)