Whisk sugar, flour, and salt in a saucepan; in a separate bowl, whisk egg yolks with milk and heavy cream. Gradually combine wet and dry ingredients in the saucepan, whisking to prevent lumps.

Cook the mixture over medium heat, stirring constantly until it gently boils. Reduce heat and cook for 1-2 minutes until thickened and coats the back of a spoon.

Remove from heat, then stir in butter and vanilla extract until melted. Pour into a clean bowl, cover with plastic wrap to prevent a skin, and cool completely or chill for at least 2 hours.

While the pudding cools, slice firm bananas into 1/4 to 1/2-inch thick rounds and gather your vanilla wafers.

Arrange a single layer of vanilla wafers at the bottom of a trifle bowl or 9x13 inch dish, breaking some if needed to fit.

Cover the wafer layer with about half of the sliced bananas, spreading them evenly. Spoon about one-third of the cooled pudding over the bananas, gently spreading it out.

Repeat layers with wafers, remaining bananas, and another third of pudding, then a final layer of wafers and the last of the pudding. Refrigerate for at least 4 hours, or overnight, for flavors to meld and wafers to soften; optionally top with whipped cream before serving.