Preheat the oven to 350F. Butter and lightly flour a 9x13 baking dish.
Add the flour, sugar, and baking soda to the bowl of an electric mixer.
On medium-low speed, add milk, eggs, and vanilla and beat until well combined.
In a small saucepan over medium heat, add water, butter, and peanut butter and cook until creamy and well blended, stirring frequently.
Add to the rest of the batter and mix on medium speed to combine, about 2 minutes. Pour the batter into the prepared baking dish.
Bake for 35-40 minutes or until a cake tester comes out clean. Cool cake to room temperature.
For the frosting, cream together the butter and peanut butter in an electric mixer with a paddle attachment. Slowly add the sugar and beat until all sugar is gone. Thin with the milk, if needed, to reach the desired consistency.
Using a spatula, spread frosting over the cooled cake.