Best Continuation for this Counter to Queen's Gambit Accepted [Title "White to move"]
[fen ""]

1.d4 d5 2.c4 dxc4 3.e3 e5


I've tried just taking the pawn at c4, but he takes my pawn at d4 and after exchanging some pieces I end with a broken pawn structure while we still have equal pieces.

I've came across this move many times and is dumb-struck at finding a solution.
At that point Black is simply equal and there is nothing you can do to change it ( you would do better to play e4 instead e3 but ...e5 still solves all of Black's problems ). If you wish to play e4 instead of e3 then you need the book Lars Schandorff-Playing the Queen's Gambit ( 2012!) and be sure to get the  second edition ( first was in 2009 ). He covers this line quite well.

Your best bet is to try and steer the game into main lines with isolated queen pawn. There you will find subtle move orders that may offer you a small advantage. I can not remember the details, so I can not show you the lines, but the position you aim for is something like this :

[Title "Typical position for QGA"]
[StartFlipped "0"]
[fen "r1bq1rk1/pp2bppp/2n1pn2/8/2BP4/2N2N2/PP2QPPP/R1BR2K1 b - - 0 1"]


You will need good coverage of this opening as theory changes rapidly. I can recommend the following books:

To learn how to play with the isolated queen's pawn you can get Andrew Soltis-Pawn Structure Chess ( 1995 ) ( perhaps my answer to this question can help you too; if you need further help feel free to ask by leaving a comment ) and for Queen's Gambit accepted I will recommend the one written by Semkov &amp; Sakaev or J.Rizzitano-How to beat d4. You can also get Starting out-Queen's Gambit Accepted ( 2006 ) if you need to grasp the basics.

SUMMARY:

Play Bxc4, then Nf3 and try to get into favorable line of the QGA. If you fail to do so, do not panic! You are both fine, better player wins here! Learn how to play these positions and you will have equal chances to win the game.

Good luck and best regards!