The assassination occurred five days after the commander of the Confederate Army of Northern Virginia , General Robert E. Lee , surrendered to Lieutenant General Ulysses S. Grant and the Union Army of the Potomac , ending the American Civil War .