Remove bad assertion
Remove bad assertion

The assertion relied on the assumption that poll() doesn't modify the
values of the fd field of pollfd structs passed to it, which is not the
case.

aidanholm/netchat,aidanholm/netchat