What does 同父异母或同母异父的兄弟 mean? I think it means half-brother, but my book says it is step brother. I thought step brother is 继兄弟. What does 继 mean?
继兄弟 = stepbrothers.  Two unrelated people became brothers when their parents married each other
Stepbrothers are a result of a second marriage
同父异母兄弟 (同父兄弟/ 异母兄弟) = half brothers. When two brothers have the same father but different mothers (A man had a son with one woman and then had another son with a different woman.)
同母异父兄弟 (同母兄弟/ 异父兄弟) = half brothers. When two brothers have the same mother but different fathers (A woman had a son with one man and then had another son with a different man.)
Half brothers are mostly a result of a second marriage (but not always)

what does 继 mean?

继 = 继任 - to succeed (a position)
继父, 继母 = stepfather, stepmother (继父 succeeds the previous father, 继母 succeeds the previous mother)
继兄弟 doesn't mean a brother succeeds a previous brother. It means the brother's relationship is through a 继父, 继母's marriage