What does 素 mean in 素数? Composite number is translated as 合数, which do make sense; 
Prime number is called 素数, what does 素 means here? 

Is it possible to trace the source of such translation?
「素」generically means basal, essential, intrinsic, primal. Hence you have vocabulary like


元素 (atom), the fundamental constituent of matter (well, the name was given before they discovered subatomic particles)
詞素 (morpheme), the fundamental constituent of words
要素 (essential factor)