When do you use 所有， 全部， 整个, 完整? I have a hard time wrapping about these specific words. 

As far as I know 所有 is most similar to "all of the..." like “所有的人”.
While 全部 is more or less like 所有 except you don't need 的 between it and the object like “全部时间". 

Meanwhile, 整个 is most similar to "the whole..." "整个下午"

But 完整 is very similar to 整个 like "完整的句子. 

So can I ask how their specific usages differ? Are there times you should only use a certain one while other times it's fine to interchange?
That's an interesting question. The answer basically boils down to differences in grammatical role of these words: 

所有 “all (of the)”, refers to all entities pertaining to a certain context. Given the meaning of 所 and 有 alone you can effectively think of it as meaning "all (things) that are here". It is mainly used with 的 before a noun or in a relative sentence and it often comes with a 都 "all".


  我们家被偷了，所有的在保险柜里的钱也被偷了 Our house was robbed, all (of the) money that was in the safe was stolen too.
  我所有认识的人都念过大学 All the people (that) I know went to university.


In general 全部 means "everything, all" and refers to one single entity in its entirety. So for instance it can't be used to refer to a group of things, unless you use a collective noun.


  他说什么，我没听到全部 I didn't hear everything he said
  这本小说，我还没全部看完 I haven't finished this book yet (haven't read it in its entirety)
  这群人我并不认识全部 I don't know everyone in this group of people


整个 "whole“ is used in place of a number (actually 一 "one" can be omitted) and before a classifier to mean "one whole thing" (obviously you can change the number to say N whole things):


  她吃掉了一整桶冰淇淋 She ate a whole bucket of ice cream.
  一整天不够讲完这个故事 A whole day isn't enough to tell this story


完整 "complete". You often use it as an adjective in noun phrases with 的, as: 


  一个完整的句子 a complete sentence 
  一个完整的解决方案 a complete solution