What's the actual difference between 文 and 语? Both mean "language" with some other related meanings.

A friend of mine told me that while 文 refers more to the written language, 语 is more related to the spoken one.

Is this a difference and is it the only one?
Yes, that is the main difference between 文 and 语. And you're right to say they refer more to written/spoken language rather than having that exclusive meaning - when someone talks about 中文, for example, they often mean the spoken language as well as just the written form, while 书面语 refers to literary words usually found in books. The other differences are explained by Fefe's post.

Here is a post (in Chinese) on Baidu Zhidao asking the same question, which comes to the same conclusion: http://zhidao.baidu.com/question/86849931.html