In modern times, what would happen to a Roman Catholic monk if he were to violate his monastic vows? So, I was watching a documentary about the Codex Gigas, or "Devil's Bible", and it said that the Codex Gigas was written by a single scribe over the course of decades as a way of doing penance of what he did when he was young and stupid. Punishment for breaking monastic vows may involve being walled up alive.

In modern times, what would happen to a monk if he were to violate his monastic vows? Would he have to take on a new calligraphic project of the Bible?
There are several different sets of rules for different communities of monks; and the specific penalties for the monks are dependent on the rule.

Most communities of monks or nuns are governed by one of two sets of rules: the Rule of St. Augustine (of Hippo), and the Rule of St. Benedict (of Nursia). There are also governing documents such as the Statutes of the Carthusians, but these are the two primary documents. Since this particular monk was a Benedictine, let's look at the penalties of the Benedictine Rule.

St. Benedict of Nursia, who lived in the early sixth century, developed a rule by which the monks under his care could live. The Order of St. Benedict has a translation of the Rule online, and the Rule has several sections dealing with how monks are to be disciplined. If a monk is a troublemaker, his superior (probably a prior) is enjoined to discuss the problem with him privately. If the behavior is still a problem, the superior will talk privately with him again. If that doesn't help, he will be called out in front of the community and rebuked publicly. If even that doesn't solve the problem, the monk will be excommunicated: that is, excluded from the community (see Chapter 23 of the Rule).

St. Benedict envisioned two kinds of excommunication: for less serious and more serious faults. For less serious faults, the monk (or sister; the same Rule applies to both male and female communities)


  shall take [his] food alone after the community meal, so that if they eat at the sixth hour, for instance, that [brother] shall eat at the ninth, while if they eat at the ninth hour [he] shall eat in the evening, until by a suitable satisfaction [he] obtains pardon.


("Rule of St. Benedict", Chapter 24)

If a more serious fault is involved, the monk is excluded both from the common meal time and from common worship time. Further, no one is allowed to talk to him or even approach him:


  Let none of the brethren join him either for company or for conversation. Let him be alone at the work assigned him, abiding in penitential sorrow and pondering that terrible sentence of the Apostle where he says that a man of that kind is handed over for the destruction of the flesh, that the spirit may be saved in the day of the Lord (1 Cor. 5:5). Let him take his meals alone in the measure and at the hour which the Abbot shall consider suitable for him. 


("Rule", Chapter 25)

The abbot of the monastery is enjoined to send spiritual help to the monk, in the hope of helping him and solving the problem. He is also entitled to use corporal punishment ("the rod") in severe cases. I highly doubt this punishment is used any longer, though I have no evidence either way.

The (second to) last resort is to have the community pray for the afflicted brother, in the hope that the Lord will help where the Abbot cannot; but if even this doesn't work, the abbot is told to expel the monk from the monastery:


  But if [he] is not healed even in this way, then let the [Abbot] use the knife of amputation, according to the Apostle's words, "Expel the evil one from your midst" (1 Cor. 5:13), and again, "If the faithless one departs, let [him] depart" (1 Cor. 7:15) lest one diseased sheep contaminate the whole flock. 


("Rule", Chapter 28)

I don't see anything so harsh as walling up a monk alive.