According to proponents of penal substitutionary atonement, were the Levitical sacrifices also penal substitutions? I am not a proponent of penal substitutionary atonement. While considering this question, I considered whether my position is incorrect, and the language of Isaiah 53 necessarily implies penal substitution. I asked myself if a Hebrew father would be reasonable to say something similar to these words while consoling his own son who asked him why they slaughter a sin offering. 

It occurred to me that, Christ's atoning sacrifice and the Levitical sacrifices being considered in parallel, if these latter are penal substitutions according to the Old Covenant, I might have to reconsider my position on the matter.

So, my question is:

According to proponents of penal substitutionary atonement, are the Levitical sin offerings in their very nature penal substitutions? If so, what exegetical arguments are given to support that conclusion?
Charles Hodge (Systematic Theology, 3.7.6) and Herman Bavinck (Reformed Dogmatics, v3, III.7) are two Reformed theologians who make a number of exegetical arguments from Old Testament passages to defend penal substitution.  I'll focus on their treatments of the sin offering and similar sacrifices, on the following points:

Significance of blood (Leviticus 17)
Placing of hands and bearing of sin (Leviticus 16)
Satisfaction by a substitute (Deuteronomy 21)

The significance of blood
Hodge begins by arguing for the significance of blood in OT sacrifices for expiation, that is, &quot;to cover [sin] from the sight of God’s justice.&quot;  He cites the prohibition on eating blood in Leviticus 17:10–12, and writes:

The last clause of the verse, which in our version is rendered, “For it is the blood that maketh an atonement for the soul,” is more literally and correctly rendered, “For blood by (its) soul or life makes atonement;” or, as Bähr and Fairbairn translate it, “The blood atones through the soul.” The latter writer correctly remarks, “This is the only sense of the passage that can be grammatically justified; for the preposition ב after the verb to atone (כפר) invariably denotes that by which the atonement is made; while as invariably the person or object for which is denoted by ל or על.”

Based on this analysis, Hodge argues that this and similar passages cannot be considered mere symbols of devotion or means of reforming a sinner, and must be understood as &quot;giving soul for soul, life for life.&quot;
Bavinck cites the same passage to argue that the blood itself achieves atonement:

The idea was not a death for the sake of death but to obtain by it the blood that had to effect atonement. [...] the blood is the bearer of the soul, the carrier of a life that had again been made free from sin after and by the slaying.

The placing of hands and the bearing of sin
Hodge and Bavinck argue that the placing of hands on the sacrifice is significant.  Bavinck writes that &quot;by the laying on of hands, the offerer transferred his sin to the animal&quot;:

In Scripture the laying on of hands always implies some kind of transmission: of a blessing (Gen. 48:13; Matt 19:13), of a curse (Lev. 24:14), of an office (Num. 27:18; Deut. 34:9), of the Holy Spirit (Acts 8:17), and so on, and similarly in the case of bloody sacrifices as well as burnt and thank offerings (Lev. 1:4; 3:2) for sin that has been acknowledged and confessed.

Hodge calls special attention to the goats of Leviticus 16, arguing that two goats are needed in order to fully portray the atonement:

In the solemn services of the great day of atonement, the import of this rite is rendered especially clear. It was commanded that two goats should be selected, one for a sin-offering and the other for a scape-goat. The two constituted one sacrifice, as it was impossible that one could signify all that was intended to be taught. Of the scape-goat it is said, “Aaron shall lay both his hands upon the head of the live goat, and confess over him all the iniquities of the children of Israel, and all their transgressions in all their sins, putting them upon the head of the goat, . . . . and the goat shall bear upon him all their iniquities unto a land not inhabited.” This renders it plain that the design of the imposition of hands was to signify the transfer of the guilt of the offender to the victim.

Once transferred, the victims are said &quot;to bear the sin&quot; of the offender.  Hodge argues that in this context this phrase must refer to expiation, not spiritual renovation:

In point of fact the words in question [נָשָּׂאצָיֹו;] always refer to bearing the punishment and thus removing the guilt of sin, and never to the removal of moral pollution. This is plain, (1.) Because נָשָּׂא is interchanged with סָבַל which never means to remove, but only to sustain, or bear as a burden. (2.) Because usage determines the meaning of the phrase and is uniform. In Numbers xiv. 34, it is said, “Ye shall bear your iniquities forty years.” Leviticus v. 1, “If a soul . . . . hear the voice of swearing, and is a witness; . . . . if he do not utter it, he shall bear his iniquity.” Leviticus v. 17, “He is guilty, and shall bear his iniquity.” Leviticus vii. 18, “The soul that eateth of it shall bear his iniquity.” Leviticus xvii. 16, “If he wash not . . . . then he shall bear his iniquity.” Leviticus xix. 8; xx. 17; xxii. 9, “They shall keep my ordinance, lest they bear sin for it.” Numbers ix. 13, If a man forbear to keep the passover, he shall be cut off from the people, “he shall bear his sin.” See also Numbers xviii. 22, 32. Ezekiel iv. 4, 5, it is said to the prophet enduring penance, “So shalt thou bear the iniquity of the house of Israel.” “Thou shalt bear the iniquity of the house of Judah forty days.” “Lie thou upon thy left side . . . . according to the number of the days that thou shalt lie upon it, thou shalt bear their iniquity.” Ezekiel xviii. 20, “The son shall not bear the iniquity of the father; neither shall the father bear the iniquity of the son.” In all these, and in other like cases, it is simply impossible that “bearing sin” should mean the removal of sin by moral renovation. The expression occurs some forty times in the Bible, and always in the sense of bearing the guilt or punishment of sin.

Satisfaction
For treatment of the idea of satisfaction more explicitly, we must turn to other types of offerings.  Regarding Numbers 35:31, Hodge writes:

In Numbers xxxv. 31, it is said, “Ye shall take no satisfaction (כֹּפֶר, λύτρα, pretium), for the life of a murderer, which is guilty of death; but he shall be surely put to death . . . . the land cannot be cleansed (יְכֻפַּר; Septuagint, ἐξιλασθήσεται; Vulgate, nec aliter expiari potest) of the blood that is shed therein, but by the blood of him that shed it.” Here again there can be no mistake. To cover sin, כּפֵּר, is to expiate it by a penal satisfaction; that expiation is expressed, as we have seen, by כֹּכֶּר, which literally magnifies that which covers, and, in such connections, that which covers sin so that it no longer demands punishment.

Here, only the muderer's own blood can satisfy.  But in Deuteronomy 21:1–9, when the murderer cannot be found, the death of a heifer can satisfy:

A כֹּפֶר is a satisfaction. This satisfaction must be made either by the offender or by some one in his stead. In the case of murder, if the perpetrator could not be discovered, a victim was to be slain in his stead, and thus satisfaction was to be made. The law in reference to this case makes the nature and design of sin offerings perfectly plain. The elders of the nearest city were commanded to take a heifer which had not borne the yoke, and wash their hands over it in attestation of their innocence of the blood of the murdered man; the priests being present. The heifer was to be slain, and thus expiation made for the offence.

Verses 7 and 8 contain the testimony of the elders:

‘Our hands did not shed this blood, nor did our eyes see it shed. 8 Accept atonement, O Lord, for your people Israel, whom you have redeemed, and do not set the guilt of innocent blood in the midst of your people Israel, so that their blood guilt be atoned for.’ [ESV]

Hodge concludes, &quot;The removal of guilt by a vicarious death is, therefore, the Scriptural idea of a sin offering.&quot;
Summary
Bavinck and Hodge are both quick to note that the Old Testament sacrificial system had a number of deficiencies, such as its applicability to only a few sins, the sinfulness of the priests, and its requirement of endlessly repeating sacrifices.  Its effectiveness thus ultimately depended on Christ's atonement, but it nonetheless served as a type that pointed to that perfect sacrifice.  Thus, they argue that OT indications of the significance of the blood, the transmission of guilt, and the substitute's satisfaction are key elements in Christ's sacrifice, and evidence for the penal substitutionary view of atonement.