Can I use a multimeter to test the outgoing voltage of this power supply? I don't want to blow out my multimeter or my electronics, so is it safe to test the DC output voltage with a multimeter on my OEM power supply?
If your multimeter can handle the voltage, it should be safe. 12&nbsp;V should be in the capability range of common multimeters, so I would tend to say yes, - but make sure that your multimeter can handle the voltage and be sure it is set to voltage mode in the appropriate range (if set to current measurement mode for example, it will not survive).

Also take the typical precautions when measuring with a multimeter, as described in, for example, Safety equipment and precautions for DC circuit experimentation and development.