What makes a contemporary chess problem decent? One can set up some pieces and concoct a situation that has, say, a mate in 3 (yes, I know it is not straightforward, but if you sit for a few hours you can do it).

Such problem would probably not be accepted for a competition because the modern standards are already too high. There has to be a relatively original idea and/or theme, but I do not know what qualifies as mediocre these days, and what qualifies as novel.
One might simply look at the judging metrics at the World Chess Composition Tournament (WCCT) to derive such an answer.

This is from their Rules Annex and details how points are allocated to a composition entry. I've stressed key words that probably represent a figure of merit.


  Points : Description
  
  4.0 : An outstanding problem: an accurate and intensive rendering of the set theme, without blemishes in any of the main lines, and showing
  originality and flair. Perfect construction and economy.
  
  3.5 : As above, but some small constructional weaknesses, and perhaps not ideally economical.
  
  3.0 : Either: a very good problem showing the theme clearly but perhaps not intensively or very originally Or: a task rendering of the theme
  which does not reach the highest artistic standard. In either case,
  good construction and economy.
  
  2.5 : As above, but constructional weakness and/or less than ideal economy because of the intensive or task setting.
  
  2.0 : A good problem, very likely worth an honourable mention or commendation in a reasonably strong tourney, but not a very intensive
  rendering of the theme, and perhaps not very original. Adequate to
  good construction and economy.
  
  1.5 : As above, but with some artistic weakness or constructional blemish.
  
  1.0 : A very ordinary piece of work, typical of many average columns but hardly up to award standard. Adequate construction and economy.
  
  0.5 : As above, but with serious constructional defects.
  
  0.0 : Unsound, unthematic or fully anticipated.