Fix bag/set (^) without affecting set
Fix bag/set (^) without affecting set

This should address the issues that led to the previous revert.

jonathanstowe/rakudo,salortiz/rakudo,raydiak/rakudo,nbrown/rakudo,usev6/rakudo,usev6/rakudo,sjn/rakudo,tony-o/rakudo,jonathanstowe/rakudo,labster/rakudo,sjn/rakudo,labster/rakudo,sjn/rakudo,nbrown/rakudo,jonathanstowe/rakudo,salortiz/rakudo,salortiz/rakudo,softmoth/rakudo,awwaiid/rakudo,Gnouc/rakudo,usev6/rakudo,awwaiid/rakudo,nbrown/rakudo,samcv/rakudo,MasterDuke17/rakudo,salortiz/rakudo,softmoth/rakudo,tony-o/rakudo,tbrowder/rakudo,rakudo/rakudo,tbrowder/rakudo,sjn/rakudo,MasterDuke17/rakudo,tony-o/rakudo,samcv/rakudo,salortiz/rakudo,awwaiid/rakudo,tbrowder/rakudo,Gnouc/rakudo,usev6/rakudo,tony-o/rakudo,awwaiid/rakudo,nbrown/rakudo,raydiak/rakudo,Gnouc/rakudo,lucasbuchala/rakudo,rakudo/rakudo,raydiak/rakudo,Gnouc/rakudo,MasterDuke17/rakudo,tony-o/rakudo,labster/rakudo,tony-o/rakudo,rakudo/rakudo,sjn/rakudo,tbrowder/rakudo,jonathanstowe/rakudo,nbrown/rakudo,awwaiid/rakudo,labster/rakudo,labster/rakudo,Gnouc/rakudo,salortiz/rakudo,samcv/rakudo,samcv/rakudo,labster/rakudo,softmoth/rakudo,lucasbuchala/rakudo,MasterDuke17/rakudo,rakudo/rakudo,nbrown/rakudo,softmoth/rakudo,lucasbuchala/rakudo,Gnouc/rakudo,rakudo/rakudo,MasterDuke17/rakudo,MasterDuke17/rakudo,softmoth/rakudo,raydiak/rakudo,jonathanstowe/rakudo,lucasbuchala/rakudo,tbrowder/rakudo,tbrowder/rakudo,usev6/rakudo,lucasbuchala/rakudo,rakudo/rakudo,samcv/rakudo