feature: extended strip_proppers() to remove URL and HTML artifacts
feature: extended strip_proppers() to remove URL and HTML artifacts

The function strip_proppers() we inherited from Brandon Rose's implementation; now it also looks for a forward-slash prefix and an equals-suffix and discards tokens that meet these criteria. This consolidates some patterns we were seeing where recurring HTML tokens and URLs were recurring often enough to end up in topics.

It's worth considering whether we really want to ignore the URLs altogether, as frequently-recurring URLs probably represent newsletters sent by email.
We should probably handle these elsewhere so they can be ranked up or down for special treatment. The topic model probably isn't the place to do this, though.

...

mikedelong/aarhus