PERF: Categorical getitem perf (#30747)
PERF: Categorical getitem perf (#30747)

Convert to an array earlier on.
Closes https://github.com/pandas-dev/pandas/issues/30744
gfyoung/pandas,pandas-dev/pandas,jorisvandenbossche/pandas,dsm054/pandas,gfyoung/pandas,jorisvandenbossche/pandas,jreback/pandas,jreback/pandas,datapythonista/pandas,rs2/pandas,jreback/pandas,rs2/pandas,TomAugspurger/pandas,pandas-dev/pandas,gfyoung/pandas,gfyoung/pandas,dsm054/pandas,datapythonista/pandas,pandas-dev/pandas,datapythonista/pandas,datapythonista/pandas,TomAugspurger/pandas,TomAugspurger/pandas,gfyoung/pandas,rs2/pandas,TomAugspurger/pandas,dsm054/pandas,jorisvandenbossche/pandas,jreback/pandas,jorisvandenbossche/pandas,rs2/pandas,dsm054/pandas,jreback/pandas,dsm054/pandas,pandas-dev/pandas