references: Only find references in unseen packages
references: Only find references in unseen packages

This will allow us to find packages in parts of the graph we have not seen yet.
This change with the use of cached import graphs should allow us to find
references much faster, but still return correct results.

sourcegraph/go-langserver,lambdalab/go-langserver,sourcegraph/go-langserver,sourcegraph/go-langserver,lambdalab/go-langserver,lambdalab/go-langserver