<jupyter_start><jupyter_text>Hash Vectorizer  Default number of featuresUsing the Hash vectorizer function which only looks at word countsCan limit the number of features - will go back to that but start with default<jupyter_code>hashVector = HashingVectorizer()
Hash_dataSet = hashVector.transform(dataSet)
print(Hash_dataSet.shape, Hash_dataSet.size,"\n")
#print(Hash_dataSet,"\n")<jupyter_output>(1479, 1048576) 46993<jupyter_text>Cosine Similarity<jupyter_code>cossim = sklearn.metrics.pairwise.cosine_similarity(Hash_dataSet)
print(cossim,"\n")
cosineSimilaritySorted = numpy.argsort(cossim, axis=1)
print(cosineSimilaritySorted,"\n")
top5Similar = cosineSimilaritySorted[:,-6:-1]
print(top5Similar)<jupyter_output>[[1.         0.19720266 0.08175191 ... 0.23314715 0.12942072 0.31934404]
 [0.19720266 1.         0.16927779 ... 0.27586342 0.11136921 0.22327675]
 [0.08175191 0.16927779 1.         ... 0.29110162 0.11109411 0.12460152]
 ...
 [0.23314715 0.27586342 0.29110162 ... 1.         0.25136748 0.29720128]
 [0.12942072 0.11136921 0.11109411 ... 0.25136748 1.         0.39451112]
 [0.31934404 0.22327675 0.12460152 ... 0.29720128 0.39451112 1.        ]] 

[[ 465  327  317 ... 1191  495    0]
 [ 879  985  465 ...  268  932    1]
 [ 334  398  640 ...  208 1132    2]
 ...
 [1251  183  350 ...  795 1170 1476]
 [1010  640  993 ... 1063  894 1477]
 [ 208  914 1305 ...  546  279 1478]] 

[[ 606 1315 1120 1191  495]
 [1241  401  555  268  932]
 [ 181 1173 1050  208 1132]
 ...
 [ 818 1435 1371  795 1170]
 [ 178 1094  966 1063  894]
 [ 163  602  499  546  279]]<jupyter_text>This provides the same output as the Count Vectorizer cosine similarity, so may have to change the feature number to see if that does anything.<jupyter_code>print(dataSet[932])
print(dataSet[268])<jupyter_output><empty_output><jupyter_text>Euclidean Distance<jupyter_code>euclid = pairwise_distances(Hash_dataSet, metric='euclidean')
euclidSorted = numpy.argsort(euclid, axis=1)
top5Similar = euclidSorted[:,1:6]
print(top5Similar)

print(dataSet[1241])
print(dataSet[1365])<jupyter_output>white woman with brown hair and eyes and clear even skin upper middle class probably a democrat and a feminist
white female early 20 s with blond hair and green eyes direct personality straight forward attitude confident and honest<jupyter_text>Manhattan distanceThis took too long to run and unlikely to produce better results<jupyter_code>manhat = pairwise_distances(Hash_dataSet, metric='manhattan')

manhatSorted = numpy.argsort(manhat, axis=1)
top5Similar = manhatSorted[:,1:6]
print(top5Similar)

print(dataSet[894])
print(dataSet[1094])<jupyter_output>this person is 5 2 weighs 180 pounds her shoe size is 6 5 she enjoys reading biking and going to movies she is not married and has three cats that she calls her furbabies she is a teacher so she is very busy
she is fair complected with light freckles she has beautiful large blue eyes and full pink lips she is shy she enjoys reading she is intelligent she is compassionate<jupyter_text>Hash Vectorizer - Limit to 1000 features<jupyter_code>hashVector = HashingVectorizer(n_features=1000)
Hash_dataSet = hashVector.transform(dataSet)
print(Hash_dataSet.shape, Hash_dataSet.size,"\n")
#print(Hash_dataSet,"\n")<jupyter_output>(1479, 1000) 46376<jupyter_text>Cosine Similarity<jupyter_code>cossim = sklearn.metrics.pairwise.cosine_similarity(Hash_dataSet)
print(cossim,"\n")
cosineSimilaritySorted = numpy.argsort(cossim, axis=1)
print(cosineSimilaritySorted,"\n")
top5Similar = cosineSimilaritySorted[:,-6:-1]
print(top5Similar)<jupyter_output>[[1.         0.23816526 0.08035434 ... 0.22481776 0.12942072 0.31934404]
 [0.23816526 1.         0.17861771 ... 0.34073314 0.13076645 0.27426491]
 [0.08035434 0.17861771 1.         ... 0.27590308 0.11912174 0.12247143]
 ...
 [0.22481776 0.34073314 0.27590308 ... 1.         0.24238715 0.2865835 ]
 [0.12942072 0.13076645 0.11912174 ... 0.24238715 1.         0.39451112]
 [0.31934404 0.27426491 0.12247143 ... 0.2865835  0.39451112 1.        ]] 

[[ 350  327  465 ... 1191  495    0]
 [1108 1010  839 ...  401  932    1]
 [ 640  183  176 ...  208 1132    2]
 ...
 [ 183  669  350 ... 1170  795 1476]
 [ 768  712 1407 ...  894 1063 1477]
 [ 208 1010   21 ...  279  546 1478]] 

[[1120 1084  606 1191  495]
 [  71 1241  268  401  932]
 [1270  181 1050  208 1132]
 ...
 [ 401 1049  815 1170  795]
 [ 430  927  966  894 1063]
 [ 163  602  499  279  546]]<jupyter_text>Now somewhat different values than the Count vectorizer but some still the same<jupyter_code>#print(dataSet[163])
#print(dataSet[602])
#print(dataSet[499])
#print(dataSet[279])
#print(dataSet[546])
##good set except for 499

print(dataSet[815])
print(dataSet[1170])<jupyter_output>this man is about 5 feet 10 inches tall he is of medium build and looks slightly athletic his skin is very pale as he stays out of the sun he has moderately hairy arms and freckles on his arms and hands he trims his nails regularly his legs are long and he walks swiftly wherever he goes this man deals with some slight anger issues he is very competitive and likes to play softball and football at the moment he is not in a relationship and he is often too busy to maintain relationships his job requires him to travel a lot he enjoys sports bars
he has shaggy short straight red hair he has very light but bushy eyebrows and a long narrow face he has full lips and a thin mustache as well as a bit of scruff on his chin his ears are small and close to his head he nose is long and wide he s intense and hard working he has is curious about the world and likes to spend time with his friends and family he goes out and enjoys drinking and going to concerts<jupyter_text>Euclidean Distance<jupyter_code>euclid = pairwise_distances(Hash_dataSet, metric='euclidean')
euclidSorted = numpy.argsort(euclid, axis=1)
top5Similar = euclidSorted[:,1:6]
print(top5Similar)

print(dataSet[401])
print(dataSet[268])<jupyter_output>he s got a short and newly growing in beard with a mustache he s got bushy eyebrows and short straight hair his eyes are bright and wide and his ears are very close to his head his nose is long and thin he seems fun loving and kind of laid back he doesn t get too worked up over things and likes to keep his life simple and easy going
caucasian male with blonde hair and blue eyes has a beard and a mustache he looks geeky kind of grumpy and maybe into technology<jupyter_text>Gives a similar output to the cosine similairty output  Manhattan distanceRuns faster with less features<jupyter_code>manhat = pairwise_distances(Hash_dataSet, metric='manhattan')

manhatSorted = numpy.argsort(manhat, axis=1)
top5Similar = manhatSorted[:,1:6]
print(top5Similar)

print(dataSet[894])
print(dataSet[1094])<jupyter_output>this person is 5 2 weighs 180 pounds her shoe size is 6 5 she enjoys reading biking and going to movies she is not married and has three cats that she calls her furbabies she is a teacher so she is very busy
she is fair complected with light freckles she has beautiful large blue eyes and full pink lips she is shy she enjoys reading she is intelligent she is compassionate
StarcoderdataJupyterStructuredCleanDedup