<jupyter_start><jupyter_text>Copyright 2018 Google LLCThis library is free software; you can redistribute it and/ormodify it under the terms of the GNU Lesser General PublicLicense as published by the Free Software Foundation; eitherversion 2.1 of the License, or (at your option) any later version.<jupyter_code># This library is free software; you can redistribute it and/or
# modify it under the terms of the GNU Lesser General Public
# License as published by the Free Software Foundation; either
# version 2.1 of the License, or (at your option) any later version.
# 
# This library is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# Lesser General Public License for more details.
# 
# You should have received a copy of the GNU Lesser General Public
# License along with this library; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA<jupyter_output><empty_output><jupyter_text>Mitigating Unwanted Biases with Adversarial LearningAuthors: <NAME>, <NAME>, <NAME>, <NAME>, <NAME>---  Summary of this NotebookThis notebook is a guide to the paper ([archiv](https://arxiv.org/pdf/1801.07593.pdf))> ```<NAME>, <NAME> and <NAME>. Mitigating Unwanted Biases with Adversarial Learning. AAAI Conference on AI, Ethics and Society, 2018.```  Intro statement of problemEmbeddings are a powerful mechanism for projecting a discrete variable (e.g. words, locales, urls) into a multi-dimensional real valued space.  Several strong methods have been developed for learning embeddings.  One example is the [Skipgram](http://www.cs.brandeis.edu/~marc/misc/proceedings/lrec-2006/pdf/357_pdf.pdf) algorithm.  In that algorithm the surrounding context is used to predict the presence of a word.  Unfortunately, much real world textual data has subtle bias that machine learning algorithms will implicitly include in the embeddings created from that data.  This bias can be illustrated by performing a word analogy task using the learned embeddings.It is worth noting that the usages of terms like _fair_ and _bias_ are used in this notebook in the context of a particular definition of fairness sometimes referred to as "Demographic Parity" or "Equality of Outcomes" ([Hardt et. al 2016](http://papers.nips.cc/paper/6373-equality-of-opportunity-in-supervised-learning)).  This definition of fairness effectively says that any relationship at all between a variable of interest and a _protected variable_ is an example of unwanted bias.  Other definitions of fairness such as "Equality of Odds" can be employed when there is believed to be some form of proper relationship between the variable of interest and the protected variable.  However, all uses of _fair_ and _bias_ here should be interpreted in the context of "Demographic Parity". First, we'll import all the packages that we'll need.<jupyter_code>!pip install -U gensim~=3.2.0
import gensim
import gzip
import numpy as np
import os
import pandas as pd
!pip install --upgrade-strategy=only-if-needed tensorflow~=1.6.0rc0
import tensorflow as tf<jupyter_output><empty_output><jupyter_text>First let's copy the required data files from Google Cloud Storage to a local directory.To do this we'll first need to authenticate to Google Cloud Storage. Executing the following cell will generate a link which you'll need to follow to get a verification code.<jupyter_code>from google.colab import auth
auth.authenticate_user()<jupyter_output><empty_output><jupyter_text>Now we'll sync the data for the colab to a tmp directory from Google Cloud Storage.<jupyter_code>project_id = 'mledu-fairness'
!gcloud config set project {project_id}

gcs_bucket_name = 'mledu-fairness/colabs/debias_word_embeddings'
local_dir_name = '/tmp/debias_word_embeddings'
if not os.path.exists(local_dir_name):
  print "creating dir %s" % local_dir_name
  !mkdir {local_dir_name}
  
!gsutil rsync gs://{gcs_bucket_name} {local_dir_name}

!ls -al {local_dir_name}  

WORD2VEC_FILE = os.path.join(local_dir_name, "GoogleNews-vectors-negative300.bin.gz")
ANALOGIES_FILE = os.path.join(local_dir_name, "questions-words.txt")

def load_word2vec_format(f, max_num_words=None):
  """Loads word2vec data from a file handle.

  Similar to gensim.models.keyedvectors.KeyedVectors.load_word2vec_format
  but takes a file handle as input rather than a filename. This lets us use
  GFile. Also only accepts binary files.

  Args:
    f: file handle
    max_num_words: number of words to load. If None, load all.

  Returns:
    Word2vec data as keyedvectors.EuclideanKeyedVectors.
  """
  header = f.readline()
  vocab_size, vector_size = (
      int(x) for x in header.rstrip().split())  # throws for invalid file format
  print "vector_size =  %d" % vector_size
  result = gensim.models.keyedvectors.EuclideanKeyedVectors()
  num_words = 0
  result.vector_size = vector_size
  result.syn0 = np.zeros((vocab_size, vector_size), dtype=np.float32)
  
  def add_word(word, weights):
    word_id = len(result.vocab)
    if word in result.vocab:
      print("duplicate word '%s', ignoring all but first", word)
      return
    result.vocab[word] = gensim.models.keyedvectors.Vocab(
        index=word_id, count=vocab_size - word_id)
    result.syn0[word_id] = weights
    result.index2word.append(word)

  if max_num_words and max_num_words < vocab_size:
    num_embeddings = max_num_words
  else:
    num_embeddings = vocab_size
  print "Loading %d embeddings" % num_embeddings
  
  binary_len = np.dtype(np.float32).itemsize * vector_size
  for _ in xrange(vocab_size):
    # mixed text and binary: read text first, then binary
    word = []
    while True:
      ch = f.read(1)
      if ch == b' ':
        break
      if ch == b'':
        raise EOFError("unexpected end of input; is count incorrect or file otherwise damaged?")
      if ch != b'\n':  # ignore newlines in front of words (some binary files have)
        word.append(ch)
    word = gensim.utils.to_unicode(b''.join(word), encoding='utf-8', errors='strict')
    weights = np.frombuffer(f.read(binary_len), dtype=np.float32)
    add_word(word, weights)
    num_words = num_words + 1
    if max_num_words and num_words == max_num_words:
      break
  if result.syn0.shape[0] != len(result.vocab):
    print(
        "duplicate words detected, shrinking matrix size from %i to %i",
        result.syn0.shape[0], len(result.vocab))
  result.syn0 = np.ascontiguousarray(result.syn0[:len(result.vocab)])
  assert (len(result.vocab), vector_size) == result.syn0.shape

  print("loaded %s matrix", result.syn0.shape)
  return result
%%time
# Initialize the embeddings client if this hasn't been done yet.
# For the efficiency of this notebook we just load the first 2M words, and don't
# re-initialize the client if it already exists. You could of course filter the
# word list in other ways.
if not 'client' in vars():
  print "Loading word embeddings from %s" % WORD2VEC_FILE
  with gzip.GzipFile(fileobj=open(WORD2VEC_FILE, 'r')) as f:
    client = load_word2vec_format(f, max_num_words=2000000)<jupyter_output><empty_output><jupyter_text>The following blocks load a data file with analogy training examples and displays some of them as examples.  By changing the indices selected in the final block you can change which analogies from the training set are being displayed.<jupyter_code>def print_knn(client, v, k):
  print "%d closest neighbors to A-B+C:" % k
  for neighbor, score in client.similar_by_vector(
      v.flatten().astype(float), topn=k):
    print "%s : score=%f" % (neighbor, score)<jupyter_output><empty_output><jupyter_text>Let's take a look at the analogies that the model generates for *man*:*woman*::*boss*:$\underline{\quad}$.Try changing ``"boss"`` to ``"friend"`` to see further examples of problematic analogies.<jupyter_code># Use a word embedding to compute an analogy
# Edit the parameters below to get different analogies
A = "man"
B = "woman"
C = "boss"
NUM_ANALOGIES = 5

in_arr = []
for i, word in enumerate((A, B, C)):
  in_arr.append(client.word_vec(word))
in_arr = np.array([in_arr])

print_knn(client, -in_arr[0, 0, :] + in_arr[0, 1, :] + in_arr[0, 2, :],
          NUM_ANALOGIES)
def load_analogies(filename):
  """Loads analogies.

  Args:
    filename: the file containing the analogies.

  Returns:
    A list containing the analogies.
  """
  analogies = []
  with open(filename, "r") as fast_file:
    for line in fast_file:
      line = line.strip()
      # in the analogy file, comments start with :
      if line[0] == ":":
        continue
      words = line.split()
      # there are no misformatted lines in the analogy file, so this should
      # only happen once we're done reading all analogies.
      if len(words) != 4:
        print "Invalid line: %s" % line
        continue
      analogies.append(words)
  print "loaded %d analogies" % len(analogies)
  return analogies
analogies = load_analogies(ANALOGIES_FILE)
print "\n".join("%s is to %s as %s is to %s" % tuple(x) for x in analogies[:5])<jupyter_output><empty_output><jupyter_text>Adversarial Networks for Bias MitigationThe method presented here for removing some of the bias from embeddings is based on the idea that those embeddings are intended to be used to predict some outcome $Y$ based on an input $X$ but that outcome should, in a fair world, be completely unrelated to some protected variable $Z$.  If that were the case then knowing $Y$ would not help you predict $Z$ any better than chance.  This principle can be directly translated into two networks in series as illustrated below.  The first attempts to predict $Y$ using $X$ as input.  The second attempts to use the predicted value of $Y$ to predict $Z$.  See Figure 1 of [the paper](https://arxiv.org/pdf/1801.07593.pdf).However, simply training the weights in W based on $\nabla_WL_1$ and the weights in $U$ based on $\nabla_UL_2$ won’t actually achieve an unbiased model.  In order to do that you need to incorporate into $W$’s update function the concept that $U$ should be no better than chance at predicting $Z$.  The way that you can achieve that is analogous to how Generative Adversarial Networks (GANs) ([Goodfellow et al. 2014](http://papers.nips.cc/paper/5423-generative-adversarial-nets)) train their generators.In addition to $\nabla_WL_1$ you incorporate the negation of $\nabla_WL_2$ into $W$’s update function.  However, it’s possible that $\nabla_WL_1$ is changing $W$ in a way which will improve accuracy by using the biased information you are trying to protect.  In order to avoid that you also incorporate a term which removes that component of $\nabla_WL_1$ by projecting it onto $\nabla_WL_2$.  Once you’ve incorporated those two terms, the update function for $W$ becomes:$\nabla_WL_1-proj_{(\nabla_WL_2)}\nabla_WL_1 - \nabla_WL_2$  Defining the Protected Variable of EmbeddingsThe description of how to incorporate adversarial networks into machine learned models above is very generic because the technique is generally applicable for any type of systems which can be described in terms of input $X$ being predictive of $Y$ but potentially containing information about a protected variable $Z$.  So long as you can construct the relevant update functions you can apply this technique.  However, that doesn’t tell you much about the nature of $X$, $Y$ and $Z$.  In the case of the word analogies task above, $X = B + C - A$ and $Y = D$.  Figuring out what $Z$ should be is a little bit trickier though.  For that we can look to a paper by [Bulokbasi et. al.](http://papers.nips.cc/paper/6227-man-is-to-computer-programmer-as-woman-is-to-homemaker-debiasing-word-embeddings) where they developed an unsupervised methodology for removing gendered semantics from word embeddings.The first step is to select pairs of words which are relevant to the type of bias you are trying to remove.  In the case of gender you can choose word pairs like “man”:”woman” and “boy”:girl” which have gender as the only difference in their semantics.  Once you have these word pairs you can compute the difference between their embeddings to produce vectors in the embeddings’ semantic space which are roughly parallel to the semantics of gender.  Performing Principal Components Analysis (PCA) on those vectors then gives you the major components of the semantics of gender as defined by the gendered word pairs provided.<jupyter_code>def _np_normalize(v):
  """Returns the input vector, normalized."""
  return v / np.linalg.norm(v)


def load_vectors(client, analogies):
  """Loads and returns analogies and embeddings.

  Args:
    client: the client to query.
    analogies: a list of analogies.

  Returns:
    A tuple with:
    - the embedding matrix itself
    - a dictionary mapping from strings to their corresponding indices
      in the embedding matrix
    - the list of words, in the order they are found in the embedding matrix
  """
  words_unfiltered = set()
  for analogy in analogies:
    words_unfiltered.update(analogy)
  print "found %d unique words" % len(words_unfiltered)

  vecs = []
  words = []
  index_map = {}
  for word in words_unfiltered:
    try:
      vecs.append(_np_normalize(client.word_vec(word)))
      index_map[word] = len(words)
      words.append(word)
    except KeyError:
      print "word not found: %s" % word
  print "words not filtered out: %d" % len(words)

  return np.array(vecs), index_map, words
embed, indices, words = load_vectors(client, analogies)

embed_dim = len(embed[0].flatten())
print "word embedding dimension: %d" % embed_dim
def find_gender_direction(embed,
                          indices):
  """Finds and returns a 'gender direction'."""
  pairs = [
      ("woman", "man"),
      ("her", "his"),
      ("she", "he"),
      ("aunt", "uncle"),
      ("niece", "nephew"),
      ("daughters", "sons"),
      ("mother", "father"),
      ("daughter", "son"),
      ("granddaughter", "grandson"),
      ("girl", "boy"),
      ("stepdaughter", "stepson"),
      ("mom", "dad"),
  ]
  m = []
  for wf, wm in pairs:
    m.append(embed[indices[wf]] - embed[indices[wm]])
  m = np.array(m)

  # the next three lines are just a PCA.
  m = np.cov(np.array(m).T)
  evals, evecs = np.linalg.eig(m)
  return _np_normalize(np.real(evecs[:, np.argmax(evals)]))
# Using the embeddings, find the gender vector.
gender_direction = find_gender_direction(embed, indices)
print "gender direction: %s" % str(gender_direction.flatten())<jupyter_output><empty_output><jupyter_text>Once you have the first principal component of the embedding differences, you can start projecting the embeddings of words onto it.  That projection is roughly the degree to which a word is relevant to the latent protected variable defined by the first principle component of the word pairs given.  This projection can then be taken as the protected variable $Z$ which the adversary is attempting to predict on the basis of the predicted value of $Y$.  The code below illustrates how to construct a function which computes $Z$ from $X$ in this way. Try editing the WORD param in the next cell to see the projection of other words onto the gender direction.<jupyter_code>WORD = "she"

word_vec = client.word_vec(WORD)
print word_vec.dot(gender_direction)<jupyter_output><empty_output><jupyter_text>Let's now look at the words with the largest *negative* projection onto the gender dimension.<jupyter_code>words = set()
for a in analogies:
  words.update(a)

df = pd.DataFrame(data={"word": list(words)})
df["gender_score"] = df["word"].map(
    lambda w: client.word_vec(w).dot(gender_direction))
df.sort_values(by="gender_score", inplace=True)
print df.head(10)<jupyter_output><empty_output><jupyter_text>Let's now look at the words with the largest *positive* projection onto the gender dimension.<jupyter_code>df.sort_values(by="gender_score", inplace=True, ascending=False)
print df.head(10)<jupyter_output><empty_output><jupyter_text>Training the modelTraining adversarial networks is hard. They are touchy, and if touched the wrong way, they blow up VERY quickly. One must be very careful to train both models slowly enough, so that the parameters in the models do not diverge. In practice, this usually entails significantly lowering the step size of both the classifier and the adversary. It is also probably beneficial to initialize the parameters of the adversary to be extremely small, to ensure that the classifier does not overfit against a particular (sub-optimal) adversary (such overfitting can very quickly cause divergence!).  It is also possible that if the classifier is too good at hiding the protected variable from the adversary then the adversary will impose updates that diverge in an effort to improve its performance.  The solution to that can sometimes be to actually increase the adversary’s learning rate to prevent divergence (something almost unheard of in most learning systems).  Below is a system for learning the debiasing model for word embeddings described above.  Inspect the terminal output for the kernel to inspect the performance of the model. Try modifying the hyperparameters at the top to see how that impacts the convergence properties of the system.<jupyter_code>def tf_normalize(x):
  """Returns the input vector, normalized.

  A small number is added to the norm so that this function does not break when
  dealing with the zero vector (e.g. if the weights are zero-initialized).

  Args:
    x: the tensor to normalize
  """
  return x / (tf.norm(x) + np.finfo(np.float32).tiny)


class AdversarialEmbeddingModel(object):
  """A model for doing adversarial training of embedding models."""

  def __init__(self, client,
               data_p, embed_dim, projection,
               projection_dims, pred):
    """Creates a new AdversarialEmbeddingModel.

    Args:
      client: The (possibly biased) embeddings.
      data_p: Placeholder for the data.
      embed_dim: Number of dimensions used in the embeddings.
      projection: The space onto which we are "projecting".
      projection_dims: Number of dimensions of the projection.
      pred: Prediction layer.
    """
    # load the analogy vectors as well as the embeddings
    self.client = client
    self.data_p = data_p
    self.embed_dim = embed_dim
    self.projection = projection
    self.projection_dims = projection_dims
    self.pred = pred

  def nearest_neighbors(self, sess, in_arr,
                        k):
    """Finds the nearest neighbors to a vector.

    Args:
      sess: Session to use.
      in_arr: Vector to find nearest neighbors to.
      k: Number of nearest neighbors to return
    Returns:
      List of up to k pairs of (word, score).
    """
    v = sess.run(self.pred, feed_dict={self.data_p: in_arr})
    return self.client.similar_by_vector(v.flatten().astype(float), topn=k)

  def write_to_file(self, sess, f):
    """Writes a model to disk."""
    np.savetxt(f, sess.run(self.projection))

  def read_from_file(self, sess, f):
    """Reads a model from disk."""
    loaded_projection = np.loadtxt(f).reshape(
        [self.embed_dim, self.projection_dims])
    sess.run(self.projection.assign(loaded_projection))

  def fit(self,
          sess,
          data,
          data_p,
          labels,
          labels_p,
          protect,
          protect_p,
          gender_direction,
          pred_learning_rate,
          protect_learning_rate,
          protect_loss_weight,
          num_steps,
          batch_size,
          debug_interval=1000):
    """Trains a model.

    Args:
      sess: Session.
      data: Features for the training data.
      data_p: Placeholder for the features for the training data.
      labels: Labels for the training data.
      labels_p: Placeholder for the labels for the training data.
      protect: Protected variables.
      protect_p: Placeholder for the protected variables.
      gender_direction: The vector from find_gender_direction().
      pred_learning_rate: Learning rate for predicting labels.
      protect_learning_rate: Learning rate for protecting variables.
      protect_loss_weight: The constant 'alpha' found in
          debias_word_embeddings.ipynb.
      num_steps: Number of training steps.
      batch_size: Number of training examples in each step.
      debug_interval: Frequency at which to log performance metrics during
          training.
    """
    feed_dict = {
        data_p: data,
        labels_p: labels,
        protect_p: protect,
    }
    # define the prediction loss
    pred_loss = tf.losses.mean_squared_error(labels_p, self.pred)

    # compute the prediction of the protected variable.
    # The "trainable"/"not trainable" designations are for the predictor. The
    # adversary explicitly specifies its own list of weights to train.
    protect_weights = tf.get_variable(
        "protect_weights", [self.embed_dim, 1], trainable=False)
    protect_pred = tf.matmul(self.pred, protect_weights)
    protect_loss = tf.losses.mean_squared_error(protect_p, protect_pred)

    pred_opt = tf.train.AdamOptimizer(pred_learning_rate)
    protect_opt = tf.train.AdamOptimizer(protect_learning_rate)

    protect_grad = {v: g for (g, v) in pred_opt.compute_gradients(protect_loss)}
    pred_grad = []

    # applies the gradient expression found in the document linked
    # at the top of this file.
    for (g, v) in pred_opt.compute_gradients(pred_loss):
      unit_protect = tf_normalize(protect_grad[v])
      # the two lines below can be commented out to train without debiasing
      g -= tf.reduce_sum(g * unit_protect) * unit_protect
      g -= protect_loss_weight * protect_grad[v]
      pred_grad.append((g, v))
      pred_min = pred_opt.apply_gradients(pred_grad)

    # compute the loss of the protected variable prediction.
    protect_min = protect_opt.minimize(protect_loss, var_list=[protect_weights])

    sess.run(tf.global_variables_initializer())
    sess.run(tf.local_variables_initializer())
    step = 0
    while step < num_steps:
      # pick samples at random without replacement as a minibatch
      ids = np.random.choice(len(data), batch_size, False)
      data_s, labels_s, protect_s = data[ids], labels[ids], protect[ids]
      sgd_feed_dict = {
          data_p: data_s,
          labels_p: labels_s,
          protect_p: protect_s,
      }

      if not step % debug_interval:
        metrics = [pred_loss, protect_loss, self.projection]
        metrics_o = sess.run(metrics, feed_dict=feed_dict)
        pred_loss_o, protect_loss_o, proj_o = metrics_o
        # log stats every so often: number of steps that have passed,
        # prediction loss, adversary loss
        print("step: %d; pred_loss_o: %f; protect_loss_o: %f" % (step,
                     pred_loss_o, protect_loss_o))
        for i in range(proj_o.shape[1]):
          print("proj_o: %f; dot(proj_o, gender_direction): %f)" %
                       (np.linalg.norm(proj_o[:, i]),
                       np.dot(proj_o[:, i].flatten(), gender_direction)))
      sess.run([pred_min, protect_min], feed_dict=sgd_feed_dict)
      step += 1
      
def filter_analogies(analogies,
                     index_map):
  filtered_analogies = []
  for analogy in analogies:
    if filter(index_map.has_key, analogy) != analogy:
      print "at least one word missing for analogy: %s" % analogy
    else:
      filtered_analogies.append(map(index_map.get, analogy))
  return filtered_analogies

def make_data(
    analogies, embed,
    gender_direction):
  """Preps the training data.

  Args:
    analogies: a list of analogies
    embed: the embedding matrix from load_vectors
    gender_direction: the gender direction from find_gender_direction

  Returns:
    Three numpy arrays corresponding respectively to the input, output, and
    protected variables.
  """
  data = []
  labels = []
  protect = []
  for analogy in analogies:
    # the input is just the word embeddings of the first three words
    data.append(embed[analogy[:3]])
    # the output is just the word embeddings of the last word
    labels.append(embed[analogy[3]])
    # the protected variable is the gender component of the output embedding.
    # the extra pair of [] is so that the array has the right shape after
    # it is converted to a numpy array.
    protect.append([np.dot(embed[analogy[3]], gender_direction)])
  # Convert all three to numpy arrays, and return them.
  return tuple(map(np.array, (data, labels, protect)))<jupyter_output><empty_output><jupyter_text>Edit the training parameters below to experiment with different training runs.For example, try increasing the number of training steps to 50k.<jupyter_code># Edit the training parameters below to experiment with different training runs.
# For example, try 
pred_learning_rate = 2**-16
protect_learning_rate = 2**-16
protect_loss_weight = 1.0
num_steps = 10000
batch_size = 1000

embed_dim = 300
projection_dims = 1


sess = tf.InteractiveSession()
with tf.variable_scope('var_scope', reuse=tf.AUTO_REUSE):
    analogy_indices = filter_analogies(analogies, indices)

    data, labels, protect = make_data(analogy_indices, embed, gender_direction)
    data_p = tf.placeholder(tf.float32, shape=[None, 3, embed_dim], name="data")
    labels_p = tf.placeholder(tf.float32, shape=[None, embed_dim], name="labels")
    protect_p = tf.placeholder(tf.float32, shape=[None, 1], name="protect")

    # projection is the space onto which we are "projecting". By default, this is
    # one-dimensional, but this can be tuned by projection_dims
    projection = tf.get_variable("projection", [embed_dim, projection_dims])

    # build the prediction layer
    # pred is the simple computation of d = -a + b + c for a : b :: c : d
    pred = -data_p[:, 0, :] + data_p[:, 1, :] + data_p[:, 2, :]
    pred -= tf.matmul(tf.matmul(pred, projection), tf.transpose(projection))

    trained_model = AdversarialEmbeddingModel(
        client, data_p, embed_dim, projection, projection_dims, pred)

    trained_model.fit(sess, data, data_p, labels, labels_p, protect, protect_p, gender_direction,
              pred_learning_rate,
            protect_learning_rate, protect_loss_weight, num_steps, batch_size)<jupyter_output><empty_output><jupyter_text>Analogy generation using the embeddings with bias reduced by the adversarial modelLet's see how the model that has been trained to mitigate bias performs on the analogy task.As before, change "boss" to "friend" to see how those analogies have changed too.<jupyter_code># Parameters
A = "man"
B = "woman"
C = "boss"
NUM_ANALOGIES = 5

# Use a word embedding to compute an analogy
in_arr = []
for i, word in enumerate((A, B, C)):
  in_arr.append(client.word_vec(word))
in_arr = np.array([in_arr])

print_knn(client, sess.run(pred, feed_dict={data_p: in_arr}),
          NUM_ANALOGIES)<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup