<jupyter_start><jupyter_text>CORnet PredNet feature extractionIn this notebook we extract representations from pre-trained CORnet models. We also generate RDM submissions for fMRI and MEG data.<jupyter_code>from google.colab import drive
drive.mount('/content/gdrive',force_remount=True)
!wget -O algonauts.zip -q https://github.com/thefonseca/algonauts/archive/master.zip
!unzip -q algonauts.zip -d /content/
!mv algonauts-master/ algonauts-github
!mv algonauts-github/* /content/

!unzip -q gdrive/My\ Drive/algonauts/algonauts-master.zip -d /content/
!unzip -q gdrive/My\ Drive/algonauts/algonauts-dataset.zip -d /content/
!cp algonauts-master/Feature_Extract/create_RDMs.py /content/
#!cp prednet/*.py /content/
#!cp -r gdrive/My\ Drive/algonauts/prednet-models /content/<jupyter_output><empty_output><jupyter_text>Get CORnet code<jupyter_code>!wget -O cornet.zip -q https://github.com/dicarlolab/CORnet/archive/master.zip
!unzip -q cornet.zip -d /content/
!mkdir -p cornet-feats/92images/cornet-s
!mkdir -p cornet-feats/118images/cornet-s
!pip install fire<jupyter_output>[?25l
[K     |████▎                           | 10kB 19.1MB/s eta 0:00:01
[K     |████████▋                       | 20kB 1.6MB/s eta 0:00:01
[K     |████████████▉                   | 30kB 2.4MB/s eta 0:00:01
[K     |█████████████████▏              | 40kB 1.6MB/s eta 0:00:01
[K     |█████████████████████▍          | 51kB 2.0MB/s eta 0:00:01
[K     |█████████████████████████▊      | 61kB 2.4MB/s eta 0:00:01
[K     |██████████████████████████████  | 71kB 2.8MB/s eta 0:00:01
[K     |████████████████████████████████| 81kB 3.0MB/s 
[?25h  Building wheel for fire (setup.py) ... [?25l[?25hdone<jupyter_text>Small fixFor the feature extraction to work we need to edit the `run.py` file and change the `_store_feats` hook from:```def _store_feats(layer, inp, output):        """An ugly but effective way of accessing intermediate model features        """        _model_feats.append(np.reshape(output, (len(output), -1)).numpy())```to the following:```def _store_feats(layer, inp, output):        """An ugly but effective way of accessing intermediate model features        """        _model_feats.append(output.view(len(output), -1).cpu().numpy())```  Features for the 92 images dataset<jupyter_code>!cd /content/CORnet-master/ && python run.py test --model S --layer decoder \
--sublayer avgpool --data_path /content/Training_Data/92_Image_Set/92images/ \
-o /content/cornet-feats/92images/ --ngpus 1
!cd /content/CORnet-master/ && python run.py test --model S --layer V1 \
--sublayer output --data_path /content/Training_Data/92_Image_Set/92images/ \
-o /content/cornet-feats/92images/ --ngpus 1
!cd /content/CORnet-master/ && python run.py test --model S --layer V2 \
--sublayer output --data_path /content/Training_Data/92_Image_Set/92images/ \
-o /content/cornet-feats/92images/ --ngpus 1
!cd /content/CORnet-master/ && python run.py test --model S --layer V4 \
--sublayer output --data_path /content/Training_Data/92_Image_Set/92images/ \
-o /content/cornet-feats/92images/ --ngpus 1
!cd /content/CORnet-master/ && python run.py test --model S --layer IT \
--sublayer output --data_path /content/Training_Data/92_Image_Set/92images/ \
-o /content/cornet-feats/92images/ --ngpus 1<jupyter_output>100% 92/92 [00:00<00:00, 135.45it/s]<jupyter_text>Features for the 118 images dataset<jupyter_code>!cd /content/CORnet-master/ && python run.py test --model S --layer decoder \
--sublayer avgpool --data_path /content/Training_Data/118_Image_Set/118images/ \
-o /content/cornet-feats/118images/ --ngpus 1
!cd /content/CORnet-master/ && python run.py test --model S --layer V1 \
--sublayer output --data_path /content/Training_Data/118_Image_Set/118images/ \
-o /content/cornet-feats/118images/ --ngpus 1
!cd /content/CORnet-master/ && python run.py test --model S --layer V2 \
--sublayer output --data_path /content/Training_Data/118_Image_Set/118images/ \
-o /content/cornet-feats/118images/ --ngpus 1
!cd /content/CORnet-master/ && python run.py test --model S --layer V4 \
--sublayer output --data_path /content/Training_Data/118_Image_Set/118images/ \
-o /content/cornet-feats/118images/ --ngpus 1
!cd /content/CORnet-master/ && python run.py test --model S --layer IT \
--sublayer output --data_path /content/Training_Data/118_Image_Set/118images/ \
-o /content/cornet-feats/118images/ --ngpus 1
import numpy as np
np.load('/content/cornet-feats/92images/CORnet-S_V1_output_feats.npy').shape
!cp -r /content/cornet-feats/ /content/gdrive/My\ Drive/cornet-feats-$(date +%Y-%m-%dT%H:%M:%S)<jupyter_output><empty_output><jupyter_text>Generate RDMs<jupyter_code>import create_RDMs
import os
import glob
import scipy.io as sio
import zipfile


def create_rdm(save_dir, feat_dir, dist):
    """
    Main function to create RDM from activations
    Input:
    feat_dir: Directory containing activations generated using generate_features.py
    save_dir : directory to save the computed RDM
    dist : dist used for computing RDM (e.g. 1-Pearson's R)

    Output (in submission format):
    The model RDMs for each layer are saved in
        save_dir/layer_name/submit_fMRI.mat to compare with fMRI RDMs
        save_dir/layer_name/submit_MEG.mat to compare with MEG RDMs
    """

    # get list of layers and number of conditions(images) for RDM
    # layer_list, num_condns = get_layers_ncondns(feat_dir)
    feat_files = glob.glob(feat_dir + "/*feats.npy")
    
    layer_list = []
    for feat_file in feat_files:
        feat_file = os.path.basename(feat_file)
        layer_list.append(feat_file.replace('_feats.npy', ''))
        
    print(layer_list)
    cwd = os.getcwd() 
    print(save_dir, feat_dir, cwd)
            
    # loops over layers and create RDM for each layer
    for feat_file in feat_files:
        layer_id = os.path.basename(feat_file).replace('_feats.npy', '')
        feats = np.load(feat_file)
        num_condns = len(feats)
        
        os.chdir(cwd)
        # RDM is num_condnsxnum_condns matrix, initialized with zeros
        RDM = np.zeros((num_condns, num_condns))

        #save path for RDMs in  challenge submission format
        print(f'Processing layer {layer_id}...')
        
        
        RDM_dir = os.path.join(save_dir, layer_id)
        # print(RDM_dir, save_dir, os.path.join(save_dir, layer_id))
        if not os.path.exists(RDM_dir):
            os.makedirs(RDM_dir)
            
        RDM_filename_meg = os.path.join(RDM_dir,'submit_meg.mat')
        RDM_filename_fmri = os.path.join(RDM_dir,'submit_fmri.mat')
        RDM_filename_meg_zip = os.path.join(RDM_dir,'submit_meg.zip')
        RDM_filename_fmri_zip = os.path.join(RDM_dir,'submit_fmri.zip')
        #RDM loop
        for i in range(num_condns):
            for j in range(num_condns):
                #get feature for image index i and j
                feature_i = feats[i]
                feature_j = feats[j]
                
                # compute distance 1-Pearson's R
                if dist == 'pearson':
                    RDM[i,j] = 1-np.corrcoef(feature_i,feature_j)[0][1]
                else:
                    print("The", dist, "distance measure not implemented, please request through issues")

        #saving RDMs in challenge submission format
        rdm_fmri={}
        rdm_meg={}
        rdm_fmri['EVC_RDMs'] = RDM
        rdm_fmri['IT_RDMs'] = RDM
        rdm_meg['MEG_RDMs_late'] = RDM
        rdm_meg['MEG_RDMs_early'] = RDM
        sio.savemat(RDM_filename_fmri, rdm_fmri)
        sio.savemat(RDM_filename_meg, rdm_meg)

        #creating zipped file for submission
        zipfmri = zipfile.ZipFile(RDM_filename_fmri_zip, 'w')
        zipmeg = zipfile.ZipFile(RDM_filename_meg_zip, 'w')
        os.chdir(RDM_dir)
        zipfmri.write('submit_fmri.mat')
        zipmeg.write('submit_meg.mat')
        zipfmri.close()
        zipmeg.close()
        
    os.chdir(cwd)

feat_dir = '/content/cornet-feats/92images/'
rdm_dir = '/content/results/rdms/92images/pearson'
create_rdm(rdm_dir, feat_dir, 'pearson')
feat_dir = '/content/cornet-feats/118images/'
rdm_dir = '/content/results/rdms/118images/pearson'
create_rdm(rdm_dir, feat_dir, 'pearson')
!cp -r /content/results/rdms /content/gdrive/My\ Drive/cornet-rdms-$(date +%Y-%m-%dT%H:%M:%S)<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup