Extract anonymized games out of a pgn I'm a big fan of pgn-extract, but I am open to other linux-friendly tools. I have a big pgn file (over 5 million games) and a set of ECO codes. My goal is to fetch all the games matching those ECO codes, but with removing or fuzzying the players names and ratings, the year played, and the game result.
I then plan to open a random game with Scid and analyze it, since removing the players' data and game result will help me avoid any bias.
So far Im using pgn-extract like this:
pgn-extract -t filter_for_naked --noresults --notags --nocomments --noduplicates -bl20 --output naked.pgn my.pgn
But Scid is having problems opening the file. As expected each game is just the set of moves (no headers at all) and I suspect Scid requires at least some headers.
Any advise on how to use pgn-extract or Scid to achieve this?
Thanks for your compliment about pgn-extract (I am its author). You might be able to make the task a little easier by adding --linelength 1000 to the argument list in order to get each game on a single line, then use sed to add an unknown result to the end of the game and re-process the game with pgn-extract to add an empty 7-tag roster. Finally, use sed again to edit the White and Black tags.
The resulting command would look something like the following, although sed experts might be able to clean it up a bit:
pgn-extract -t filter_for_naked --noresults --notags --nocomments --noduplicates -bl20 --linelength 1000 my.pgn | sed -e 's/..*/&amp; */' | pgn-extract -7 | sed -e 's/White &quot;?&quot;/White &quot;AnonW&quot;/;s/Black &quot;?&quot;/Black &quot;AnonB&quot;/' &gt; naked.pgn

You will get some error reports for games that end in mate having inconsistent results but you can hide those in a logfile.