How many distinct characters are there in a typical Chinese book? This post gives some great statistics on Chinese character frequency, but I'm curious about the total number of distinct characters that appear in a "typical" book. By "typical" I mean something like a popular modern novel: say, "The Three-Body Problem," or the translation of the first Harry Potter.

I'm guessing the number will be disturbingly high, much greater than 3000, which I think I've heard quoted as a good benchmark for reading fluently. If, as suggested in the first link, the top 3000 characters cover at most 99.4% of characters that appear in random text, that's depressingly low. It means if you know 3000 characters, then on every page (I just counted around 1000 characters/page in Harry Potter) there's a rough average of six characters you need to either look up or accept not knowing how to pronounce!
Counting distinct characters/words in chinese books is really easy, but what exactly does "typical" means?

You can solve it by python. 
Then every time you meet a "typical" book(txt,mobi,azw format etc), you can count it by yourself.

input


The txt file of The Three-Body Problem I:Remembrance of Earth's Past
The txt file of The Three-Body Problem II:The Dark Forest
The txt file of The Three-Body Problem III:Death's End


program



import sys
import string
from collections import defaultdict
file_loc="tb3.txt"
try:
    f = open(file_loc,"r",encoding='gbk')
    allstr=f.read()
except UnicodeDecodeError:
    f = open(file_loc,"r",encoding='utf-8')
    allstr=f.read()

#dict to count chars, default to zero
d=defaultdict(int) 
for i in allstr:
    d[i]+=1

#count how many dinstinct words
#import jieba
#for i in jieba.cut(allstr):
#    d[i]+=1

# one pass filter, you can just use this filter
for c in string.printable:
    d.pop(c,0)

# select chinese characters directly
unicode_ranges = [0x3007,0x3007],[0x3400,0x4DBF],[0x4E00,0x9FEF],[0x20000,0x2EBFF]
for i,key in list(enumerate(d)):
    if not any(map(lambda x:x[0]&lt;=ord(key)&lt;=x[1],unicode_ranges)):
        del d[key]
print(len(d))

# map word_count to list of chars
d2=defaultdict(list)
for key,val in d.items():
    d2[val].append(key)

#sort by val
for key,val in sorted(d2.items(),key=lambda x:x[0],reverse=True): 
    print(key,val)



chinese character reference

conclusion


three body problem 1: 2854
three body problem 2: 2986
three body problem 3: 3034


(welcome to add your count results of other books here)

3000 characters cover 100%!,

You can use sum(d.values()) to count the total #chinese-characters
in three body problem 1, you will find 2109 characters covers 99.393256% 
2370 chars(delete char that occurs only once) covers 99.7%,
In addition, 

You said


  there's a rough average of six characters you need to either look up or accept not knowing how to pronounce!


That's false in the 99.4% version, you can learn a meaning of character in this way. You are likely to meet the same char again in this book if you only know 2109 characters.

If you are insterested in details, see all the resultshere