I have a very large (2.5 GB) text file with Cyrillic characters in various encodings, including Windows-1251:
=D0=A0=D0=B2=D0=B8=D1=81=D1=8C =D0=B2 =D0=B0=D1=82=D0=B0=D0=BA=D1=83 =D0=BD= =D0=B0 =C2=AB=D0=9F=D0=B5=D1=80=D1=88=D0=B8=D0=BD=D0=B3=D0=B5=C2=BB
I have already tried .encode() and .decode() with various combinations of encodings, but I cannot get the text to be readable. I have also tried reading in binary mode.
with open('myfile.mbox', 'r') as f:
unreadable_str = f.readline()
unreadable_str.encode('WINDOWS-1251').decode('utf-8')
I thought it would encode the string into bytes using the Windows encoding and then give it back as readable Unicode, but instead, it always outputs the same string.
1 Answer
That data is encoded according to RFC 1522. The quopri module can be used to decode the data to bytes, which look like UTF-8-encoded data:
>>> s='''=D0=A0=D0=B2=D0=B8=D1=81=D1=8C =D0=B2 =D0=B0=D1=82=D0=B0=D0=BA=D1=83 =D0=BD= =D0=B0 =C2=AB=D0=9F=D0=B5=D1=80=D1=88=D0=B8=D0=BD=D0=B3=D0=B5=C2=BB'''
>>> quopri.decodestring(s).decode('utf8')
'Рвись в атаку н= а «Першинге»'
