update

Update README.md (#99 )
old link doesn't work anymore.
2025-01-05 17:03:38 +08:00 · 2024-08-13 16:33:02 -07:00 · 2023-05-20 18:15:50 -07:00 · 2021-12-13 16:46:57 -08:00 · 2021-08-29 12:51:11 -07:00 · 2021-04-23 15:10:33 -07:00
113 changed files with 1551 additions and 896 deletions
@@ -3,3 +3,9 @@
 output/
 resource
 *.db
+*.db.decrypted
+*2015*
+*2017*
+2020*
+avatar.index
+emoji.cache
@@ -1,118 +1,114 @@
 ## Dump WeChat Messages from Android

-## 导出安卓微信聊天数据
+## 导出安卓微信消息记录

-WeChat(微信), as the most popular mobile IM app in China, doesn't give users any method to export well-formatted history message.
-This tool can parse and export WeChat messages on a rooted android phone.
+WeChat, as the most popular mobile IM app in China, doesn't provide any methods to export structured message history.

-Right now it can dump messages in text-only mode, or generate a single-file html containing voice messages, images, emoji, etc.
+We reverse-engineered the storage protocol of WeChat messages, and
+provide this tool to decrypt and parse WeChat messages on a rooted android phone.
+It can also render the messages into self-contained html files including voice messages, images, emojis, videos, etc.

-__NEWS__: WeChat 6.0+ uses silk to encode audio. The code is updated.
+The tool is last verified to work with latest version of wechat on 2025/01/01.
+If the tool works for you, please take a moment to add your phone/OS to [the wiki](https://github.com/ppwwyyxx/wechat-dump/wiki).

-__NEWS__: WeChat 6.3 uses a new avatar storage. The code is updated.
-
-If this tools works for you, please take a moment to __add your phone/OS to__ [the wiki](https://github.com/ppwwyyxx/wechat-dump/wiki).
-If it doesn't work, please leave an issue together with your phone/OS/wechat version.
-
-### How to use:
+## How to use:

 #### Dependencies:
-+ python-PIL
-+ [PyQuery](https://pypi.python.org/pypi/pyquery/1.2.1)
-+ [pysox](https://pypi.python.org/pypi/pysox/0.3.6.alpha)
-+ [pysqlcipher](https://pypi.python.org/pypi/pysqlcipher)
-+ numpy
-+ csscompressor (suggested, optional)
-+ adb and rooted android phone connected to a Linux/Mac OS.
-+ Silk audio decoder (included; just run `./third-party/compile_silk.sh`)
-+ gnu-sed
+ adb and rooted android phone connected to a Linux/Mac OSX/Win10+Bash.
+ Python >= 3.8
+ [sqlcipher](https://github.com/sqlcipher/sqlcipher) >= 4.1
+ sox (command line tools)
+ Silk audio decoder (included; build it with `./third-party/compile_silk.sh`)
+ Other python dependencies: `pip install -r requirements.txt`.

 #### Get Necessary Data:

-Note that commands involving `./android-interact.sh` are meant to be run on the computer.
+1. Pull database file and (for older wechat versions) avatar index:
+  + Automatic: `./android-interact.sh db`. It may use an incorrect userid.
+  + Manual:
+    + Figure out your `${userid}` by inspecting the contents of `/data/data/com.tencent.mm/MicroMsg` on the __root__ filesystem of the device.
+      It should be a 32-character-long name consisting of hexadecimal digits.
+    + Get `/data/data/com.tencent.mm/MicroMsg/${userid}/EnMicroMsg.db` from the device.
+2. Decrypt database file:
+  + Automatic: `./decrypt-db.py decrypt --input EnMicroMsg.db`
+  + Manual:
+    + Get WeChat uin (an integer), possible ways are:
+      + `./decrypt-db.py uin`, which looks for uin in `/data/data/com.tencent.mm/shared_prefs/`
+      + Login to [web wechat](https://wx.qq.com), get wxuin=1234567 from `document.cookie`
+    + Get your device id (a positive integer), possible ways are:
+      + `./decrypt-db.py imei` implements some ways to find device id.
+      + Call `*#06#` on your phone
+      + Find IMEI in system settings
+    + Decrypt database with combination of uin and device id:

-+ (Requires Linux or Mac) Get the decrypted WeChat database and the avatar index:
-	+ Automatic: `./android-interact.sh db-decrypt`
-		+ Requires rooted adb. If the OS distribution does not come with adb support, you can download an app such as https://play.google.com/store/apps/details?id=eu.chainfire.adbd
-	+ Manual:
-		+ Figure out your `${userid}` by inspecting the contents of `/data/data/com.tencent.mm/MicroMsg` on the __root__ filesystem of the device. It should be a 32-character-long name consisting of hexadecimal digits.
-		+ Get `/data/data/com.tencent.mm/MicroMsg/${userid}/{EnMicroMsg.db,sfs/avatar.index}` from the device, possible ways are:
-			+ `./android-interact.sh db`
-			+ Use your rooted file system manager app
-		+ Get WeChat uin (an integer), possible ways are:
-			+ `./android-interact.sh uin`, which pulls the value from `/data/data/com.tencent.mm/shared_prefs/system_config_prefs.xml`
-			+ Login to [web wechat](https://wx.qq.com), get wxuin=1234567 from `document.cookie`
-		+ Get your phone IMEI number (a positive integer), possible ways are:
-			+ `./android-interact.sh imei`
-			+ Call `*#06#` on your phone
-			+ Find IMEI in system settings
-		+ Decrypt database, will produce `decrypted.db`:
+      ```
+      ./decrypt-db.py decrypt --input EnMicroMsg.db --imei <device id> --uin <uin>
+      ```

-            ```
-            ./decrypt-db.py <path to EnMicroMsg.db> <imei> <uin>
-            ```
+      NOTE: you may need to try different ways to get device id and find one that can decrypt the
+      database. Some phones may have multiple IMEIs, you may need to try them all.
+      See [#33](https://github.com/ppwwyyxx/wechat-dump/issues/33).
+      The command will dump decrypted database at `EnMicroMsg.db.decrypted`.

-    NOTE: you may need to try different ways to getting imei & uin,
-    because things behave differently on different phones.
+  If the above decryption doesn't work, you can also try the [password cracker](https://github.com/chg-hou/EnMicroMsg.db-Password-Cracker)
+  to brute-force the key. The encryption key is not very strong.

-    Also, if the decryption doesn't work with pysqlcipher, maybe try the version of sqlcipher in `legacy`.
-
-
-+ Copy the WeChat user resource directory `/mnt/sdcard/tencent/MicroMsg/${userid}/{emoji,image2,sfs,video,voice2}` from the phone's SD card to the `resource` directory:
+3. Copy the WeChat user resource directory `/data/data/com.tencent.mm/MicroMsg/${userid}/{avatar,emoji,image2,sfs,video,voice2}` from the phone to the `resource` directory:
 	+ `./android-interact.sh res`
-	+ You might need to tweak `RES_DIR` in the script if the default doesn't work
-	+ This can take a __long__ time. Some ways to do this faster:
-        + If there's enough free space on the SD card, you can combine all the files via `busybox tar` without compression in the `adb shell`, use `adb pull` to copy the `tar` archive to the computer, and then extract it. BusyBox is needed as the Android system's `tar` may choke on long paths.
-        + Alternatively, you can use pipes.  This is slower, but doesn't require any free space on the SD card:
+	+ Change `RES_DIR` in the script if the location of these directories is different on your phone.
+      For older version of wechat, the directory may be `/mnt/sdcard/tencent/MicroMsg/`
+	+ This can take a while. It can be faster to first archive it with `tar` with or without compression, and then copy the archive,
+  	  `busybox tar` is recommended as the Android system's `tar` may choke on long paths.
+	+ In the end, we need a `resource` directory with the following subdir: `avatar,emoji,image2,sfs,video,voice2`.

-            ```sh
-            # copy MicroMsg to the current directory
-            adb shell 'cd /mnt/sdcard/tencent &&
-                       busybox tar czf - MicroMsg 2>/dev/null | busybox base64' |
-                base64 -di | tar xzf -
-            ```
+4. (Optional) Download the emoji cache from [here](https://github.com/ppwwyyxx/wechat-dump/releases/download/0.1/emoji.cache.tar.bz2)
+	and decompress it under `wechat-dump`. This will avoid downloading too many emojis during rendering.
+
+        wget -c https://github.com/ppwwyyxx/wechat-dump/releases/download/0.1/emoji.cache.tar.bz2
+        tar xf emoji.cache.tar.bz2

 #### Run:
-+ Parse and dump text messages of __every__ chat (requires `decrypted.db`):
+ Parse and dump text messages of __every__ chat (requires decrypted database):

    ```
    ./dump-msg.py decrypted.db output_dir
    ```

-+ List all chats (requires `decrypted.db`):
+ List all chats (required decrypted database):

    ```
    ./list-chats.py decrypted.db
    ```

-+ Generate statistical report on text messages (requires `output_dir` from `./dump-msg.py`):
+ Generate statistics report on text messages (requires `output_dir` from `./dump-msg.py`):

    ```
    ./count-message.sh output_dir
    ```

-+ Dump messages of one contact to html, containing voice messages, emojis, and images (requires `decrypted.db`, `avatar.index`, and `resource`):
+ Dump messages of one contact to html, containing voice messages, emojis, and images (requires decrypted database and `resource`):

    ```
-    ./dump-html.py decrypted.db avatar.index resource "<contact_name>" output.html
+    ./dump-html.py "<contact_display_name>"
    ```

+    The output file is `output.html`.
+
+    Check `./dump-html.py -h` to use different paths.
+
 ### Examples:
-See [here](http://ppwwyyxx.com/static/wechat/example.html) for an example html.
-
 Screenshots of generated html:

 ![byvoid](https://github.com/ppwwyyxx/wechat-dump/raw/master/screenshots/byvoid.jpg)

-### TODO List
-+ Search by uid/username
-+ Faster way to copy a directory from android (I don't know..).
-+ Fix rare unhandled types: > 10000 and < 0
-+ Better user experiences... see `grep 'TODO' wechat -R`
-+ more easy-to-use for non-programmers (GUI?)
+See [here](http://ppwwyyxx.com/static/wechat/example.html) for an example html.
+
+### TODO List (help needed!)
+* __IMPORTANT__ Some emojis and chat images are stored in a proprietary "wxgf" format. We don't yet know how to decode this format.
+* Fix rare unhandled message types: > 10000 and < 0
+* Better user experiences... see `grep 'TODO' wechat -R`

 ### Donate!
-Paypal:
 <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_donations&business=7BC299GRDLEDU&lc=US&item_name=wechat%2ddump&item_number=wechat%2ddump&currency_code=USD&bn=PP%2dDonationsBF%3abtn_donate_SM%2egif%3aNonHosted">
 <img src="https://img.shields.io/badge/Paypal-Buy%20a%20Drink-blue.svg" alt="[paypal]" />
 </a>
@@ -1,48 +1,27 @@
 #!/bin/bash
 # File: android-interact.sh
-# Date: Fri Jun 26 10:38:07 2015 +0800
-# Author: Yuxin Wu <[email protected]>

 PROG_NAME=`python -c "import os, sys; print(os.path.realpath(sys.argv[1]))" "$0"`
 PROG_DIR=`dirname "$PROG_NAME"`
 cd "$PROG_DIR"

-source compatibility.sh
-
 # Please check that your path is the same, since this might be different among devices
-RES_DIR="/mnt/sdcard/tencent/MicroMsg"
+# RES_DIR="/mnt/sdcard/tencent/MicroMsg"  # old version of wechat use this path.
+RES_DIR="/data/data/com.tencent.mm"
 MM_DIR="/data/data/com.tencent.mm"

 echo "Starting rooted adb server..."
 adb root

-if [[ $1 == "uin" ]]; then
-	adb pull $MM_DIR/shared_prefs/system_config_prefs.xml 2>/dev/null
-	uin=$($GREP 'default_uin' system_config_prefs.xml | $GREP -o 'value="\-?[0-9]*' | cut -c 8-)
-	[[ -n $uin ]] || {
-		>&2 echo "Failed to get wechat uin. You can try other methods, or report a bug."
-		exit 1
-	}
-	rm system_config_prefs.xml
-	echo "Got wechat uin: $uin"
-elif [[ $1 == "imei" ]]; then
-	imei=$(adb shell dumpsys iphonesubinfo | $GREP 'Device ID' | $GREP -o '[0-9]+')
-	[[ -n $imei ]] || {
-		imei=$(adb shell service call iphonesubinfo 1 | awk -F "'" '{print $2}' | sed 's/[^0-9A-F]*//g' | tr -d '\n')
-	}
-	[[ -n $imei ]] || {
-		>&2 echo "Failed to get imei. You can try other methods, or report a bug."
-		exit 1
-	}
-	echo "Got imei: $imei"
-elif [[ $1 == "db" || $1 == "res" ]]; then
+if [[ $1 == "db" || $1 == "res" ]]; then
 	echo "Looking for user dir name..."
-	sleep 1	# sometimes adb complains: device not found
-	userList=$(adb ls $RES_DIR | cut -f 4 -d ' ' \
+	sleep 1  	# sometimes adb complains: device not found
+	# look for dirname which looks like md5 (32 alpha-numeric chars)
+	userList=$(adb ls $RES_DIR | cut -f 4 -d ' ' | sed 's/[^0-9a-z]//g' \
 		| awk '{if (length() == 32) print}')
-	numUser=$(echo $userList | wc -l)
+	numUser=$(echo "$userList" | wc -l)
 	# choose the first user.
-	chooseUser=$(echo $userList | head -n1)
+	chooseUser=$(echo "$userList" | head -n1)
 	[[ -n $chooseUser ]] || {
 		>&2 echo "Could not find user. Please check whether your resource dir is $RES_DIR"
 		exit 1
@@ -50,54 +29,49 @@ elif [[ $1 == "db" || $1 == "res" ]]; then
 	echo "Found $numUser user(s). User chosen: $chooseUser"

 	if [[ $1 == "res" ]]; then
-		echo "Pulling resources... this might take a long time, because adb sucks..."
-		mkdir -p resource; cd resource
-		for d in image2 voice2 emoji video sfs; do
-			mkdir -p $d; cd $d
-			adb pull $RES_DIR/$chooseUser/$d
-			cd ..
-			[[ -d $d ]] || {
-				>&2 echo "Failed to download resource directory: $RES_DIR/$chooseUser/$d"
-				exit 1
-			}
-		done
-		cd ..
-		echo "Resource pulled at ./resource"
-		echo "Total size: $(du -sh resource | cut -f1)"
+		mkdir -p resource
+    (
+      cd resource || exit
+      echo "Pulling resources... "
+      for d in avatar image2 voice2 emoji video sfs; do
+        echo "Trying to download $RES_DIR/$chooseUser/$d with busybox ..."
+        adb shell "cd $RES_DIR/$chooseUser &&
+                   busybox tar czf - $d 2>/dev/null | busybox base64" |
+            base64 -di | tar xzf -
+        [[ -d $d ]] && continue
+
+        echo "Trying to download $RES_DIR/$chooseUser/$d with tar & base64 ..."
+        adb shell "cd $RES_DIR/$chooseUser &&
+                   tar czf - $d 2>/dev/null | base64" | base64 -di | tar xzf -
+        [[ -d $d ]] && continue
+
+        echo "Trying to download $RES_DIR/$chooseUser/$d with adb pull (slow) ..."
+        mkdir -p $d
+        (
+          cd $d || exit
+          adb pull "$RES_DIR/$chooseUser/$d"
+        )
+
+        [[ -d $d ]] || {
+          echo "Failed to download $RES_DIR/$chooseUser/$d"
+        }
+      done
+      echo "Resource pulled at ./resource"
+      echo "Total size: $(du -sh | cut -f1)"
+    )
 	else
 		echo "Pulling database and avatar index file..."
 		adb pull $MM_DIR/MicroMsg/$chooseUser/EnMicroMsg.db
 		[[ -f EnMicroMsg.db ]] && \
 			echo "Database successfully downloaded to EnMicroMsg.db" || {
-			>&2 echo "Failed to pull database by adb"
+			>&2 echo "Failed to pull database by adb!"
 			exit 1
 		}
 		adb pull $MM_DIR/MicroMsg/$chooseUser/sfs/avatar.index
-		[[ -f avatar.index ]] && \
-			echo "Avatar index successfully downloaded to avatar.index" || {
-			>&2 echo "Failed to pull avatar index by adb, are you using latest version of wechat?"
-			exit 1
-		}
+		[[ -f avatar.index ]] && echo "Avatar index successfully downloaded to avatar.index"
 	fi
-elif [[ $1 == "db-decrypt" ]]; then
-	echo "Getting uin..."
-	$0 uin | tail -n1 | $GREP -o '\-?[0-9]*' | tee /tmp/uin
-	echo "Getting imei..."
-	$0 imei | tail -n1 | $GREP -o '[0-9]*' | tee /tmp/imei
-	echo "Getting db..."
-	$0 db
-	echo "Decrypting db..."
-	imei=$(cat /tmp/imei)
-	uin=$(cat /tmp/uin)
-	if [[ -z $imei || -z $uin ]]; then
-		>&2 echo "Failed to get imei or uin. See README for manual methods."
-		exit 1
-	fi
-	./decrypt-db.py EnMicroMsg.db $imei $uin
-	rm /tmp/{uin,imei}
-	echo "Done. See decrypted.db"
 else
-	echo "Usage: $0 <res|db-decrypt>"
+	echo "Usage: $0 <res|db>"
 	exit 1
 fi

@@ -1,13 +0,0 @@
-#!/bin/bash -e
-# $File: compatibility.sh
-# $Date: Tue Jun 16 22:23:36 2015 +0800
-# Author: Vury Leo <i[at]vuryleo[dot]com>
-
-if [ `uname` = 'Darwin' ]; then
-  GREP='grep -E'
-  MD5SUM='md5'
-else
-  GREP='grep -E'
-  MD5SUM='md5sum'
-fi
-
@@ -1,6 +1,6 @@
 #!/bin/bash -e
 # File: count-message.sh
-# Date: Sun Apr 12 21:01:01 2015 +0900
+# Date: Wed Nov 29 02:32:40 2017 -0800
 # Author: Kangjing Huang <[email protected]>


@@ -11,7 +11,7 @@ then
 fi
 # TODO work on db directly

-echo -e "Filename\tCounts of message\tCounts of chars\tCounts of words"
+echo -e "Filename\t#Lines\t#Chars\t#Words"

 SAVEIFS=$IFS
 IFS=$(echo -en "\n\b")
@@ -23,6 +23,6 @@ do
    CHARCOUNT=$(cat "$i"| sed 's/.*:[0-9][0-9]:\(.*\)/\1/g'  | sed 's/\[.*\]//g'  | grep  -v img | wc -m)
    WORDCOUNT=$(cat "$i"| sed 's/.*:[0-9][0-9]:\(.*\)/\1/g'  | sed 's/\[.*\]//g'  | grep  -v img | wc -w)
    echo -e "$LINECOUNT\t$CHARCOUNT\t$WORDCOUNT"
-done
+done | sort -t $'\t' -k 2 -n | column -t -s $'\t'

 IFS=$SAVEIFS
@@ -1,56 +1,194 @@
-#!/usr/bin/env python2
-# -*- coding: UTF-8 -*-
-# File: decrypt-db.py
-# Author: Yuxin Wu <[email protected]>
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-

-from argparse import ArgumentParser
-from pysqlcipher import dbapi2 as sqlite
-
-from hashlib import md5
-import sys
 import os
+import shlex
+import sys
+import re
+import struct
+import argparse
+import logging
+import javaobj
+from pyquery import PyQuery
+from pysqlcipher3 import dbapi2 as sqlite
+from hashlib import md5
+
+from wechat.common.procutil import subproc_succ, subproc_call
+
+
+logger = logging.getLogger("wechat")
+
+RES_DIR = "/mnt/sdcard/tencent/MicroMsg"
+MM_DIR = "/data/data/com.tencent.mm"
+
+
+def adb_command(command):
+    return subproc_succ("adb shell su -c " + shlex.quote(command))
+
+
+def get_uin():
+    candidates = []
+    try:
+        uin = None
+        out = adb_command(f"cat {MM_DIR}/shared_prefs/system_config_prefs.xml")
+        for line in out.decode('utf-8').split("\n"):
+            if "default_uin" in line:
+                line = PyQuery(line)
+                uin = line.attr["value"]
+                break
+        uin = int(uin)
+    except Exception:
+        logger.warning("uin not found in system_config_prefs.xml")
+    else:
+        candidates.append(uin)
+        logger.info(f"found uin={uin} in system_config_prefs.xml")
+
+    try:
+        uin = None
+        out = adb_command(f"cat {MM_DIR}/shared_prefs/com.tencent.mm_preferences.xml")
+        for line in out.decode('utf-8').split("\n"):
+            if "last_login_uin" in line:
+                line = PyQuery(line)
+                uin = line.text()
+                break
+        uin = int(uin)
+    except Exception:
+        logger.warning("uin not found in com.tencent.mm_preferences.xml")
+    else:
+        candidates.append(uin)
+        logger.info(f"found uin={uin} in com.tencent.mm_preferences.xml")
+
+    try:
+        uin = None
+        out = adb_command(f"cat {MM_DIR}/shared_prefs/auth_info_key_prefs.xml")
+        for line in out.decode('utf-8').split("\n"):
+            if "auth_uin" in line:
+                line = PyQuery(line)
+                uin = line.attr["value"]
+                break
+        uin = int(uin)
+    except Exception:
+        logger.warning("uin not found in auth_info_key_prefs.xml")
+    else:
+        candidates.append(uin)
+        logger.info(f"found uin={uin} in auth_info_key_prefs.xml")
+
+    try:
+        out = adb_command(f"cat {MM_DIR}/MicroMsg/systemInfo.cfg")
+        uin = int(javaobj.loads(out).get(1, 0))
+    except:
+        logger.warning("default uin not found in systemInfo.cfg")
+    else:
+        candidates.append(uin)
+        logger.info(f"found uin={uin} in systemInfo.cfg")
+
+    candidates = list({str(x) if isinstance(x, int) else x
+                      for x in candidates if x != 0})
+    logger.info(f"Possible uin: {candidates}")
+    return candidates
+
+
+def get_imei():
+    candidates = []
+
+    class Parcel(object):
+        # https://gist.github.com/ktnr74/60ac7bcc2cd17b43f2cb
+        def __init__(self, text):
+            if text.startswith(b'Result: Parcel(') and text.endswith(b'\')'):
+                self.data = b''.join([struct.pack('<L', int(x, 16)) for x in re.findall(b'([0-9a-f]{8}) ', text)])
+                self.resultcode = self.get_int(0)
+            else:
+                raise Exception('Unexpected input!')
+
+        def get_int(self, offset=4):
+            return int(struct.unpack('<L', self.data[offset:offset+4])[0])
+
+        def get_utf16(self, offset=4):
+            return (self.data[offset + 4: offset+4+self.get_int(offset) * 2]).decode('utf-16')
+
+    out = adb_command(f"service call iphonesubinfo 1")
+    imei = Parcel(out.strip()).get_utf16()
+    logger.info(f"found imei={imei} from iphonesubinfo")
+    candidates.append(imei)
+
+    try:
+        out = adb_command(f"cat {MM_DIR}/MicroMsg/CompatibleInfo.cfg")
+        # https://gist.github.com/ChiChou/36556fd412a9e3216abecf06e084e4d9
+        jobj = javaobj.loads(out)
+        imei = jobj[258]
+    except:
+        logger.warning("imei not found in CompatibleInfo.cfg")
+    else:
+        candidates.append(imei)
+        logger.info(f"found imei={imei} in CompatibleInfo.cfg")
+    candidates.append("1234567890ABCDEF")  # https://github.com/ppwwyyxx/wechat-dump/issues/70
+    logger.info(f"Possible imei: {candidates}")
+    return list(set(candidates))

-DEFAULT_OUTPUT_DB_NAME = 'decrypted.db'
-def get_args():
-    parser = ArgumentParser()
-    parser.add_argument('db', help='path to EnMicroMsg.db')
-    parser.add_argument('imei', help='15 digit IMEI of your phone')
-    parser.add_argument('uin', help='WeChat UIN')
-    parser.add_argument('--output', help='output decrypted database',
-                        default=DEFAULT_OUTPUT_DB_NAME)
-    args = parser.parse_args()
-    return args

 def get_key(imei, uin):
+    """
+    Args:
+        imei, uin: str
+    """
+    if isinstance(uin, str):
+        uin = uin.encode('ascii')
+    if isinstance(imei, str):
+        imei = imei.encode('ascii')
    a = md5(imei + uin)
    return a.hexdigest()[:7]

-if __name__ == '__main__':
-    args = get_args()

-    output = args.output
-    if os.path.abspath(os.path.dirname(output)) != os.path.abspath('.'):
-        print "Output file must be in current directory"
-        sys.exit(1)
-    if os.path.isfile(output):
-        print "{} already exists. Remove? (y/n)".format(args.output),
-        ans = raw_input()
-        if ans not in ['y', 'Y']:
-            print "Bye!"
-            sys.exit()
-        os.unlink(output)
-
-    key = get_key(args.imei, args.uin)
-    print "KEY: {}".format(key)
-
-    print "Decrypt and dump database to {} ... ".format(output)
-    conn = sqlite.connect(args.db)
+def do_decrypt(input, output, key):
+    conn = sqlite.connect(input)
    c = conn.cursor()
+    version_str = list(conn.execute("PRAGMA cipher_version"))[0][0]
+    version = tuple([int(x) for x in version_str.split(".")[:2]])
+    assert version >= (4, 1), "Sqlcipher>=4.1 is required"
+
    c.execute("PRAGMA key = '" + key + "';")
-    c.execute("PRAGMA cipher_use_hmac = OFF;")
-    c.execute("PRAGMA cipher_page_size = 1024;")
-    c.execute("PRAGMA kdf_iter = 4000;")
-    c.execute("ATTACH DATABASE '" + args.output + "' AS db KEY '';")
+    # https://github.com/sqlcipher/sqlcipher/commit/e4b66d6cc8a2b7547a32ff2c3ac52f148eba3516
+    c.execute("PRAGMA cipher_compatibility = 1;")
+    try:
+        c.execute("ATTACH DATABASE '" + output + "' AS db KEY '';")
+    except Exception as e:
+        logger.error(f"Decryption failed: '{e}'")
+        os.unlink(output)
+        raise
+    logger.info(f"Decryption succeeded! Writing database to {output} ...")
    c.execute("SELECT sqlcipher_export('db');" )
    c.execute("DETACH DATABASE db;" )
    c.close()
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument('task', choices=['uin', 'imei', 'decrypt'])
+    parser.add_argument('--imei', help='overwrite imei')
+    parser.add_argument('--uin', help='overwrite uin')
+    parser.add_argument('--input', help='encrypted EnMicroMsg.db')
+    args = parser.parse_args()
+
+    subproc_succ("adb root")
+
+    if args.task == 'uin':
+        uin = get_uin()
+    elif args.task == 'imei':
+        imei = get_imei()
+    elif args.task == 'decrypt':
+        uins = [args.uin] if args.uin else get_uin()
+        imeis = [args.imei] if args.imei else get_imei()
+        output_file = args.input + ".decrypted"
+        assert not os.path.isfile(output_file), f"Output {output_file} exists!"
+        for uin in uins:
+            for imei in imeis:
+                key = get_key(imei, uin)
+                logger.info(f"Trying key {key} ...")
+                try:
+                    do_decrypt(args.input, output_file, key)
+                except Exception as error:
+                    logger.warning(f"An unexpected error occurred.{error}")
+                    pass
+                else:
+                    logger.info(f"Database dumped to {output_file}")
+                    sys.exit()
@@ -1,16 +1,13 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
-# File: dump-audio.py
-# Author: Yuxin Wu <[email protected]>

 import sys
+import base64
 import argparse

-from common.textutil import ensure_unicode
 from wechat.parser import WeChatDBParser
+from wechat.msg import TYPE_SPEAK
 from wechat.res import Resource
-from wechat.render import HTMLRender
-from wechat.libchathelper import LibChatHelper

 def get_args():
    parser = argparse.ArgumentParser()
@@ -23,27 +20,25 @@ def get_args():

 if __name__ == '__main__':
    args = get_args()
-
-    name = ensure_unicode(args.name)
-    output_file = args.output
-
    parser = WeChatDBParser(args.db)
-    res = Resource(args.res, '')
+    res = Resource(parser, args.res, '')

-    if name and name in parser.msgs_by_chat:
-        msgs = parser.msgs_by_chat[name]
-    else:
-        sys.stderr.write(u"Valid Contacts: {}\n".format(u'\n'.join(parser.msgs_by_chat.keys())))
-        sys.stderr.write(u"Couldn't find that contact {}.".format(name));
+    try:
+        chatid = parser.get_chat_id(args.name)
+    except KeyError:
+        sys.stderr.write(u"Valid Contacts: {}\n".format('\n'.join(parser.all_chat_nicknames)))
+        sys.stderr.write(u"Couldn't find the chat {}.".format(args.name));
        sys.exit(1)
-    print "Number of Messages: ", len(msgs)
+
+    msgs = parser.msgs_by_chat[chatid]
+    print(f"Number of Messages for {args.name}: ", len(msgs))
    assert len(msgs) > 0

-    libchat = LibChatHelper(parser, res)
-    msgs = libchat.convert_msgs(msgs)
-    voices = [m.sound for m in msgs if m.sound]
-    for idx, v in enumerate(voices):
-        p = v.find(':')
-        v = v[p:]
-        with open('/{}/{:04d}.mp3'.format(args.output, idx), 'wb') as f:
-            f.write(v)
+    voice_msgs = [m for m in msgs if m.type == TYPE_SPEAK]
+    for idx, m in enumerate(voice_msgs):
+        audio_str, duration = res.get_voice_mp3(m.imgPath)
+        audio_bytes = base64.b64decode(audio_str)
+        outf = f'/{args.output}/{idx:04d}-{duration:.1f}s.mp3'
+        with open(outf, 'wb') as f:
+            f.write(audio_bytes)
+        print(f"Audio written to {outf}")
@@ -1,23 +1,23 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: UTF-8 -*-
-# File: dump-html.py
-# Date: Wed Mar 25 17:44:20 2015 +0800
-# Author: Yuxin Wu <[email protected]>
-
+import os
 import sys
 import argparse
+import logging

-from common.textutil import ensure_unicode
 from wechat.parser import WeChatDBParser
 from wechat.res import Resource
+from wechat.common.textutil import ensure_unicode
 from wechat.render import HTMLRender

+logger = logging.getLogger("wechat")
+
 def get_args():
    parser = argparse.ArgumentParser()
    parser.add_argument('name', help='name of contact')
    parser.add_argument('--output', help='output html file', default='output.html')
    parser.add_argument('--db', default='decrypted.db', help='path to decrypted database')
-    parser.add_argument('--avt', default='avatar.index', help='path to avatar.index file')
+    parser.add_argument('--avt', default='avatar.index', help='path to avatar.index file that only exists in old version of wechat')
    parser.add_argument('--res', default='resource', help='reseource directory')
    args = parser.parse_args()
    return args
@@ -29,24 +29,31 @@ if __name__ == '__main__':
    output_file = args.output

    parser = WeChatDBParser(args.db)
-    res = Resource(parser, args.res, args.avt)

-    if name and name in parser.msgs_by_chat:
-        msgs = parser.msgs_by_chat[name]
-    else:
-        sys.stderr.write(u"Valid Contacts: {}\n".format(u'\n'.join(parser.msgs_by_chat.keys())))
-        sys.stderr.write(u"Couldn't find that contact {}.".format(name));
+    try:
+        chatid = parser.get_chat_id(args.name)
+    except KeyError:
+        sys.stderr.write(u"Valid Contacts: {}\n".format(
+            u'\n'.join(parser.all_chat_nicknames)))
+        sys.stderr.write(u"Couldn't find the chat {}.".format(name));
        sys.exit(1)
-    print "Number of Messages: ", len(msgs)
+
+    res = Resource(parser, args.res, args.avt)
+    msgs = parser.msgs_by_chat[chatid]
+    logger.info(f"Number of Messages for chatid {chatid}: {len(msgs)}")
    assert len(msgs) > 0

    render = HTMLRender(parser, res)
    htmls = render.render_msgs(msgs)

+    os.makedirs(os.path.dirname(os.path.abspath(output_file)), exist_ok=True)
    if len(htmls) == 1:
        with open(output_file, 'w') as f:
-            print >> f, htmls[0].encode('utf-8')
+            f.write(htmls[0])
    else:
+        assert output_file.endswith(".html")
+        basename = output_file[:-5]
        for idx, html in enumerate(htmls):
-            with open(output_file + '.{}'.format(idx), 'w') as f:
-                print >> f, html.encode('utf-8')
+            with open(basename + f'{idx:02d}.html', 'w') as f:
+                f.write(html)
+    res.emoji_reader.flush_cache()
@@ -1,30 +1,40 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: UTF-8 -*-
-# File: dump-msg.py
-# Date: Mon May 25 15:23:05 2015 +0800
-# Author: Yuxin Wu <[email protected]>

+import logging
 from wechat.parser import WeChatDBParser
-from common.textutil import safe_filename
+from wechat.common.textutil import safe_filename
 import sys, os

-if len(sys.argv) != 3:
-    sys.exit("Usage: {0} <path to decrypted_database.db> <output_dir>".format(sys.argv[0]))
+logger = logging.getLogger("wechat")

-db_file = sys.argv[1]
-output_dir = sys.argv[2]
-try:
-    os.mkdir(output_dir)
-except:
-    pass
-if not os.path.isdir(output_dir):
-    sys.exit("Error creating directory {}".format(output_dir))
+if __name__ == '__main__':
+    if len(sys.argv) != 3:
+        sys.exit("Usage: {0} <path to decrypted_database.db> <output_dir>".format(sys.argv[0]))

-parser = WeChatDBParser(db_file)
+    db_file = sys.argv[1]
+    output_dir = sys.argv[2]
+    try:
+        os.mkdir(output_dir)
+    except:
+        pass
+    if not os.path.isdir(output_dir):
+        sys.exit("Error creating directory {}".format(output_dir))

-for name, msgs in parser.msgs_by_chat.iteritems():
-    print u"Writing msgs for {}".format(name)
-    safe_name = safe_filename(name)
-    with open(os.path.join(output_dir, safe_name + '.txt'), 'w') as f:
-        for m in msgs:
-            print >> f, m
+    parser = WeChatDBParser(db_file)
+
+    for chatid, msgs in parser.msgs_by_chat.items():
+        name = parser.contacts[chatid]
+        if len(name) == 0:
+            logger.info(f"Chat {chatid} doesn't have a valid display name.")
+            name = str(id(chatid))
+        logger.info(f"Writing msgs for {name}")
+        safe_name = safe_filename(name)
+        outf = os.path.join(output_dir, safe_name + '.txt')
+        if os.path.isfile(outf):
+            logger.info(f"File {outf} exists! Skip contact {name}")
+            continue
+        with open(outf, 'w') as f:
+            for m in msgs:
+                f.write(str(m))
+                f.write("\n")
@@ -0,0 +1,46 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+
+import pickle
+import sys
+import os
+import imghdr
+import base64
+
+if __name__ == '__main__':
+    if len(sys.argv) != 3:
+        print("""\
+Usage:
+ {} unpack output-dir
+ {} pack input-dir
+""".format(sys.argv[0], sys.argv[0]))
+        sys.exit(1)
+
+    if sys.argv[1] == 'unpack':
+        with open('emoji.cache', 'rb') as f:
+            dic = pickle.load(f)
+        outdir = sys.argv[2]
+        assert os.path.isdir(outdir)
+        for md5, img in dic.items():
+            data = img[0]
+            if not isinstance(data, bytes):
+                data = data.encode('ascii')
+            name = os.path.join(outdir, md5 + '.' + img[1].lower())
+            print(name)
+            with open(name, 'wb') as f:
+                f.write(base64.decodebytes(data))
+    elif sys.argv[1] == 'pack':
+        ret = {}
+        indir = sys.argv[2]
+        files = os.listdir(indir)
+        for fname in files:
+            try:
+                md5, format = fname.split('.')
+            except:
+                print("Unable to parse", fname)
+                continue
+            with open(os.path.join(indir, fname), 'rb') as f:
+                b64 = base64.encodebytes(f.read()).decode('ascii')
+            ret[md5] = (b64, format)
+        with open('emoji.cache', 'wb') as f:
+            pickle.dump(ret, f)
@@ -1 +0,0 @@
-../compatibility.sh
@@ -0,0 +1,13 @@
+#!/bin/bash -e
+# $File: compatibility.sh
+# $Date: Tue Jun 16 22:23:36 2015 +0800
+# Author: Vury Leo <i[at]vuryleo[dot]com>
+
+if [ `uname` = 'Darwin' ]; then
+  GREP='grep -E'
+  MD5SUM='md5'
+else
+  GREP='grep -E'
+  MD5SUM='md5sum'
+fi
+
@@ -1,7 +1,6 @@
 #!/bin/bash -e
 # File: decrypt-db.sh
 # Date: Tue Jun 16 22:23:13 2015 +0800
-# Author: Yuxin Wu <[email protected]>

 source compatibility.sh

@@ -21,24 +20,15 @@ if [[ -f $output ]]; then
 	[[ $r == "y" ]] && rm -v $output || exit 1
 fi

-
 KEY=$(echo -n "$imei$uin" | $MD5SUM | cut -b 1-7)
 echo "KEY: $KEY"

-uname | grep Darwin > /dev/null && os=darwin || os=linux
-uname -m | grep x86_64 > /dev/null && version=64bit || version=32bit
-echo "Use $version sqlcipher of $os."
+echo "Dump decrypted database... "

-echo "Dump decrypted database... (Don't worry about libcrypt.so version warning.)"
-
-
-SQLCIPHER=./sqlcipher/$os/$version
-export LD_LIBRARY_PATH=$SQLCIPHER
-"$SQLCIPHER"/sqlcipher "$MSGDB" << EOF
+# https://github.com/sqlcipher/sqlcipher/commit/e4b66d6cc8a2b7547a32ff2c3ac52f148eba3516
+sqlcipher "$MSGDB" << EOF
 PRAGMA key='$KEY';
-PRAGMA cipher_use_hmac = off;
-PRAGMA cipher_page_size = 1024;
-PRAGMA kdf_iter = 4000;
+PRAGMA cipher_compatibility = 1;
 ATTACH DATABASE "$output" AS db KEY "";
 SELECT sqlcipher_export("db");
 DETACH DATABASE db;
@@ -1,8 +1,8 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: UTF-8 -*-
 # File: create_table.py
 # Date: Wed Mar 25 16:43:22 2015 +0800
-# Author: Yuxin Wu <[email protected]>
+# Author: Yuxin Wu

 import sys
 import os
@@ -10,7 +10,7 @@ import os
 from libchat import SqliteLibChat

 if len(sys.argv) != 2:
-    print "Usage: {} <DB file name>"
+    print("Usage: {} <DB file name>")
    sys.exit()

 db_name = sys.argv[1]
@@ -1,8 +1,5 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: UTF-8 -*-
-# File: libchat.py
-# Date: Sun Apr 12 21:08:51 2015 +0900
-# Author: Yuxin Wu <[email protected]>
 import sqlite3
 import os
 from datetime import datetime
@@ -99,7 +96,7 @@ class SqliteLibChat(object):
        else:
            self.c.execute("SELECT * FROM message WHERE {}".format(
                ' AND '.join(["{} = {}".format(k, v)
-                              for k, v in predicate.iteritems()])))
+                              for k, v in predicate.items()])))
        for row in self.c.fetchall():
            yield ChatMsg(*SqliteLibChat.postfilter(row))

@@ -113,5 +110,5 @@ if __name__ == '__main__':

    for k in db.iterate_all_msg():
        from IPython import embed; embed()
-        print k
+        print(k)

@@ -1,8 +1,4 @@
-#!/usr/bin/env python2
 # -*- coding: UTF-8 -*-
-# File: libchathelper.py
-# Date: Thu Jun 18 00:02:35 2015 +0800
-# Author: Yuxin Wu <[email protected]>

 import base64
 from pyquery import PyQuery
@@ -12,8 +8,8 @@ logger = logging.getLogger(__name__)

 from libchat.libchat import SqliteLibChat, ChatMsg
 from .msg import *
-from common.timer import timing
-from common.progress import ProgressReporter
+from .common.timer import timing
+from .common.progress import ProgressReporter

 class LibChatHelper(object):
    """ Build LibChat messages from WeChat Msg"""
@@ -46,21 +42,27 @@ class LibChatHelper(object):
            return img, 'jpeg'
        elif msg.type == TYPE_EMOJI:
            md5 = msg.imgPath
-            emoji_img, format = self.res.get_emoji_by_md5(md5)
-            return emoji_img, format
+            if md5:
+                emoji_img, format = self.res.get_emoji_by_md5(md5)
+                return emoji_img, format
+            else:
+                return '', ''
        elif msg.type == TYPE_CUSTOM_EMOJI:
            pq = PyQuery(msg.content)
            md5 = pq('emoticonmd5').text()
-            img, format = self.res.get_emoji(md5, None)
-            return img, format
+            if md5:
+                img, format = self.res.get_emoji(md5, None)
+                return img, format
+            else:
+                return '', ''
        else:
            return '', ''

    def _get_sound(self, msg):
        if msg.type == TYPE_SPEAK:
            audio_str, duration = self.res.get_voice_mp3(msg.imgPath)
-            return '{}:{}'.format(duration, base64.b64decode(audio_str))
-        return ''
+            return base64.b64decode(audio_str)
+        return b''

    def _get_extra(self, msg):
        ret = {}
@@ -77,7 +79,7 @@ class LibChatHelper(object):
        if img:
            # TODO don't use b64, directly return image content
            img = base64.b64decode(img)
-# TODO do we need to save format?
+        # TODO do we need to save format or voice duration?
        sound = self._get_sound(msg)
        extra = self._get_extra(msg)

@@ -1,24 +0,0 @@
-Copyright (c) 2008, ZETETIC LLC
-All rights reserved.
-
-Redistribution and use in source and binary forms, with or without
-modification, are permitted provided that the following conditions are met:
-    * Redistributions of source code must retain the above copyright
-      notice, this list of conditions and the following disclaimer.
-    * Redistributions in binary form must reproduce the above copyright
-      notice, this list of conditions and the following disclaimer in the
-      documentation and/or other materials provided with the distribution.
-    * Neither the name of the ZETETIC LLC nor the
-      names of its contributors may be used to endorse or promote products
-      derived from this software without specific prior written permission.
-
-THIS SOFTWARE IS PROVIDED BY ZETETIC LLC ''AS IS'' AND ANY
-EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
-DISCLAIMED. IN NO EVENT SHALL ZETETIC LLC BE LIABLE FOR ANY
-DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
-(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
-LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
-ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
-SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
@@ -1,7 +0,0 @@
-
-## NOTE
-
-For convenience, I include the binary version of sqlcipher 2.1.1.
-
-User can also choose to download and install sqlcipher 2.1.1 from their trusted source, such as:
-https://launchpad.net/ubuntu/+source/sqlcipher/2.1.1-2
@@ -1,130 +0,0 @@
-#! /bin/sh
-
-# sqlite3 - temporary wrapper script for .libs/sqlite3
-# Generated by ltmain.sh (GNU libtool) 2.2.6
-#
-# The sqlite3 program cannot be directly executed until all the libtool
-# libraries that it depends on are installed.
-#
-# This wrapper script should never be moved out of the build directory.
-# If it is, it will not operate correctly.
-
-# Sed substitution that helps us do robust quoting.  It backslashifies
-# metacharacters that are still active within double-quoted strings.
-Xsed='/opt/local/bin/gsed -e 1s/^X//'
-sed_quote_subst='s/\([`"$\\]\)/\\\1/g'
-
-# Be Bourne compatible
-if test -n "${ZSH_VERSION+set}" && (emulate sh) >/dev/null 2>&1; then
-  emulate sh
-  NULLCMD=:
-  # Zsh 3.x and 4.x performs word splitting on ${1+"$@"}, which
-  # is contrary to our usage.  Disable this feature.
-  alias -g '${1+"$@"}'='"$@"'
-  setopt NO_GLOB_SUBST
-else
-  case `(set -o) 2>/dev/null` in *posix*) set -o posix;; esac
-fi
-BIN_SH=xpg4; export BIN_SH # for Tru64
-DUALCASE=1; export DUALCASE # for MKS sh
-
-# The HP-UX ksh and POSIX shell print the target directory to stdout
-# if CDPATH is set.
-(unset CDPATH) >/dev/null 2>&1 && unset CDPATH
-
-relink_command=""
-
-# This environment variable determines our operation mode.
-if test "$libtool_install_magic" = "%%%MAGIC variable%%%"; then
-  # install mode needs the following variables:
-  generated_by_libtool_version='2.2.6'
-  notinst_deplibs=' libsqlite3.la'
-else
-  # When we are sourced in execute mode, $file and $ECHO are already set.
-  if test "$libtool_execute_magic" != "%%%MAGIC variable%%%"; then
-    ECHO="/bin/echo"
-    file="$0"
-    # Make sure echo works.
-    if test "X$1" = X--no-reexec; then
-      # Discard the --no-reexec flag, and continue.
-      shift
-    elif test "X`{ $ECHO '\t'; } 2>/dev/null`" = 'X\t'; then
-      # Yippee, $ECHO works!
-      :
-    else
-      # Restart under the correct shell, and then maybe $ECHO will work.
-      exec /bin/sh "$0" --no-reexec ${1+"$@"}
-    fi
-  fi
-
-  # Find the directory that this script lives in.
-  thisdir=`$ECHO "X$file" | $Xsed -e 's%/[^/]*$%%'`
-  test "x$thisdir" = "x$file" && thisdir=.
-
-  # Follow symbolic links until we get to the real thisdir.
-  file=`ls -ld "$file" | /opt/local/bin/gsed -n 's/.*-> //p'`
-  while test -n "$file"; do
-    destdir=`$ECHO "X$file" | $Xsed -e 's%/[^/]*$%%'`
-
-    # If there was a directory component, then change thisdir.
-    if test "x$destdir" != "x$file"; then
-      case "$destdir" in
-      [\\/]* | [A-Za-z]:[\\/]*) thisdir="$destdir" ;;
-      *) thisdir="$thisdir/$destdir" ;;
-      esac
-    fi
-
-    file=`$ECHO "X$file" | $Xsed -e 's%^.*/%%'`
-    file=`ls -ld "$thisdir/$file" | /opt/local/bin/gsed -n 's/.*-> //p'`
-  done
-
-
-  # Usually 'no', except on cygwin/mingw when embedded into
-  # the cwrapper.
-  WRAPPER_SCRIPT_BELONGS_IN_OBJDIR=no
-  if test "$WRAPPER_SCRIPT_BELONGS_IN_OBJDIR" = "yes"; then
-    # special case for '.'
-    if test "$thisdir" = "."; then
-      thisdir=`pwd`
-    fi
-    # remove .libs from thisdir
-    case "$thisdir" in
-    *[\\/].libs ) thisdir=`$ECHO "X$thisdir" | $Xsed -e 's%[\\/][^\\/]*$%%'` ;;
-    .libs )   thisdir=. ;;
-    esac
-  fi
-
-  # Try to get the absolute directory name.
-  absdir=`cd "$thisdir" && pwd`
-  test -n "$absdir" && thisdir="$absdir"
-
-  program='sqlite3'
-  progdir="$thisdir/.libs"
-
-
-  if test -f "$progdir/$program"; then
-    # Add our own library path to DYLD_LIBRARY_PATH
-    DYLD_LIBRARY_PATH="/Users/vuryleo/Projects/sqlcipher/.libs:$DYLD_LIBRARY_PATH"
-
-    # Some systems cannot cope with colon-terminated DYLD_LIBRARY_PATH
-    # The second colon is a workaround for a bug in BeOS R4 sed
-    DYLD_LIBRARY_PATH=`$ECHO "X$DYLD_LIBRARY_PATH" | $Xsed -e 's/::*$//'`
-
-    export DYLD_LIBRARY_PATH
-
-    if test "$libtool_execute_magic" != "%%%MAGIC variable%%%"; then
-      # Run the actual program with our arguments.
-
-      exec "$progdir/$program" ${1+"$@"}
-
-      $ECHO "$0: cannot exec $program $*" 1>&2
-      exit 1
-    fi
-  else
-    # The program doesn't exist.
-    $ECHO "$0: error: \`$progdir/$program' does not exist" 1>&2
-    $ECHO "This script is just a wrapper for $program." 1>&2
-    /bin/echo "See the libtool documentation for more information." 1>&2
-    exit 1
-  fi
-fi
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: UTF-8 -*-
 # File: list-chats.py
 # Author: Yuxin Wu <[email protected]>
@@ -6,7 +6,7 @@
 from wechat.parser import WeChatDBParser
 import sys
 if len(sys.argv) != 2:
-    print "Usage: {} db_file".format(sys.argv[0])
+    print("Usage: {} db_file".format(sys.argv[0]))
    sys.exit(1)

 db_file = sys.argv[1]
@@ -14,4 +14,4 @@ db_file = sys.argv[1]
 parser = WeChatDBParser(db_file)
 chats = parser.msgs_by_chat.keys()
 for k in chats:
-    print k.encode('utf-8'),  '\t', parser.contacts_rev[k].encode('utf-8')
+    print(parser.contacts[k], '\t', k)
@@ -1,11 +1,8 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: UTF-8 -*-
-# File: plot-num-msg-by-time.py
-# Date: Wed Mar 25 17:44:39 2015 +0800
-# Author: Yuxin Wu <[email protected]>

 from wechat.parser import WeChatDBParser
-from common.textutil import ensure_unicode
+from wechat.common.textutil import ensure_unicode

 from datetime import timedelta, datetime
 import numpy as np
@@ -39,7 +36,7 @@ plt.show()
 # I'm in a different time zone in this period:
 #TZ_DELTA = {(datetime(2014, 7, 13), datetime(2014, 10, 1)): -15}
 #def real_hour(x):
-    #for k, v in TZ_DELTA.iteritems():
+    #for k, v in TZ_DELTA.items():
        #if x > k[0] and x < k[1]:
            #print x
            #return (x.hour + v + 24) % 24
@@ -0,0 +1,8 @@
+pyquery
+javaobj-py3>=0.4.1
+pycryptodome
+pillow>=7.0
+requests
+pysqlcipher3>=1.0.3
+csscompressor
+numpy
@@ -1,7 +1,4 @@
 #!/bin/bash -e
-# File: compile_silk.sh
-# Date: Tue Jun 16 22:26:49 2015 +0800
-# Author: Yuxin Wu <[email protected]>

 cd `dirname "$0"`/silk
 make
@@ -47,7 +47,7 @@ ifeq (yes,$(USE_NEON))
 endif


-CFLAGS	+= -Wall -enable-threads -O3
+CFLAGS	+= -Wall -O3

 CFLAGS  += $(call cppflags-from-defines,$(CDEFINES))
 CFLAGS  += $(call cppflags-from-defines,$(ADDED_DEFINES))
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: UTF-8 -*-

 import logging
@@ -26,5 +26,5 @@ set_level_color(logging.ERROR, '\033[1;31m')

 if __name__ == '__main__':
    logger.info("info")
-    logger.warn("warn")
+    logger.warning("warn")

@@ -1,78 +1,71 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: UTF-8 -*-
-# File: audio.py
-# Date: Fri Jun 26 10:42:41 2015 +0800
-# Author: Yuxin Wu <[email protected]>

 import os
-from subprocess import PIPE, Popen, call
+import tempfile
 import logging
 logger = logging.getLogger(__name__)

-import pysox
-
-from common.textutil import get_file_b64
+from .common.textutil import get_file_b64
+from .common.procutil import subproc_succ

 SILK_DECODER = os.path.join(os.path.dirname(__file__),
                            '../third-party/silk/decoder')
-if not os.path.exists(SILK_DECODER):
-    logger.error("Silk decoder is not compiled. Please see README.md.")
-    raise RuntimeError()

 def parse_wechat_audio_file(file_name):
    try:
        return do_parse_wechat_audio_file(file_name)
    except Exception as e:
-        logger.error("Pase audio file {} error!".format(file_name))
-        logger.error(e)
+        logger.error(f"Error when parsing audio file {file_name}: {str(e)}")
        return "", 0

 def do_parse_wechat_audio_file(file_name):
-    """ return a mp3 base64 string, and the duration"""
+    """ return a mp3 stored in base64 unicode string, and the duration"""
    if not file_name: return "", 0

-    mp3_file = os.path.join('/tmp',
-                            os.path.basename(file_name)[:-4] + '.mp3')
-    with open(file_name) as f:
-        header = f.read(10)
-    if 'AMR' in header:
-        # maybe this is faster than calling sox from command line?
-        infile = pysox.CSoxStream(file_name)
-        outfile = pysox.CSoxStream(mp3_file, 'w', infile.get_signal())
-        chain = pysox.CEffectsChain(infile, outfile)
-        chain.flow_effects()
-        outfile.close()
+    with tempfile.TemporaryDirectory(prefix="wechatdump_audio") as temp:
+        mp3_file = os.path.join(temp,
+                                os.path.basename(file_name)[:-4] + '.mp3')
+        with open(file_name, 'rb') as f:
+            header = f.read(10)
+        if b'AMR' in header:
+            cmd = f"sox -e signed -c 1 {file_name} {mp3_file}"
+            subproc_succ(cmd)
+            cmd = f"soxi -D {mp3_file}"
+            duration = float(subproc_succ(cmd))

-        signal = infile.get_signal().get_signalinfo()
-        duration = signal['length'] * 1.0 / signal['rate']
-    elif 'SILK' in header:
-        raw_file = os.path.join('/tmp',
-                                os.path.basename(file_name)[:-4] + '.raw')
-        proc = Popen('{0} {1} {2}'.format(SILK_DECODER,
-                                                file_name, raw_file),
-                    shell=True, stdout=PIPE, stderr=PIPE)
-        stdout = proc.communicate()[0]
-        for line in stdout.split('\n'):
-            if 'File length' in line:
-                duration = float(line[13:-3].strip())
-                break
+            # The below is python2 only. It should be equivalent to using sox from command line
+            # import pysox
+            # infile = pysox.CSoxStream(file_name)
+            # outfile = pysox.CSoxStream(mp3_file, 'w', infile.get_signal())
+            # chain = pysox.CEffectsChain(infile, outfile)
+            # chain.flow_effects()
+            # outfile.close()
+            # signal = infile.get_signal().get_signalinfo()
+            # duration = signal['length'] * 1.0 / signal['rate']
+        elif b'SILK' in header:
+            if not os.path.exists(SILK_DECODER):
+                raise RuntimeError("Silk decoder is not compiled. Please see README.md.")
+
+            raw_file = os.path.join(temp,
+                                    os.path.basename(file_name)[:-4] + '.raw')
+            cmd = '{0} {1} {2}'.format(SILK_DECODER, file_name, raw_file)
+            out = subproc_succ(cmd)
+            for line in out.split(b'\n'):
+                if b'File length' in line:
+                    duration = float(line[13:-3].strip())
+                    break
+            else:
+                raise RuntimeError("Error decoding silk audio file!" + out.decode('utf-8'))
+
+            # TODO don't know how to do this with python
+            subproc_succ('sox -r 24000 -e signed -b 16 -c 1 {} {}'.format(raw_file, mp3_file))
        else:
-            raise RuntimeError("Error decoding silk audio file!")
-
-        # I don't know how to do this with pysox
-        proc = call('sox -r 24000 -e signed -b 16 -c 1 {} {}'.format(
-            raw_file, mp3_file), shell=True)
-        os.unlink(raw_file)
-    else:
-        raise NotImplementedError("Unsupported Audio Format! This is a bug!")
-    try:
+            raise NotImplementedError("Audio file format cannot be recognized.")
        mp3_string = get_file_b64(mp3_file)
-        os.unlink(mp3_file)
-    except:
-        raise RuntimeError("Failed to decode audio file: {}".format(file_name))
    return mp3_string, duration

 if __name__ == '__main__':
    import sys
    fname = sys.argv[1]
-    print parse_wechat_audio_file(fname)[1]
+    print(parse_wechat_audio_file(fname)[1])
@@ -1,71 +1,129 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: UTF-8 -*-
-# File: avatar.py
-# Date: Thu Jun 18 00:02:07 2015 +0800
-# Author: Yuxin Wu <[email protected]>

 from PIL import Image
-import cStringIO
+import io
+import glob
 import os
 import numpy as np
 import logging
 import sqlite3
 logger = logging.getLogger(__name__)

-from common.textutil import ensure_bin_str, md5
+from .common.textutil import ensure_unicode, md5
+
+
+def _filename_priority(s):
+    if "_hd" in s and s.endswith(".png"):
+        return 10
+    else:
+        return 1
+

 class AvatarReader(object):
-    def __init__(self, avt_dir, avt_db="avatar.index"):
-        self.avt_dir = avt_dir
+    def __init__(self, res_dir, avt_db="avatar.index"):
+        self.sfs_dir = os.path.join(res_dir, 'sfs')
+
+        # new location of avatar, see #50
+        self.avt_dir = os.path.join(res_dir, 'avatar')
+        if not os.path.isdir(self.avt_dir) or len(os.listdir(self.avt_dir)) == 0:
+            self.avt_dir = None

        self.avt_db = avt_db
-        if self.avt_db is None or not os.path.isfile(self.avt_db):
-            logger.warn(
-                    "Avatar database {} not found. Will not use avatar!".format(avt_db))
-            self.avt_db = None
+        self._use_avt = True
+        if self.avt_db is not None:
+            if len(glob.glob(os.path.join(self.sfs_dir, 'avatar*'))) == 0:
+                # has sfs/avatar*
+                self.avt_db = None
+        if self.avt_dir is None and self.avt_db is None:
+            logger.warn("Cannot find avatar storage. Will not use avatar!")
+            self._use_avt = False
+
+    def get_avatar_from_avtdb(self, avtid):
+        try:
+            candidates = self._search_avt_db(avtid)
+            candidates = sorted(candidates, key=lambda x: _filename_priority(x[0]), reverse=True)
+            for c in candidates:
+                path, offset, size = c
+                return self.read_img_from_block(path, offset, size)
+        except Exception:
+            pass
+
+    def get_avatar_from_avtdir(self, avtid):
+        dir1, dir2 = avtid[:2], avtid[2:4]
+        candidates = glob.glob(os.path.join(self.avt_dir, dir1, dir2, f"*{avtid}*"))
+        candidates = sorted(set(candidates), key=_filename_priority, reverse=True)
+        for cand in candidates:
+            if os.path.isdir(cand):
+                candidates.extend(os.path.join(cand, x) for x in os.listdir(cand))
+        for cand in candidates:
+            if os.path.isdir(cand):
+                continue
+            try:
+                if cand.endswith(".bm"):
+                    return self.read_bm_file(cand)
+                else:
+                    return Image.open(cand)
+            except Exception:
+                logger.exception("")
+                pass

    def get_avatar(self, username):
        """ username: `username` field in db.rcontact"""
-        if self.avt_db is None: return None
-
-        username = ensure_bin_str(username)
-        filename = md5(username)
-        dir1, dir2 = filename[:2], filename[2:4]
-        filename = os.path.join(dir1, dir2,
-                                "user_{}.png".format(filename))
-
-        try:
-            try:
-                pos, size = self.query_index(filename)
-                return self.read_img(pos, size)
-            except TypeError:
-                logger.warn("Avatar for {} not found in avatar database.".format(username))
-                return None
-        except Exception as e:
-            raise
-            print e
-            logger.warn("Failed to retrieve avatar!")
+        if not self._use_avt:
            return None
+        username = ensure_unicode(username)
+        avtid = md5(username.encode('utf-8'))

+        if self.avt_db is not None:
+            ret = self.get_avatar_from_avtdb(avtid)
+            if ret is not None:
+                return ret

-    def read_img(self, pos, size):
+        if self.avt_dir is not None:
+            ret = self.get_avatar_from_avtdir(avtid)
+            if ret is not None:
+                return ret
+        logger.warning("Avatar for {} not found anywhere.".format(username))
+
+    def read_img_from_block(self, filename, pos, size):
        file_idx = pos >> 32
-        fname = os.path.join(self.avt_dir,
+        fname = os.path.join(self.sfs_dir,
                'avatar.block.' + '{:05d}'.format(file_idx))
-        # a 64-byte offset of each block file
-        start_pos = pos - file_idx * (2**32) + 64
+        # offset of each block file: 17 + len(path)
+        start_pos = pos - file_idx * (2**32) + 16 + len(filename) + 1
        try:
            with open(fname, 'rb') as f:
                f.seek(start_pos)
                data = f.read(size)
-                im = Image.open(cStringIO.StringIO(data))
+                im = Image.open(io.BytesIO(data))
                return im
        except IOError as e:
            logger.warn("Cannot read avatar from {}: {}".format(fname, str(e)))
            return None

-    def query_index(self, filename):
+    def read_bm_file(self, fname):
+        # history at https://github.com/ppwwyyxx/wechat-dump/pull/14
+        with open(fname, 'rb') as f:
+            # filesize is 36880=96x96x4+16
+            size = (96, 96, 3)
+            img = np.zeros(size, dtype='uint8')
+            for i in range(96):
+                for j in range(96):
+                    r, g, b, a = f.read(4)
+                    img[i,j] = (r, g, b)
+            return Image.fromarray(img, mode="RGB")
+
+    def _search_avt_db(self, avtid):
        conn = sqlite3.connect(self.avt_db)
-        cursor = conn.execute("select Offset,Size from Index_avatar where FileName='{}'".format(filename))
-        pos, size = cursor.fetchone()
-        return pos, size
+        cursor = conn.execute("select FileName,Offset,Size from Index_avatar")
+        candidates = []
+        for path, offset, size in cursor:
+            if avtid in path:
+                candidates.append((path, offset, size))
+        return candidates
+
+if __name__ == '__main__':
+    import sys
+    r = AvatarReader(sys.argv[1], sys.argv[2])
+    print(r.get_avatar(sys.argv[3]))
@@ -0,0 +1,48 @@
+# -*- coding: utf-8 -*-
+
+import subprocess
+import logging
+logger = logging.getLogger(__name__)
+
+def subproc_call(cmd, timeout=None):
+    """
+    Execute a command with timeout, and return STDOUT and STDERR
+
+    Args:
+        cmd(str): the command to execute.
+        timeout(float): timeout in seconds.
+
+    Returns:
+        output(bytes), retcode(int). If timeout, retcode is -1.
+    """
+    try:
+        output = subprocess.check_output(
+            cmd, stderr=subprocess.STDOUT,
+            shell=True, timeout=timeout)
+        return output, 0
+    except subprocess.TimeoutExpired as e:
+        logger.warn("Command '{}' timeout!".format(cmd))
+        if e.output:
+            logger.warn(e.output.decode('utf-8'))
+            return e.output, -1
+        else:
+            return "", -1
+    except subprocess.CalledProcessError as e:
+        logger.warn("Command '{}' failed, return code={}".format(cmd, e.returncode))
+        logger.warn(e.output.decode('utf-8'))
+        return e.output, e.returncode
+    except Exception:
+        logger.warn("Command '{}' failed to run.".format(cmd))
+        return "", -2
+
+
+def subproc_succ(cmd):
+    """
+    Like subproc_call, but expect the cmd to succeed.
+    """
+    output, ret = subproc_call(cmd)
+    assert ret == 0
+    return output
+
+
+
@@ -1,8 +1,4 @@
-#!/usr/bin/env python2
 # -*- coding: UTF-8 -*-
-# File: progress.py
-# Date: Wed Jun 17 23:59:52 2015 +0800
-# Author: Yuxin Wu <[email protected]>

 import time
 import sys
@@ -36,12 +32,12 @@ class ProgressReporter(object):
        self._prev_report_time = now
        dt = now - self._start_time
        if self._total and self._cnt > 0:
-            eta_msg = '{}/{} ETA: {:.2f}'.format(self._cnt, self._total,
+            eta_msg = '{}/{} ETA: {:.1f}'.format(self._cnt, self._total,
                    (self._total-self._cnt)*dt/self._cnt)
        else:
            eta_msg = '{} done'.format(self._cnt)
-        self._fout.write(u'{}: avg {:.3f}/sec'
-                         u', passed {:.3f}sec, {}  {} \r'.format(
+        self._fout.write(u'{}: avg {:.1f}/sec'
+                         u', passed {:.1f}sec, {}  {} \r'.format(
            self._name, self._cnt / dt, dt, eta_msg, extra_msg))
        self._fout.flush()

@@ -1,23 +1,14 @@
-#!/usr/bin/env python2
 # -*- coding: UTF-8 -*-
-# File: utils.py
-# Date: Wed Jun 17 23:59:25 2015 +0800
-# Author: Yuxin Wu <[email protected]>

 import hashlib
 import base64

-def ensure_bin_str(s):
-    if type(s) == str:
-        return s
-    if type(s) == unicode:
-        return s.encode('utf-8')
-
 def ensure_unicode(s):
    if type(s) == str:
-        return s.decode('utf-8')
-    if type(s) == unicode:
        return s
+    elif type(s) == bytes:
+        return s.decode('utf-8')
+    raise TypeError(f"type of string is {type(s)}")


 def md5(s):
@@ -26,8 +17,12 @@ def md5(s):
    return m.hexdigest()

 def get_file_b64(fname):
-    data = open(fname, 'rb').read()
-    return base64.b64encode(data)
+    with open(fname, 'rb') as f:
+        return base64.b64encode(f.read()).decode('ascii')
+
+def get_file_md5(fname):
+    with open(fname, 'rb') as f:
+        return md5(f.read())

 def safe_filename(fname):
    filename = ensure_unicode(fname)
@@ -1,8 +1,4 @@
-#!/usr/bin/env python2
 # -*- coding: UTF-8 -*-
-# File: timer.py
-# Date: Wed Jun 17 23:25:54 2015 +0800
-# Author: Yuxin Wu <[email protected]>

 import time, functools
 from collections import defaultdict
@@ -20,7 +16,7 @@ class TotalTimer(object):
        self.times = defaultdict(float)

    def __del__(self):
-        for k, v in self.times.iteritems():
+        for k, v in self.times.items():
            logger.info("{} took {} seconds in total.".format(k, v))

 _total_timer = TotalTimer()
@@ -0,0 +1,214 @@
+import os
+from pathlib import Path
+import logging
+import tempfile
+import io
+import requests
+import base64
+import imghdr
+from PIL import Image
+import pickle
+from Crypto.Cipher import AES
+
+from .parser import WeChatDBParser
+from .common.textutil import md5 as get_md5_hex, get_file_b64, get_file_md5
+
+
+LIB_PATH = os.path.dirname(os.path.abspath(__file__))
+DEFAULT_EMOJI_CACHE = os.path.join(LIB_PATH, '..', 'emoji.cache')
+logger = logging.getLogger(__name__)
+
+
+def _get_aes_key(md5):
+    # ascii representation of the first half of md5 is used as aes key
+    assert len(md5) == 32
+    return md5[:16].encode('ascii')
+    # ret = ""
+    # for ch in md5[:16]:
+        # ret += format(ord(ch), 'x')
+    # return ret
+
+
+class EmojiReader:
+    def __init__(self,
+        resource_dir: str,
+        parser: WeChatDBParser,
+        cache_file: str=None):
+        """
+        Args:
+            resource_dir: path to resource/
+            parser: Database parser
+            cache_file: a cache file to store emoji downloaded from URLs.
+                default to a emoji.cache file under wechat-dump.
+        """
+        self.emoji_dir = Path(resource_dir) / 'emoji'
+        assert self.emoji_dir.is_dir(), self.emoji_dir
+        self.parser = parser
+        self.emoji_info = parser.emoji_info or {}
+        # mapping from md5 to the (cdnurl, encrypturl, aeskey)
+        # columns in EmojiInfo table.
+        self.cache_file = cache_file or DEFAULT_EMOJI_CACHE
+
+        # cache stores md5 -> (base64str, format)
+        if os.path.isfile(self.cache_file):
+            with open(self.cache_file, "rb") as f:
+                self._cache = pickle.load(f)
+        else:
+            self._cache = {}
+        self._cache_size = len(self._cache)
+        self.encryption_key = parser.get_emoji_encryption_key()
+        if self.encryption_key is not None:
+            self.encryption_key = _get_aes_key(self.encryption_key)
+
+    def get_emoji(self, md5):
+        """ Returns: (b64 encoded img string, format) """
+
+        assert md5, f"Invalid md5 {md5}!"
+        # check cache
+        img, format = self._cache_query(md5)
+        if format:
+            return img, format
+
+        # check resource/
+        subdir = self.parser.emoji_groups.get(md5, '')
+        dir_to_search = self.emoji_dir / subdir
+        img, format = self._search_in_res(dir_to_search, md5, False)
+        if format:
+            return img, format
+
+        emoji_info = self.emoji_info.get(md5, None)
+        if emoji_info:
+            catalog, cdnurl, encrypturl, aeskey = emoji_info
+            img, format = self._fetch(md5, cdnurl, encrypturl, aeskey)
+            if format:
+                return img, format
+
+        img, format = self._search_in_res(dir_to_search, md5, True)
+        if format:
+            logger.info(f"Using fallback for emoji {md5}")
+            return img, format
+        else:
+            emoji_in_table = emoji_info is not None
+            msg = "not in database" if not emoji_in_table else f"group='{subdir}'"
+            logger.warning(f"Cannot find emoji {md5}: {msg}")
+            return None, None
+
+    def _cache_query(self, md5):
+        data, format = self._cache.get(md5, (None, None))
+        if data is not None and not isinstance(data, str):
+            data = data.decode('ascii')
+        return data, format
+
+    def _cache_add(self, md5, values):
+        self._cache[md5] = values
+        if len(self._cache) >= self._cache_size + 15:
+            self.flush_cache()
+
+    def flush_cache(self):
+        if len(self._cache) > self._cache_size:
+            self._cache_size = len(self._cache)
+            with open(self.cache_file, 'wb') as f:
+                pickle.dump(self._cache, f, protocol=-1)
+
+    def _search_in_res(self, dir, md5, allow_fallback=False):
+        if allow_fallback:
+            candidates = dir.glob(f'{md5}*')
+            # There are misc low-quality matches, e.g.:
+            # 'md5_{0..15}' for each frame of gif, non-animated md5_thumb, md5_cover
+            # candidates = [k for k in candidates if not re.match('.*_[0-9]+$', k)]
+            # candidates = [k for k in candidates if (not k.endswith('_cover') and not k.endswith('_thumb')))]
+        else:
+            if (dir / md5).is_file():
+                candidates = [dir / md5]
+            else:
+                candidates = []
+
+        def get_data_no_fallback(fname):
+            if imghdr.what(fname):
+                data_md5 = get_file_md5(fname)
+                if data_md5 == md5:
+                    return get_file_b64(fname), imghdr.what(fname)
+
+            try:
+                content = self._decrypt_emoji(fname)
+                data_md5 = get_md5_hex(content)
+                if data_md5 != md5:
+                    if content.startswith(b"wxgf"):
+                        raise ValueError("Unsupported mysterious image format: wxgf")
+                    raise ValueError("Decrypted data mismatch md5!")
+                im = Image.open(io.BytesIO(content))
+                return (base64.b64encode(content).decode('ascii'), im.format.lower())
+            except Exception as e:
+                logger.error(f"Error decrypting emoji {fname} : {str(e)}")
+
+        def get_data_fallback(fname):
+            if not imghdr.what(fname):
+                return  # fallback files are not encrypted
+            return get_file_b64(fname), imghdr.what(fname)
+
+        get_data_func = get_data_fallback if allow_fallback else get_data_no_fallback
+        results = [(x, get_data_func(x)) for x in candidates]
+        results = [(a, b) for a, b in results if b is not None]
+        # maybe sort candidates by heuristics?
+        if len(results):
+            return results[0][1]
+        return (None, None)
+
+    def _decrypt_emoji(self, fname):
+        cipher = AES.new(self.encryption_key, AES.MODE_ECB)
+        with open(fname, 'rb') as f:
+            head = f.read(1024)
+            plain_head = cipher.decrypt(head)
+            data = plain_head + f.read()
+        return data
+
+    def _fetch(self, md5, cdnurl, encrypturl, aeskey):
+        ret = None
+        if cdnurl:
+            try:
+                logger.info("Requesting emoji {} from {} ...".format(md5, cdnurl))
+                r = requests.get(cdnurl).content
+                emoji_md5 = get_md5_hex(r)
+                im = Image.open(io.BytesIO(r))
+                ret = (base64.b64encode(r).decode('ascii'), im.format.lower())
+                if emoji_md5 == md5:
+                    self._cache_add(md5, ret)
+                    return ret
+                else:
+                    raise ValueError("Emoji MD5 from CDNURL does not match")
+            except Exception:
+                logger.debug("Error processing cdnurl {}".format(cdnurl))
+
+        if encrypturl:
+            try:
+                logger.info("Requesting encrypted emoji {} from {} ...".format(md5, encrypturl))
+                buf = requests.get(encrypturl).content
+                aeskey = bytes.fromhex(aeskey)
+                cipher = AES.new(aeskey, AES.MODE_CBC, iv=aeskey)
+                decrypted_buf = cipher.decrypt(buf)
+
+                im = Image.open(io.BytesIO(decrypted_buf))
+                ret = (base64.b64encode(decrypted_buf).decode('ascii'), im.format.lower())
+                self._cache_add(md5, ret)
+                return ret
+            except Exception:
+                logger.exception("Error processing encrypturl {}".format(encrypturl))
+        if ret is not None:
+            # ret may become something with wrong md5. Try it anyway, but don't cache.
+            return ret
+        return None, None
+
+if __name__ == "__main__":
+    logger.setLevel(logging.DEBUG)
+    handler = logging.StreamHandler()
+    logger.addHandler(handler)
+
+    class Dummy():
+        def _cache_add(self, md5, ret):
+            pass
+    # test decryption
+    md5 = '5a7fc462d63ef845e6d99c1523bbc91e'
+    encurl = 'http://emoji.qpic.cn/wx_emoji/CQmBgayyMuvscRVEKN9s4HyTjKVU9iacqqhyCpdtqOVcCql5JaibjDFg/'
+    enckey = '8ba7f51f9f3ac58cf8ed937fc90200a6'
+    b64, format = EmojiReader._fetch(Dummy(), md5, None, encurl, enckey)
+    print("format=", format)
@@ -1,8 +1,4 @@
-#!/usr/bin/env python2
 # -*- coding: UTF-8 -*-
-# File: msg.py
-# Date: Thu Jun 18 00:01:00 2015 +0800
-# Author: Yuxin Wu <[email protected]>
 TYPE_MSG = 1
 TYPE_IMG = 3
 TYPE_SPEAK = 34
@@ -10,22 +6,27 @@ TYPE_NAMECARD = 42
 TYPE_VIDEO_FILE = 43
 TYPE_EMOJI = 47
 TYPE_LOCATION = 48
-TYPE_LINK = 49  # link share OR file from web
+TYPE_LINK = 49  # link share OR file from web, see https://github.com/ppwwyyxx/wechat-dump/issues/52
 TYPE_VOIP = 50
 TYPE_WX_VIDEO = 62  # video took by wechat
 TYPE_SYSTEM = 10000
 TYPE_CUSTOM_EMOJI = 1048625
+TYPE_REDENVELOPE = 436207665
+TYPE_MONEY_TRANSFER = 419430449  # 微信转账
 TYPE_LOCATION_SHARING = -1879048186
+TYPE_REPLY = 822083633  # 回复的消息.
 TYPE_APP_MSG = 16777265

 _KNOWN_TYPES = [eval(k) for k in dir() if k.startswith('TYPE_')]

 import re
+import io
 from pyquery import PyQuery
+import xml.etree.ElementTree as ET
 import logging
 logger = logging.getLogger(__name__)

-from common.textutil import ensure_unicode
+from .common.textutil import ensure_unicode


 class WeChatMsg(object):
@@ -37,7 +38,7 @@ class WeChatMsg(object):
        return False

    def __init__(self, values):
-        for k, v in values.iteritems():
+        for k, v in values.items():
            setattr(self, k, v)
        if self.type not in _KNOWN_TYPES:
            logger.warn("Unhandled message type: {}".format(self.type))
@@ -46,25 +47,27 @@ class WeChatMsg(object):

    def msg_str(self):
        if self.type == TYPE_LOCATION:
-            pq = PyQuery(self.content_xml_ready, parser='xml')
-            loc = pq('location').attr
-            label = loc['label']
            try:
+                pq = PyQuery(self.content_xml_ready, parser='xml')
+                loc = pq('location').attr
+                label = loc['label']
                poiname = loc['poiname']
                if poiname:
                    label = poiname
+                return "LOCATION:" + label + " ({},{})".format(loc['x'], loc['y'])
            except:
-                pass
-            return "LOCATION:" + label + " ({},{})".format(loc['x'], loc['y'])
+                return "LOCATION: unknown"
        elif self.type == TYPE_LINK:
            pq = PyQuery(self.content_xml_ready)
            url = pq('url').text()
            if not url:
+                # TODO: see https://github.com/ppwwyyxx/wechat-dump/issues/52 for
+                # more logic to implement
                title = pq('title').text()
-                assert title, \
-                        u"No title or url found in TYPE_LINK: {}".format(self.content)
-                return u"FILE:{}".format(title)
-            return u"URL:{}".format(url)
+                if title:  # may not be correct
+                    return "FILE:{}".format(title)
+                return "NOT IMPLEMENTED: " + self.content_xml_ready
+            return "URL:{}".format(url)
        elif self.type == TYPE_NAMECARD:
            pq = PyQuery(self.content_xml_ready, parser='xml')
            msg = pq('msg').attr
@@ -73,7 +76,7 @@ class WeChatMsg(object):
                name = msg['alias']
            if not name:
                name = ""
-            return u"NAMECARD: {}".format(self.content_xml_ready)
+            return "NAMECARD: {}".format(self.content_xml_ready)
        elif self.type == TYPE_APP_MSG:
            pq = PyQuery(self.content_xml_ready, parser='xml')
            return pq('title').text()
@@ -88,6 +91,31 @@ class WeChatMsg(object):
        elif self.type == TYPE_EMOJI:
            # TODO add emoji name
            return self.content
+        elif self.type == TYPE_REDENVELOPE:
+            data_to_parse = io.BytesIO(self.content.encode('utf-8'))
+            try:
+                for event, elem in ET.iterparse(data_to_parse, events=('end',)):
+                    if elem.tag == 'sendertitle':
+                        title = elem.text
+                        return "[RED ENVELOPE]\n{}".format(title)
+            except:
+                pass
+            return "[RED ENVELOPE]"
+        elif self.type == TYPE_MONEY_TRANSFER:
+            data_to_parse = io.BytesIO(self.content.encode('utf-8'))
+            try:
+                for event, elem in ET.iterparse(data_to_parse, events=('end',)):
+                    if elem.tag == 'des':
+                        title = elem.text
+                        return "[Money Transfer]\n{}".format(title)
+            except:
+                pass
+            return "[Money Transfer]"
+        elif self.type == TYPE_REPLY:
+            pq = PyQuery(self.content_xml_ready)
+            msg = pq('title').text()
+            # TODO parse reply.
+            return msg
        else:
            # TODO replace smiley with text
            return self.content
@@ -100,14 +128,14 @@ class WeChatMsg(object):
        return msg

    def __repr__(self):
-        ret = u"{}|{}:{}:{}".format(
+        ret = "{}|{}:{}:{}".format(
            self.type,
-            self.talker if not self.isSend else 'me',
+            self.talker_nickname if not self.isSend else 'me',
            self.createTime,
-            ensure_unicode(self.msg_str())).encode('utf-8')
+            ensure_unicode(self.msg_str()))
        if self.imgPath:
-            ret = u"{}|img:{}".format(ensure_unicode(ret.strip()), self.imgPath)
-            return ret.encode('utf-8')
+            ret = "{}|img:{}".format(ensure_unicode(ret.strip()), self.imgPath)
+            return ret
        else:
            return ret

@@ -1,10 +1,6 @@
-#!/usr/bin/env python2
 # -*- coding: UTF-8 -*-
-# File: msgslice.py
-# Date: Thu Jan 08 00:15:49 2015 +0800
-# Author: Yuxin Wu <[email protected]>

-class MessageSlicerByTime(object):
+class MessageSlicerByTime:
    """ Separate messages into slices by time,
        for time display in html.
        A new day always begins a new slice.
@@ -32,11 +28,11 @@ class MessageSlicerByTime(object):
        assert len(msgs) == sum([len(k) for k in ret])
        return ret

-class MessageSlicerBySize(object):
+class MessageSlicerBySize:
    """ Separate messages into slices by max slice size,
        to avoid too large html.
    """
-    def __init__(self, size=1000):
+    def __init__(self, size=1500):
        """ a slice will have <= 1.5 * cnt messages"""
        self.size = size
        assert self.size > 1
@@ -1,8 +1,4 @@
-#!/usr/bin/env python2
 # -*- coding: UTF-8 -*-
-# File: parser.py
-# Date: Thu Jun 18 00:03:53 2015 +0800
-# Author: Yuxin Wu <[email protected]>

 import sqlite3
 from collections import defaultdict
@@ -11,8 +7,8 @@ from datetime import datetime
 import logging
 logger = logging.getLogger(__name__)

-from .msg import WeChatMsg
-from common.textutil import ensure_unicode
+from .msg import WeChatMsg, TYPE_SYSTEM
+from .common.textutil import ensure_unicode

 """ tables in concern:
 emojiinfo
@@ -31,11 +27,12 @@ class WeChatDBParser(object):
        self.db_fname = db_fname
        self.db_conn = sqlite3.connect(self.db_fname)
        self.cc = self.db_conn.cursor()
-        self.contacts = {}
+        self.contacts = {}      # username -> nickname
+        self.contacts_rev = defaultdict(list)
        self.msgs_by_chat = defaultdict(list)
        self.emoji_groups = {}
-        self.emoji_url = {}
-        self.internal_emojis = {}
+        self.emoji_info = {}
+        self.emoji_encryption_key = None
        self._parse()

    def _parse_contact(self):
@@ -50,8 +47,9 @@ SELECT username,conRemark,nickname FROM rcontact
            else:
                self.contacts[username] = ensure_unicode(nickname)

-        self.contacts_rev = {v: k for k, v in self.contacts.iteritems()}
-        logger.info("Found {} contacts.".format(len(self.contacts)))
+        for k, v in self.contacts.items():
+            self.contacts_rev[v].append(k)
+        logger.info("Found {} names in `contact` table.".format(len(self.contacts)))

    def _parse_msg(self):
        msgs_tot_cnt = 0
@@ -68,7 +66,7 @@ SELECT {} FROM message
            if not WeChatMsg.filter_type(msg.type):
                self.msgs_by_chat[msg.chat].append(msg)

-        for k, v in self.msgs_by_chat.iteritems():
+        for k, v in self.msgs_by_chat.items():
            self.msgs_by_chat[k] = sorted(v, key=lambda x: x.createTime)
            msgs_tot_cnt += len(v)
        logger.info("Found {} message records.".format(msgs_tot_cnt))
@@ -76,7 +74,10 @@ SELECT {} FROM message
    def _parse_userinfo(self):
        userinfo_q = self.cc.execute(""" SELECT id, value FROM userinfo """)
        userinfo = dict(userinfo_q)
-        self.username = userinfo[2]
+        self.username = userinfo.get(2, None)
+        if self.username is None:
+            logger.error("Cannot find username in userinfo table!")
+            self.username = input("Please enter your username:")
        logger.info("Your username is: {}".format(self.username))

    def _parse_imginfo(self):
@@ -96,22 +97,22 @@ SELECT {} FROM message

    def _parse_emoji(self):
        # wechat provided emojis
-        emojiinfo_q = self.cc.execute(
+        query = self.cc.execute(
 """ SELECT md5, groupid FROM EmojiInfoDesc """)
-        for row in emojiinfo_q:
+        for row in query:
            md5, group = row
            self.emoji_groups[md5] = group

-        NEEDED_EMOJI_CATALOG = [49, 50, 17]
-        emojiinfo_q = self.cc.execute(
-""" SELECT md5, catalog, name, cdnUrl FROM EmojiInfo""")
-        for row in emojiinfo_q:
-            md5, catalog, name, cdnUrl = row
-            if cdnUrl:
-                self.emoji_url[md5] = cdnUrl
-            if catalog not in NEEDED_EMOJI_CATALOG:
-                continue
-            self.internal_emojis[md5] = name
+        try:
+            query = self.cc.execute(
+    """ SELECT md5, catalog, name, cdnUrl, encrypturl, aeskey FROM EmojiInfo""")
+        except: # old database does not have cdnurl
+            pass
+        else:
+            for row in query:
+                md5, catalog, name, cdnUrl, encrypturl, aeskey = row
+                if cdnUrl or encrypturl:
+                    self.emoji_info[md5] = (catalog, cdnUrl, encrypturl, aeskey)


    def _parse(self):
@@ -121,6 +122,16 @@ SELECT {} FROM message
        self._parse_imginfo()
        self._parse_emoji()

+    def get_emoji_encryption_key(self):
+        # obtain local encryption key in a special entry in the database
+        # this also equals to md5(imei)
+        query = self.cc.execute("SELECT md5 FROM EmojiInfo where catalog == 153")
+        results = list(query)
+        if len(results):
+            assert len(results) == 1, "Found > 1 encryption keys in EmojiInfo. This is a bug!"
+            return results[0][0]
+        return None
+
    # process the values in a row
    def _parse_msg_row(self, row):
        """ parse a record of message into my format"""
@@ -128,26 +139,61 @@ SELECT {} FROM message
        if values['content']:
            values['content'] = ensure_unicode(values['content'])
        else:
-            values['content'] = u''
+            values['content'] = ''
        values['createTime'] = datetime.fromtimestamp(values['createTime']/ 1000)
        values['chat'] = values['talker']
        try:
            if values['chat'].endswith('@chatroom'):
-                values['chat'] = self.contacts[values['chat']]
+                values['chat_nickname'] = self.contacts[values['chat']]
                content = values['content']
-                talker = content[:content.find(':')]
-                try:
-                    values['talker'] = self.contacts[talker]
-                    values['content'] = content[content.find('\n') + 1:]
-                except KeyError:
-                    # system messages have no talker
-                    values['talker'] = u''
+
+                if values['isSend'] == 1:
+                    values['talker'] = self.username
+                elif values['type'] == TYPE_SYSTEM:
+                    values['talker'] = 'SYSTEM'
+                else:
+                    talker = content[:content.find(':')]
+                    values['talker'] = talker
+                    values['talker_nickname'] = self.contacts.get(talker, talker)
+
+                values['content'] = content[content.find('\n') + 1:]
            else:
                tk_id = values['talker']
-                values['chat'] = self.contacts[tk_id]
-                values['talker'] = self.contacts[tk_id]
+                values['chat'] = tk_id
+                values['chat_nickname'] = self.contacts[tk_id]
+                values['talker'] = tk_id
+                values['talker_nickname'] = self.contacts[tk_id]
        except KeyError:
            # It's possible that messages are kept in database after contacts been deleted
-            logger.warn("Unknown contact, probably deleted: {}".format(tk_id))
+            logger.warn("Unknown contact: {}".format(values.get('talker', '')))
            return None
        return values
+
+    @property
+    def all_chat_ids(self):
+        return self.msgs_by_chat.keys()
+
+    @property
+    def all_chat_nicknames(self):
+        return [self.contacts[k] for k in self.all_chat_ids if len(self.contacts[k])]
+
+    def get_id_by_nickname(self, nickname):
+        """
+        Get chat id by nickname.
+        """
+        l = self.contacts_rev[nickname]
+        if len(l) == 0:
+            raise KeyError("No contacts have nickname {}".format(nickname))
+        if len(l) > 1:
+            logger.warn("More than one contacts have nickname {}! Using the first contact".format(nickname))
+        return l[0]
+
+    def get_chat_id(self, nick_name_or_id):
+        """
+        Get the unique chat id by either chat id itself, or the nickname of the chat.
+        """
+        if nick_name_or_id in self.contacts:
+            return nick_name_or_id
+        else:
+            return self.get_id_by_nickname(nick_name_or_id)
+
@@ -1,8 +1,5 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: UTF-8 -*-
-# File: render.py
-# Date: Thu Jun 18 00:03:10 2015 +0800
-# Author: Yuxin Wu <[email protected]>

 import os
 import base64
@@ -23,9 +20,9 @@ except ImportError:
    css_compress = lambda x: x

 from .msg import *
-from common.textutil import ensure_unicode
-from common.progress import ProgressReporter
-from common.timer import timing
+from .common.textutil import ensure_unicode, get_file_b64
+from .common.progress import ProgressReporter
+from .common.timer import timing
 from .smiley import SmileyProvider
 from .msgslice import MessageSlicerByTime, MessageSlicerBySize

@@ -33,10 +30,14 @@ TEMPLATES_FILES = {TYPE_MSG: "TP_MSG",
                   TYPE_IMG: "TP_IMG",
                   TYPE_SPEAK: "TP_SPEAK",
                   TYPE_EMOJI: "TP_EMOJI",
-                   TYPE_CUSTOM_EMOJI: "TP_IMG",
-                   TYPE_LINK: "TP_MSG"}
-TEMPLATES = {k: ensure_unicode(open(os.path.join(STATIC_PATH, '{}.html'.format(v))).read())
-    for k, v in TEMPLATES_FILES.iteritems()}
+                   TYPE_CUSTOM_EMOJI: "TP_EMOJI",
+                   TYPE_LINK: "TP_MSG",
+                   TYPE_VIDEO_FILE: "TP_VIDEO_FILE"
+                  }
+TEMPLATES = {
+    k: open(os.path.join(STATIC_PATH, '{}.html'.format(v))).read()
+    for k, v in TEMPLATES_FILES.items()
+}

 class HTMLRender(object):
    def __init__(self, parser, res=None):
@@ -95,6 +96,11 @@ class HTMLRender(object):
        sender = u'you ' + msg.talker if not msg.isSend else 'me'
        format_dict = {'sender_label': sender,
                       'time': msg.createTime }
+        if(not msg.isSend and msg.is_chatroom()):
+            format_dict['nickname'] = '>\n       <pre align=\'left\'>'+msg.talker_nickname+'</pre'
+        else:
+            format_dict['nickname'] = ' '
+
        def fallback():
            template = TEMPLATES[TYPE_MSG]
            content = msg.msg_str()
@@ -122,16 +128,21 @@ class HTMLRender(object):
            # TODO do not show fancybox when no bigimg found
            format_dict['img'] = (img, 'jpeg')
            return template.format(**format_dict)
-        elif msg.type == TYPE_EMOJI:
-            md5 = msg.imgPath
-            emoji_img, format = self.res.get_emoji_by_md5(md5)
-            format_dict['emoji_format'] = format
-            format_dict['emoji_img'] = emoji_img
-            return template.format(**format_dict)
-        elif msg.type == TYPE_CUSTOM_EMOJI:
-            pq = PyQuery(msg.content)
-            md5 = pq('emoticonmd5').text()
-            format_dict['img'] = self.res.get_emoji(md5, None)
+        elif msg.type == TYPE_EMOJI or msg.type == TYPE_CUSTOM_EMOJI:
+            if 'emoticonmd5' in msg.content:
+                pq = PyQuery(msg.content)
+                md5 = pq('emoticonmd5').text()
+            else:
+                md5 = msg.imgPath
+                # TODO md5 could exist in both.
+                # first is emoji md5, second is image2/ md5
+                # can use fallback here.
+            if md5:
+                emoji_img, format = self.res.get_emoji_by_md5(md5)
+                format_dict['emoji_format'] = format
+                format_dict['emoji_img'] = emoji_img
+            else:
+                import IPython as IP; IP.embed()
            return template.format(**format_dict)
        elif msg.type == TYPE_LINK:
            content = msg.msg_str()
@@ -141,6 +152,20 @@ class HTMLRender(object):
                content = u'URL:<a target="_blank" href="{0}">{0}</a>'.format(url)
                format_dict['content'] = content
                return template.format(**format_dict)
+        elif msg.type == TYPE_VIDEO_FILE:
+            video = self.res.get_video(msg.imgPath)
+            if video.endswith(".mp4"):
+                video_str = get_file_b64(video)
+                format_dict["video_str"] = video_str
+                return template.format(**format_dict)
+            elif video.endswith(".jpg"):
+                # only has thumbnail
+                image_str = get_file_b64(video)
+                format_dict["img"] = (image_str, 'jpeg')
+                return TEMPLATES[TYPE_IMG].format(**format_dict)
+            # fallback
+            format_dict['content'] = f"VIDEO FILE {msg.imgPath}"
+            return TEMPLATES_FILES[TYPE_MSG].format(**format_dict)
        elif msg.type == TYPE_WX_VIDEO:
            # TODO: fetch video from resource
            return fallback()
@@ -148,7 +173,7 @@ class HTMLRender(object):

    def _render_partial_msgs(self, msgs):
        """ return single html"""
-        self.smiley.used_smiley_id.clear()
+        self.smiley.reset()
        slicer = MessageSlicerByTime()
        slices = slicer.slice(msgs)

@@ -167,7 +192,7 @@ class HTMLRender(object):
        # string operation is extremely slow
        return self.html.format(extra_css=self.all_css,
                            extra_js=self.all_js,
-                            chat=msgs[0].chat,
+                            chat=msgs[0].chat_nickname,
                            messages=u''.join(blocks)
                           )

@@ -177,23 +202,21 @@ class HTMLRender(object):
        css = avatar_tpl.format(name='me', avatar=my_avatar)

        for talker in talkers:
-            avatar = self.res.get_contact_avatar(talker)
+            avatar = self.res.get_avatar(talker)
            css += avatar_tpl.format(name=talker, avatar=avatar)
        self.css_string.append(css)

    def render_msgs(self, msgs):
        """ render msgs of one chat, return a list of html"""
-        chat = msgs[0].chat
        if msgs[0].is_chatroom():
-            talkers = set()
-            for msg in msgs:
-                talkers.add(msg.talker)
+            talkers = set([m.talker for m in msgs])
        else:
-            talkers = set([chat])
+            talkers = set([msgs[0].talker])
        self.prepare_avatar_css(talkers)

        self.res.cache_voice_mp3(msgs)

+        chat = msgs[0].chat_nickname
        logger.info(u"Rendering {} messages of {}".format(
            len(msgs), chat))

@@ -1,93 +1,61 @@
-#!/usr/bin/env python2
 # -*- coding: UTF-8 -*-
-# File: res.py
-# Date: Thu Jun 18 00:02:21 2015 +0800
-# Author: Yuxin Wu <[email protected]>

 import glob
 import os
 import re
-# TODO: perhaps we don't need to introduce PIL and numpy. libjpeg might be enough
 from PIL import Image
-import cStringIO
+import io
 import base64
 import logging
 logger = logging.getLogger(__name__)
 import imghdr
 from multiprocessing import Pool
 import atexit
-import cPickle as pickle
-import requests

+from .emoji import EmojiReader
 from .avatar import AvatarReader
-from common.textutil import md5, get_file_b64
-from common.timer import timing
+from .common.textutil import md5 as get_md5_hex, get_file_b64
+from .common.timer import timing
 from .msg import TYPE_SPEAK
 from .audio import parse_wechat_audio_file

 LIB_PATH = os.path.dirname(os.path.abspath(__file__))
-INTERNAL_EMOJI_DIR = os.path.join(LIB_PATH, 'static', 'internal_emoji')
 VOICE_DIRNAME = 'voice2'
 IMG_DIRNAME = 'image2'
 EMOJI_DIRNAME = 'emoji'
-AVATAR_DIRNAME = 'sfs'
+VIDEO_DIRNAME = 'video'

 JPEG_QUALITY = 50

-class EmojiCache(object):
-    def __init__(self, fname):
-        self.fname = fname
-        if os.path.isfile(fname):
-            self.dic = pickle.load(open(fname))
-        else:
-            self.dic = {}
-
-    def query(self, md5):
-        return self.dic.get(md5, (None, None))
-
-    def fetch(self, md5, url):
-        try:
-            logger.info("Requesting emoji {} from {} ...".format(md5, url))
-            r = requests.get(url).content
-            im = Image.open(cStringIO.StringIO(r))
-            format = im.format.lower()
-            ret = (base64.b64encode(r), format)
-            self.dic[md5] = ret
-            self.flush()
-            return ret
-        except Exception as e:
-            logger.exception("Error processing emoji from {}".format(url))
-            return None, None
-
-    def flush(self):
-        with open(self.fname, 'wb') as f:
-            pickle.dump(self.dic, f)
-
 class Resource(object):
    """ multimedia resources in chat"""
-    def __init__(self, parser, res_dir, avt_db,
-            emoji_cache_file='emoji.cache'):
+    def __init__(self, parser, res_dir, avt_db):
+        """
+        Args:
+            res_dir: path to the resource directory
+            avt_db: "avatar.index" file that only exists in old versions of wechat
+        """
        def check(subdir):
-            assert os.path.isdir(os.path.join(res_dir, subdir)), \
-                    "No such directory: {}".format(subdir)
-        [check(k) for k in ['', AVATAR_DIRNAME, IMG_DIRNAME, EMOJI_DIRNAME, VOICE_DIRNAME]]
+            dir_to_check = os.path.join(res_dir, subdir)
+            assert os.path.isdir(dir_to_check), f"No such directory: {dir_to_check}"
+        [check(k) for k in ['', IMG_DIRNAME, EMOJI_DIRNAME, VOICE_DIRNAME]]

-        self.emoji_cache = EmojiCache(emoji_cache_file)
        self.res_dir = res_dir
        self.parser = parser
        self.voice_cache_idx = {}
        self.img_dir = os.path.join(res_dir, IMG_DIRNAME)
        self.voice_dir = os.path.join(res_dir, VOICE_DIRNAME)
-        self.emoji_dir = os.path.join(res_dir, EMOJI_DIRNAME)
-        self.avt_reader = AvatarReader(os.path.join(res_dir, AVATAR_DIRNAME), avt_db)
+        self.video_dir = os.path.join(res_dir, VIDEO_DIRNAME)
+        self.avt_reader = AvatarReader(res_dir, avt_db)
+        self.emoji_reader = EmojiReader(res_dir, self.parser)

-    def get_voice_filename(self, imgpath):
-        fname = md5(imgpath)
+    def _get_voice_filename(self, imgpath):
+        fname = get_md5_hex(imgpath.encode('ascii'))
        dir1, dir2 = fname[:2], fname[2:4]
        ret = os.path.join(self.voice_dir, dir1, dir2,
                           'msg_{}.amr'.format(imgpath))
        if not os.path.isfile(ret):
-            logger.error("Voice file not found for {}".format(imgpath))
+            logger.error(f"Cannot find voice file {imgpath}, {fname}")
            return ""
        return ret

@@ -96,34 +64,37 @@ class Resource(object):
        idx = self.voice_cache_idx.get(imgpath)
        if idx is None:
            return parse_wechat_audio_file(
-                self.get_voice_filename(imgpath))
+                self._get_voice_filename(imgpath))
        return self.voice_cache[idx].get()

    def cache_voice_mp3(self, msgs):
        """ for speed.
        msgs: a collection of WeChatMsg, to cache for later fetch"""
        voice_paths = [msg.imgPath for msg in msgs if msg.type == TYPE_SPEAK]
+        # NOTE: remove all the caching code to debug serial decoding
        self.voice_cache_idx = {k: idx for idx, k in enumerate(voice_paths)}
        pool = Pool(3)
        atexit.register(lambda x: x.terminate(), pool)
        self.voice_cache = [pool.apply_async(parse_wechat_audio_file,
-                                             (self.get_voice_filename(k),)) for k in voice_paths]
-# single-threaded version, for debug
-        #self.voice_cache = map(parse_wechat_audio_file,
-                             #(self.get_voice_filename(k) for k in voice_paths))
+                                             (self._get_voice_filename(k),)) for k in voice_paths]

    def get_avatar(self, username):
-        """ return base64 string"""
+        """ return base64 unicode string"""
        im = self.avt_reader.get_avatar(username)
        if im is None:
+            logger.warning(f"Cannot find avatar for {username}.")
            return ""
-        buf = cStringIO.StringIO()
-        im.save(buf, 'JPEG', quality=JPEG_QUALITY)
+        buf = io.BytesIO()
+        try:
+            im.save(buf, 'JPEG', quality=JPEG_QUALITY)
+        except IOError:
+            try:
+                # sometimes it works the second time...
+                im.save(buf, 'JPEG', quality=JPEG_QUALITY)
+            except IOError:
+                return ""
        jpeg_str = buf.getvalue()
-        return base64.b64encode(jpeg_str)
-
-    def get_contact_avatar(self, nickname):
-        return self.get_avatar(self.parser.contacts_rev[nickname])
+        return base64.b64encode(jpeg_str).decode('ascii')

    def _get_img_file(self, fnames):
        """ fnames: a list of filename to search for
@@ -159,12 +130,11 @@ class Resource(object):
                logger.warn("Found big image but not thumbnail: {}".format(fname))
                return (name, "")
        big = cands[-1]
-        ths = filter(name_is_thumbnail, [k[0] for k in cands])
+        ths = list(filter(name_is_thumbnail, [k[0] for k in cands]))
        if not ths:
            return (big[0], "")
        return (big[0], ths[0])

-
    def get_img(self, fnames):
        """
        :params fnames: possible file paths
@@ -177,65 +147,35 @@ class Resource(object):
            if not img_file:
                return None
            if not img_file.endswith('jpg') and \
-               imghdr.what(img_file) != 'jpeg':
-                im = Image.open(open(img_file, 'rb'))
-                buf = cStringIO.StringIO()
+                   imghdr.what(img_file) != 'jpeg':
+                try:
+                    im = Image.open(open(img_file, 'rb'))
+                except:
+                    return None
+                buf = io.BytesIO()
                im.convert('RGB').save(buf, 'JPEG', quality=JPEG_QUALITY)
-                return base64.b64encode(buf.getvalue())
+                return base64.b64encode(buf.getvalue()).decode('ascii')
+            with open(img_file, 'rb') as f:
+                if f.read(4) == b'wxgf':
+                    logger.warning(f"Don't know how to decode wxgf image {img_file}")
+                    return None
            return get_file_b64(img_file)
+
        big_file = get_jpg_b64(big_file)
        if big_file:
            return big_file
        return get_jpg_b64(small_file)

-    def _get_res_emoji(self, md5, pack_id):
-        path = self.emoji_dir
-        if pack_id:
-            path = os.path.join(path, pack_id)
-        candidates = glob.glob(os.path.join(path, '{}*'.format(md5)))
-        candidates = [k for k in candidates if not k.endswith('_thumb') \
-                and not re.match('.*_[0-9]+$', k)]
-
-        def try_use(f):
-            if not f: return None
-            if not imghdr.what(f[0]):   # cannot recognize file type
-                return None
-            return f[0]
-
-        f = try_use([k for k in candidates if not k.endswith('_cover')])
-        if f:
-            return get_file_b64(f), imghdr.what(f)
-
-        f = try_use([k for k in candidates if k.endswith('_cover')])
-        if f:
-            return get_file_b64(f), imghdr.what(f)
-        return None, None
-
-    def _get_internal_emoji(self, fname):
-        f = os.path.join(INTERNAL_EMOJI_DIR, fname)
-        return get_file_b64(f), imghdr.what(f)
-
    def get_emoji_by_md5(self, md5):
-        """ :returns: (b64 img, format)"""
-        if md5 in self.parser.internal_emojis:
-            emoji_img, format = self._get_internal_emoji(self.parser.internal_emojis[md5])
-            logger.warn("Cannot get emoji {}".format(md5))
-            return None, None
-        else:
-            img, format = self.emoji_cache.query(md5)
-            if format:
-                return img, format
-            group = self.parser.emoji_groups.get(md5, None)
-            emoji_img, format = self._get_res_emoji(md5, group)
-            if format:
-                return emoji_img, format
-            url = self.parser.emoji_url.get(md5, None)
-            if url:
-                emoji_img, format = self.emoji_cache.fetch(md5, url)
-                if format:
-                    return emoji_img, format
-
-            logger.warn("Cannot get emoji {} in {}".format(md5, group))
-            return None, None
-
+        """ Returns: (b64 encoded img string, format) """
+        return self.emoji_reader.get_emoji(md5)

+    def get_video(self, videoid):
+        video_file = os.path.join(self.video_dir, videoid + ".mp4")
+        video_thumbnail_file = os.path.join(self.video_dir, videoid + ".jpg")
+        if os.path.exists(video_file):
+            return video_file
+        elif os.path.exists(video_thumbnail_file):
+            return video_thumbnail_file
+        logger.warning(f"Cannot find video {videoid}")
+        return ""
@@ -1,21 +1,16 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # -*- coding: UTF-8 -*-
-# File: smiley.py
-# Date: Thu Jun 18 00:02:43 2015 +0800
-# Author: Yuxin Wu <[email protected]>

 import os
+import functools
 import re
 import json
 import struct

-from common.textutil import get_file_b64
+from .common.textutil import get_file_b64

 STATIC_PATH = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'static')
-
-UNICODE_SMILEY_FILE = os.path.join(STATIC_PATH, 'unicode-smiley.json')
 TENCENT_SMILEY_FILE = os.path.join(STATIC_PATH, 'tencent-smiley.json')
-TENCENT_EXTRASMILEY_FILE = os.path.join(STATIC_PATH, 'tencent-smiley-extra.json')

 try:
    UNICODE_SMILEY_RE = re.compile(
@@ -40,10 +35,16 @@ HEAD = """.smiley {
 }
 """

-TEMPLATE = """.smiley{name} {{
+TEMPLATE = """.{name} {{
    background-image: url("data:image/png;base64,{b64}");
+    background-size: 24px 24px;
 }}"""

+def _css_class_name(s):
+    s = s.replace("/", "_")
+    s = s.replace(".", "_")
+    return "smiley_" + s
+
 class SmileyProvider(object):
    def __init__(self, html_replace=True):
        """ html_replace: replace smileycode by html.
@@ -53,74 +54,40 @@ class SmileyProvider(object):
        if not html_replace:
            raise NotImplementedError()

-        # [微笑] -> 0
+        # [微笑] -> smiley/0.png
        self.tencent_smiley = json.load(open(TENCENT_SMILEY_FILE))
+        self.used_smileys = set()

-        # some extra smiley from javascript on wx.qq.com
-        extra_smiley = json.load(open(TENCENT_EXTRASMILEY_FILE))
-        extra_smiley = {u'[' + k + u']': v for k, v in
-                            extra_smiley.iteritems()}
-        self.tencent_smiley.update(extra_smiley)
+    def reset(self):
+        self.used_smileys.clear()

-        # 1f35c -> "\ue340"
-        #self.unicode_smiley_code = gUnicodeCodeMap
-
-        # u'\U0001f35c' -> "e340"   # for iphone
-        # u'\ue415' -> 'e415'       # for android
-        unicode_smiley_dict = json.load(open(UNICODE_SMILEY_FILE))
-        self.unicode_smiley = {(self.unichar(int(k, 16))): hex(ord(v))[2:] for k, v in
-                                unicode_smiley_dict.iteritems()}
-        self.unicode_smiley.update({v: hex(ord(v))[2:] for _, v in
-                                unicode_smiley_dict.iteritems()})
-        self.used_smiley_id = set()
-
-    def unichar(self, i):
-        try:
-            return unichr(i)
-        except ValueError:
-            return struct.pack('i', i).decode('utf-32')
-
-    def gen_replace_elem(self, smiley_id):
-        self.used_smiley_id.add(str(smiley_id))
-        return '<span class="smiley smiley{}"></span>'.format(smiley_id)
-
-    def _replace_unicode(self, msg):
-        if not UNICODE_SMILEY_RE.findall(msg):
-        # didn't find the code
-            return msg
-        for k, v in self.unicode_smiley.iteritems():
-            if k in msg:
-                msg = msg.replace(k, self.gen_replace_elem(v))
-        return msg
-
-    def _replace_tencent(self, msg):
-        if (not '[' in msg or not ']' in msg) \
-           and (not '/:' in msg) and (not '/' in msg):
-            return msg
-        for k, v in self.tencent_smiley.iteritems():
-            if k in msg:
-                msg = msg.replace(k, self.gen_replace_elem(v))
-        return msg
+    def gen_replace_elem(self, smiley_path):
+        self.used_smileys.add(str(smiley_path))
+        return '<span class="smiley {}"></span>'.format(_css_class_name(smiley_path))

    def replace_smileycode(self, msg):
        """ replace the smiley code in msg
            return a html
        """
-        msg = self._replace_unicode(msg)
-        msg = self._replace_tencent(msg)
+        # pre-filter:
+        if ('[' not in msg) and ('/' not in msg) and not UNICODE_SMILEY_RE.findall(msg):
+            return msg
+        for k, v in self.tencent_smiley.items():
+            if k in msg:
+                msg = msg.replace(k, self.gen_replace_elem(v))
+        return msg
        return msg

    def gen_used_smiley_css(self):
        ret = HEAD
-        for sid in self.used_smiley_id:
-            fname = os.path.join(STATIC_PATH, 'smileys', '{}.png'.format(sid))
+        for path in self.used_smileys:
+            fname = os.path.join(STATIC_PATH, path)
            b64 = get_file_b64(fname)
-            ret = ret + TEMPLATE.format(name=sid, b64=b64)
+            ret = ret + TEMPLATE.format(name=_css_class_name(path), b64=b64)
        return ret

 if __name__ == '__main__':
    smiley = SmileyProvider()
    msg = u"[挥手]哈哈呵呵ｈｉｈｉ\U0001f684\u2728\u0001 /::<\ue415"
    msg = smiley.replace_smileycode(msg)
-    #print msg
    smiley.gen_used_smiley_css()
@@ -0,0 +1,10 @@
+
+Sources:
+
+* web wechat
+* /data/data/com.tencent.mm/files/public/emoji/newemoji
+
+
+Generated file:
+
+* tencent-smiley.json generated by  `parse_tencent_smiley.py`
@@ -2,7 +2,7 @@
  <div class="chatItemContent">
    <span class="avatar"></span>
    <div class="cloud cloudImg">
-      <div class="cloudPannel" title="{time}">
+      <div class="cloudPannel" title="{time}" {nickname}>
        <div class="cloudBody">
          <div class="cloudContent">
            <span class="img_wrap">
@@ -2,7 +2,7 @@
  <div class="chatItemContent">
    <span class="avatar"></span>
    <div class="cloud cloudImg">
-      <div class="cloudPannel" title="{time}">
+      <div class="cloudPannel" title="{time}" {nickname}>
        <div class="cloudBody">
          <div class="cloudContent">
            <span class="img_wrap">
@@ -9,8 +9,8 @@

 <body>
  <div id="chat" class="chatPanel normalPanel">
-    <div class="chatMainPanel" id="chatMainPanel" style="padding-top:40px;">
-      <div class="chatTitle" style="margin-top: -40px;">
+    <div class="chatMainPanel" id="chatMainPanel">
+      <div class="chatTitle">
        <div class="chatNameWrap">
          <p class="chatName" id="messagePanelTitle">{chat}</p>
        </div>
@@ -2,7 +2,7 @@
  <div class="chatItemContent">
    <span class="avatar"></span>
    <div class="cloud cloudText">
-      <div class="cloudPannel" title="{time}">
+       <div class="cloudPannel" title="{time}" {nickname}>
        <div class="cloudBody">
          <div class="cloudContent">
            <pre style="white-space:pre-wrap">{content}</pre>
@@ -2,9 +2,9 @@
  <div class="chatItemContent">
    <span class="avatar"></span>
    <div class="cloud cloudVoice" onclick="playVoice(event)" style="width:80px">
-      <div class="cloudPannel" title="{time}">
+      <div class="cloudPannel" title="{time}" {nickname}>
        <div class="sendStatus">
-          <span class="second">{voice_duration}"</span>
+          <span class="second">{voice_duration:.1f}"</span>
        </div>
        <div class="cloudBody">
          <div class="cloudContent">
@@ -0,0 +1,14 @@
+<div class="chatItem {sender_label}">
+  <div class="chatItemContent">
+    <span class="avatar"></span>
+    <div class="cloud cloudVideo">
+      <div class="cloudPannel" title="{time}" {nickname}>
+        <div class="cloudBody">
+            <video controls width="300">
+              <source type="video/mp4" src="data:video/mp4;base64,{video_str}" />
+            </video>
+        </div>
+      </div>
+    </div>
+  </div>
+</div>
@@ -1,6 +1,6 @@
 //File: main.js
 //Date: Sun Jan 11 23:32:26 2015 +0800
-//Author: Yuxin Wu <[email protected]>
+//Author: Yuxin Wu

 var playVoice = function(event) {
  var target = event.target;
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Yuxin Wu	5320456e80	update	2025-01-05 17:03:38 +08:00
JakeandGitHub	5da9994eef	Update README.md (#99 ) old link doesn't work anymore.	2024-08-13 16:33:02 -07:00
Yuxin Wu	baef03db85	fix #90	2023-05-20 18:15:50 -07:00
Yuxin WuandGitHub	3971729641	Update README.md	2021-12-13 16:46:57 -08:00
SeaandGitHub	d4b8cf3530	Update decrypt-db.py (#87 ) Add error output for try block of do_decrypt	2021-08-29 12:51:11 -07:00
Yuxin Wu	d0ec0886c0	check sqlcipher version (fix #85 )	2021-04-23 15:10:33 -07:00
Yuxin Wu	46cda89270	update docs	2021-04-19 01:22:36 -07:00
lqzhgood GitHub lqzh	76f0a8f44e	🌈style(view): fix body scroll (#84 ) Co-authored-by: lqzh <[email protected]>	2021-04-13 15:58:53 -07:00
lqzhgoodandGitHub	1009d8fcb2	Update requirements.txt (#82 ) add `numpy` module	2021-04-13 00:19:28 -07:00
Yuxin Wu	0bd5a7d6f5	update docs about sfs/avatar.index	2021-02-02 11:00:48 -08:00
Yuxin Wu	a98c923d8f	convert javaobj to int	2021-02-01 18:19:34 -08:00
Yuxin Wu	9b1315a9c5	update	2021-01-07 19:12:20 -08:00
Yuxin Wu	6b50191420	uin int to str	2020-11-27 15:34:30 -08:00
Yuxin Wu	7361d53bd1	make uin a str	2020-11-27 15:33:17 -08:00
Yuxin Wu	70f2925026	catch decryption error (#72 )	2020-10-03 04:38:19 -07:00
Yuxin Wu	9d5682485c	fix #70	2020-09-19 11:54:12 -07:00
Yuxin Wu	dfdcd4ba0f	use tempfile for audio decoding	2020-07-17 12:26:25 -07:00
Yuxin Wu	9b64982af0	fix encoding (fix #67 )	2020-07-17 12:06:16 -07:00
Kai JiaandGitHub	c5ee3026f0	Fix a typo in README (#66 )	2020-07-06 10:11:34 -07:00
Yuxin Wu	100b8e2391	update readme	2020-07-02 17:00:46 -07:00
Yuxin Wu	a914018579	update readme	2020-07-01 01:26:15 -07:00
Yuxin Wu	8e8b1d3534	better error message	2020-06-30 23:31:56 -07:00
Yuxin Wu	e1157a18b7	remove "internal_emoji"	2020-06-30 23:16:19 -07:00
Yuxin Wu	641af7a3cd	decrypt local emoji	2020-06-30 23:09:46 -07:00
Yuxin Wu	45c28312d3	check md5 of local emoji as well	2020-06-29 11:21:07 -07:00
Yuxin Wu	146855d1fb	use encrypturl if cdnurl does not produce correct md5	2020-06-29 11:07:44 -07:00
Yuxin Wu	a14e270ff2	remove failed decryption	2020-06-29 03:30:15 -07:00
Yuxin Wu	fd0a065b6e	aes-decrypt emoji from encrypturl	2020-06-28 02:57:14 -07:00
Yuxin Wu	02f81eb505	check for two avatar storage	2020-06-28 00:33:07 -07:00
Yuxin Wu	99c5ed6d33	fix old avatar storage	2020-06-28 00:12:39 -07:00
Yuxin Wu	bb44745bb4	move common/ into wechat/	2020-06-27 20:34:05 -07:00
Yuxin Wu	5f729e4227	support both chatid and nickname (fix #51 )	2020-06-27 20:04:34 -07:00
Yuxin Wu	b272858b1a	avoid assertion for type 49 (close #52 )	2020-06-27 19:56:28 -07:00
Yuxin Wu	58ce6ea853	update readme	2020-06-27 19:45:42 -07:00
Yuxin Wu	ceec29afc9	add missing file	2020-06-27 19:32:21 -07:00
Yuxin Wu	d8ac9d8fad	compatibility with an old database file	2020-06-27 19:00:43 -07:00
Yuxin Wu	b295d62f41	update	2020-06-27 17:34:52 -07:00
Yuxin Wu	394b1d75aa	support latest smileys	2020-06-27 17:31:35 -07:00
Yuxin Wu	5f93b39916	move libchat to legacy	2020-06-27 16:34:35 -07:00
Yuxin Wu	d439dfe72d	handle emoji thumbnail	2020-06-27 16:32:07 -07:00
Yuxin Wu	49c4ba4346	add support for video file	2020-06-27 15:54:03 -07:00
Yuxin Wu	476bdd5971	rewrite with python3	2020-06-27 15:21:17 -07:00
Yuxin Wu	659742c929	better logging	2020-06-27 12:07:54 -07:00
Yuxin Wu	16de676f91	update	2020-06-27 02:21:57 -07:00
Yuxin Wu	b246807304	update	2020-06-27 01:49:45 -07:00
Yuxin Wu	23b797b1f6	rewrite decryption logic	2020-06-27 01:45:15 -07:00
Yuxin Wu	ce84066d85	sqlcipher compatibilty	2020-06-26 19:01:18 -07:00
Yuxin Wu	6bea581fa0	update in parser	2020-06-26 16:53:20 -07:00
Yuxin Wu	2d618281d9	remove my emails	2018-08-26 01:21:45 -07:00
Yuxin Wu	47141e9356	update docs	2017-11-29 04:04:22 -08:00
Yuxin Wu	14ffca8532	update docs	2017-11-29 04:02:49 -08:00
Yuxin Wu	a1fd932a77	use emoji cover for failures	2017-11-29 03:55:09 -08:00
Yuxin Wu	5026cbde84	fix avatar rendering	2017-11-29 03:26:40 -08:00
Yuxin Wu	d5c097277d	update docs	2017-11-29 02:33:15 -08:00
futurewrgandYuxin Wu	dcad5ed99d	Add support for new avatar storage (#50 ) * Add support for new avatar storage,modify the README and android-interact.sh * Show the nickname for chatroom, fix the IOError in avatar.py * style change	2017-11-29 01:34:51 -08:00
Yuxin WuandGitHub	265b57755d	fix dependencies (fix #49 )	2017-10-14 23:55:42 -07:00
Yuxin Wu	7f875c761b	use userid instead of nickname as identifiers everywhere. (#39 )	2017-03-02 17:46:30 +08:00
Yuxin Wu	1406453e47	update README (#39 )	2017-03-02 12:09:30 +08:00
Yuxin Wu	9eecc6dd38	bug fix on chatroom msg handling	2017-02-09 13:24:37 +08:00
Yuxin Wu	080c751eb2	update readme	2016-10-12 14:03:10 -04:00
Yuxin Wu	1e2a625452	bug fix	2016-10-12 13:53:45 -04:00
Yuxin Wu	6ef50f5685	bugfix	2016-10-12 13:53:11 -04:00
Yuxin WuandGitHub	b069338a9f	Update README.md	2016-10-11 12:11:07 -04:00
Yuxin Wu	a7b6d437d3	update readme	2016-10-10 01:19:59 -04:00
Yuxin Wu	32f41ae31d	update readme	2016-10-10 01:19:21 -04:00
Yuxin Wu	eb5066f74a	bug fix	2016-09-06 02:01:32 -04:00
Jakukyo FrielandYuxin Wu	49ba23c3cd	Refine README: (#31 ) * Add commands to install dependencies on debian. * README: mention `requests` in dependencies. Thanks @h82k to point out this in #29 * README: fix broken link. * Improve instructions on uncompress emoji.cache. See #29 * README: remove unnecessary unpack command. > unpacking emoji should not be necessary. > I think things should work without running emoji-cache-tool.py. > That tool is mainly for me to see what's in the cache. -- @ppwwyyxx in #29 * README: fix `dump-html.py` command line. See #29	2016-07-26 16:56:38 -07:00
Yuxin Wu	37dd9f0909	see if readme looks better	2016-07-02 22:54:39 -07:00
Yuxin Wu	870a1852bf	avoid using cover, and use compressed emoji.cache	2016-06-18 23:37:04 -07:00
Yuxin Wu	386cfe39b7	script improvements	2016-06-18 22:24:20 -07:00
Yuxin Wu	76efd3836a	update readme	2016-06-18 10:16:41 -07:00
Yuxin Wu	d0e428f2ac	add help	2016-06-18 10:12:41 -07:00
Yuxin Wu	92e3d3d31c	emoji cache downloadd	2016-06-18 10:04:53 -07:00
Yuxin Wu	98cc7c7fc8	support red envelope	2016-06-17 23:20:08 -07:00