[client-app] Add initial sync benchmarking script

Summary:
We want to be able to benchmark initial sync, so this diff adds two
scripts. The first, drop-stuff.sh, drops all data from the app and sync
databases that isn't related to account credentials. The second,
benchmark-initial-sync.sh, runs a fixed number of iterations (current 5) that
invokes drop-stuff.sh then opens the app, waits a fixed amount of time
(currently 120 seconds), and then kills the app and measures how many messages
it synced which it prints to the console. This is sufficient for us to start
measuring how quickly we can sync messages. This diff also includes the
sqlite3 amalgamation which drop-stuff.sh requires to function correctly
due to depending on the FTS5 extension which doesn't come built-in on
some platforms.

Test Plan: Run benchmark locally

Reviewers: evan, spang, juan

Reviewed By: juan

Differential Revision: https://phab.nylas.com/D4275
This commit is contained in:
Mark Hahnenberg 2017-03-28 12:24:51 -07:00
parent c7e4c2bb87
commit fa50657af1
3 changed files with 106 additions and 0 deletions

3
.gitignore vendored
View file

@ -60,3 +60,6 @@ packages/client-app/spec/isomorphic-core
.elasticbeanstalk/*
!.elasticbeanstalk/*.cfg.yml
!.elasticbeanstalk/*.global.yml
# Sqlite amalgamation for scripts
scripts/sqlite

View file

@ -0,0 +1,53 @@
#!/bin/bash
set -e
function print_help {
OUTPUT=$(cat <<- EOM
A script to benchmark the initial sync performance of Nylas Mail. To use,
simply run the script after authing whatever accounts you wish to measure
in your development version of Nylas Mail. The benchmarking script will
clear all of the data except for your accounts and open and close Nylas
Mail several times, printing out the number of messages synced after each
iteration.
)
echo "$OUTPUT"
}
if [[ $1 == '-h' || $1 == '--help' ]]
then
print_help
exit 0
fi
# Run sudo to prime root privileges
sudo ls > /dev/null
CWD="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
NYLAS_DIR="$HOME/.nylas-dev"
EDGEHILL_DB="$NYLAS_DIR/edgehill.db"
TIME_LIMIT=120
ITERS=5
for i in `seq 1 $ITERS`
do
bash $CWD/drop-data-except-accounts.sh > /dev/null
(npm start &> /dev/null &)
sleep $TIME_LIMIT
ELECTRON_PID=`ps aux | grep "Electron packages/client-app" | grep -v grep | awk '{print $2}'`
sudo kill -9 $ELECTRON_PID
MESSAGE_COUNT=`sqlite3 $EDGEHILL_DB 'SELECT COUNT(*) FROM Message'`
echo "Synced Messages: $MESSAGE_COUNT"
# Sometimes it takes a while to shutdown
while [[ $ELECTRON_PID != '' ]]
do
sleep 1
ELECTRON_PID=`ps aux | grep "Electron packages/client-app" | grep -v grep | awk '{print $2}'`
done
done

View file

@ -0,0 +1,50 @@
#!/bin/bash
set -e
CWD="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
NYLAS_DIR="$HOME/.nylas-dev"
TRUNCATE_TABLES="
Account
AccountPluginMetadata
Calendar
Category
Contact
ContactSearch
Event
EventSearch
File
Message
MessageBody
MessagePluginMetadata
ProviderSyncbackRequest
Thread
ThreadCategory
ThreadContact
ThreadCounts
ThreadPluginMetadata
ThreadSearch"
SQLITE_DIR=$CWD/sqlite
SQLITE_BIN=$SQLITE_DIR/sqlite3
SQLITE_SRC_DIR="$SQLITE_DIR/sqlite-amalgamation-3170000"
# Build the sqlite3 amalgamation if necessary because we need FTS5 support.
if [[ ! -e "$SQLITE_BIN" ]]
then
mkdir -p $SQLITE_DIR
curl -s "https://www.sqlite.org/2017/sqlite-amalgamation-3170000.zip" > "$SQLITE_DIR/sqlite-amalgamation.zip"
unzip -o -d $SQLITE_DIR "$SQLITE_DIR/sqlite-amalgamation.zip"
clang -DSQLITE_ENABLE_FTS5 "$SQLITE_SRC_DIR/sqlite3.c" "$SQLITE_SRC_DIR/shell.c" -o $SQLITE_BIN
fi
rm -f $NYLAS_DIR/a-*.sqlite
EDGEHILL_DB=$NYLAS_DIR/edgehill.db
for TABLE in $TRUNCATE_TABLES
do
COMMAND="DELETE FROM $TABLE"
$SQLITE_BIN $EDGEHILL_DB "DELETE FROM $TABLE"
done
$SQLITE_BIN $EDGEHILL_DB 'DELETE FROM JSONBlob WHERE client_id != "NylasID"'