Mailspring/spec/stores/thread-counts-store-spec.coffee

265 lines
10 KiB
CoffeeScript
Raw Normal View History

feat(counts): Unread counts for all folders and labels across all accounts Summary: This diff replaces the UnreadCountStore with a better approach that is able to track unread counts for all folders/labels without continuous (and cripplingly slow) SELECT COUNT(*) queries. When models are written to the database, we currently don't send out notifications with the "previous" state of those objects in the database. This makes it hard to determine how to update counters. (In the future, we may need to do this for live queries). Unfortunately, getting the "previous" state is going to be very hard, because multiple windows write to the database and the "previous" state we have might be outdated. We'd almost have to run a "SELECT" right before every "REPLACE INTO". I created an API that allows you to register observers around persistModel and unpersistModel. With this API, you can run queries before and after the database changes are made and pluck just the "before" state you're interested in. The `ThreadCountsStore` uses this API to determine the impact of persisting a set of threads on the unread counts of different labels. Before the threads are saved, it says "how much do these thread IDs contribute to unread counts currently?". After the write is complete it looks at the models and computes the difference between the old count impact and the new count impact, and updates the counters. I decided not to attach the unread count to the Label objects themselves because 1) they update frequently and 2) most things observing the DatabaseStore for categories do not care about counts, so they would be updating unnecessarily. The AccountSidebar now listens to the ThreadCountsStore as well as the CategoryStore, and there's a new preference in the General tab for turning off the counts. Test Plan: Tests are a work in progress, want to get feedback first! Reviewers: juan, evan Reviewed By: evan Differential Revision: https://phab.nylas.com/D2232
2015-11-24 09:12:22 +08:00
_ = require 'underscore'
DatabaseStore = require '../../src/flux/stores/database-store'
feat(transactions): Explicit (and faster) database transactions Summary: Until now, we've been hiding transactions beneath the surface. When you call persistModel, you're implicitly creating a transaction. You could explicitly create them with `atomically`..., but there were several critical problems that are fixed in this diff: - Calling persistModel / unpersistModel within a transaction could cause the DatabaseStore to trigger. This could result in other parts of the app making queries /during/ the transaction, potentially before the COMMIT occurred and saved the changes. The new, explicit inTransaction syntax holds all changes until after COMMIT and then triggers. - Calling atomically and then calling persistModel inside that resulted in us having to check whether a transaction was present and was gross. - Many parts of the code ran extensive logic inside a promise chained within `atomically`: BAD: ``` DatabaseStore.atomically => DatabaseStore.persistModel(draft) => GoMakeANetworkRequestThatReturnsAPromise ``` OVERWHELMINGLY BETTER: ``` DatabaseStore.inTransaction (t) => t.persistModel(draft) .then => GoMakeANetworkRequestThatReturnsAPromise ``` Having explicit transactions also puts us on equal footing with Sequelize and other ORMs. Note that you /have/ to call DatabaseStore.inTransaction (t) =>. There is no other way to access the methods that let you alter the database. :-) Other changes: - This diff removes Message.labels and the Message-Labels table. We weren't using Message-level labels anywhere, and the table could grow very large. - This diff changes the page size during initial sync from 250 => 200 in an effort to make transactions a bit faster. Test Plan: Run tests! Reviewers: juan, evan Reviewed By: juan, evan Differential Revision: https://phab.nylas.com/D2353
2015-12-18 03:46:05 +08:00
DatabaseTransaction = require '../../src/flux/stores/database-transaction'
feat(counts): Unread counts for all folders and labels across all accounts Summary: This diff replaces the UnreadCountStore with a better approach that is able to track unread counts for all folders/labels without continuous (and cripplingly slow) SELECT COUNT(*) queries. When models are written to the database, we currently don't send out notifications with the "previous" state of those objects in the database. This makes it hard to determine how to update counters. (In the future, we may need to do this for live queries). Unfortunately, getting the "previous" state is going to be very hard, because multiple windows write to the database and the "previous" state we have might be outdated. We'd almost have to run a "SELECT" right before every "REPLACE INTO". I created an API that allows you to register observers around persistModel and unpersistModel. With this API, you can run queries before and after the database changes are made and pluck just the "before" state you're interested in. The `ThreadCountsStore` uses this API to determine the impact of persisting a set of threads on the unread counts of different labels. Before the threads are saved, it says "how much do these thread IDs contribute to unread counts currently?". After the write is complete it looks at the models and computes the difference between the old count impact and the new count impact, and updates the counters. I decided not to attach the unread count to the Label objects themselves because 1) they update frequently and 2) most things observing the DatabaseStore for categories do not care about counts, so they would be updating unnecessarily. The AccountSidebar now listens to the ThreadCountsStore as well as the CategoryStore, and there's a new preference in the General tab for turning off the counts. Test Plan: Tests are a work in progress, want to get feedback first! Reviewers: juan, evan Reviewed By: evan Differential Revision: https://phab.nylas.com/D2232
2015-11-24 09:12:22 +08:00
ThreadCountsStore = require '../../src/flux/stores/thread-counts-store'
Thread = require '../../src/flux/models/thread'
Folder = require '../../src/flux/models/folder'
Label = require '../../src/flux/models/label'
Matcher = require '../../src/flux/attributes/matcher'
WindowBridge = require '../../src/window-bridge'
feat(counts): Unread counts for all folders and labels across all accounts Summary: This diff replaces the UnreadCountStore with a better approach that is able to track unread counts for all folders/labels without continuous (and cripplingly slow) SELECT COUNT(*) queries. When models are written to the database, we currently don't send out notifications with the "previous" state of those objects in the database. This makes it hard to determine how to update counters. (In the future, we may need to do this for live queries). Unfortunately, getting the "previous" state is going to be very hard, because multiple windows write to the database and the "previous" state we have might be outdated. We'd almost have to run a "SELECT" right before every "REPLACE INTO". I created an API that allows you to register observers around persistModel and unpersistModel. With this API, you can run queries before and after the database changes are made and pluck just the "before" state you're interested in. The `ThreadCountsStore` uses this API to determine the impact of persisting a set of threads on the unread counts of different labels. Before the threads are saved, it says "how much do these thread IDs contribute to unread counts currently?". After the write is complete it looks at the models and computes the difference between the old count impact and the new count impact, and updates the counters. I decided not to attach the unread count to the Label objects themselves because 1) they update frequently and 2) most things observing the DatabaseStore for categories do not care about counts, so they would be updating unnecessarily. The AccountSidebar now listens to the ThreadCountsStore as well as the CategoryStore, and there's a new preference in the General tab for turning off the counts. Test Plan: Tests are a work in progress, want to get feedback first! Reviewers: juan, evan Reviewed By: evan Differential Revision: https://phab.nylas.com/D2232
2015-11-24 09:12:22 +08:00
describe "ThreadCountsStore", ->
describe "unreadCountForCategoryId", ->
it "returns null if no count exists for the category id", ->
expect(ThreadCountsStore.unreadCountForCategoryId('nan')).toBe(null)
it "returns the count plus any unsaved deltas", ->
ThreadCountsStore._counts =
'b': 3
'a': 5
ThreadCountsStore._deltas =
'a': -1
expect(ThreadCountsStore.unreadCountForCategoryId('a')).toBe(4)
expect(ThreadCountsStore.unreadCountForCategoryId('b')).toBe(3)
describe "when the mutation observer reports count changes", ->
describe "in the work window", ->
beforeEach ->
spyOn(NylasEnv, 'isWorkWindow').andReturn(true)
it "should merge count deltas into existing count detlas", ->
ThreadCountsStore._deltas =
'l1': -1
'l2': 2
ThreadCountsStore._onCountsChanged({'l1': -1, 'l2': 1, 'l3': 2})
expect(ThreadCountsStore._deltas).toEqual({
'l1': -2,
'l2': 3,
'l3': 2
})
feat(counts): Unread counts for all folders and labels across all accounts Summary: This diff replaces the UnreadCountStore with a better approach that is able to track unread counts for all folders/labels without continuous (and cripplingly slow) SELECT COUNT(*) queries. When models are written to the database, we currently don't send out notifications with the "previous" state of those objects in the database. This makes it hard to determine how to update counters. (In the future, we may need to do this for live queries). Unfortunately, getting the "previous" state is going to be very hard, because multiple windows write to the database and the "previous" state we have might be outdated. We'd almost have to run a "SELECT" right before every "REPLACE INTO". I created an API that allows you to register observers around persistModel and unpersistModel. With this API, you can run queries before and after the database changes are made and pluck just the "before" state you're interested in. The `ThreadCountsStore` uses this API to determine the impact of persisting a set of threads on the unread counts of different labels. Before the threads are saved, it says "how much do these thread IDs contribute to unread counts currently?". After the write is complete it looks at the models and computes the difference between the old count impact and the new count impact, and updates the counters. I decided not to attach the unread count to the Label objects themselves because 1) they update frequently and 2) most things observing the DatabaseStore for categories do not care about counts, so they would be updating unnecessarily. The AccountSidebar now listens to the ThreadCountsStore as well as the CategoryStore, and there's a new preference in the General tab for turning off the counts. Test Plan: Tests are a work in progress, want to get feedback first! Reviewers: juan, evan Reviewed By: evan Differential Revision: https://phab.nylas.com/D2232
2015-11-24 09:12:22 +08:00
it "should queue a save of the counts", ->
spyOn(ThreadCountsStore, '_saveCountsSoon')
ThreadCountsStore._onCountsChanged({'l1': -1, 'l2': 1, 'l3': 2})
expect(ThreadCountsStore._saveCountsSoon).toHaveBeenCalled()
describe "in other windows", ->
beforeEach ->
spyOn(NylasEnv, 'isWorkWindow').andReturn(false)
it "should use the WindowBridge to forward the invocation to the work window", ->
spyOn(WindowBridge, 'runInWorkWindow')
payload = {'l1': -1, 'l2': 1, 'l3': 2}
ThreadCountsStore._onCountsChanged(payload)
expect(WindowBridge.runInWorkWindow).toHaveBeenCalledWith('ThreadCountsStore', '_onCountsChanged', [payload])
feat(counts): Unread counts for all folders and labels across all accounts Summary: This diff replaces the UnreadCountStore with a better approach that is able to track unread counts for all folders/labels without continuous (and cripplingly slow) SELECT COUNT(*) queries. When models are written to the database, we currently don't send out notifications with the "previous" state of those objects in the database. This makes it hard to determine how to update counters. (In the future, we may need to do this for live queries). Unfortunately, getting the "previous" state is going to be very hard, because multiple windows write to the database and the "previous" state we have might be outdated. We'd almost have to run a "SELECT" right before every "REPLACE INTO". I created an API that allows you to register observers around persistModel and unpersistModel. With this API, you can run queries before and after the database changes are made and pluck just the "before" state you're interested in. The `ThreadCountsStore` uses this API to determine the impact of persisting a set of threads on the unread counts of different labels. Before the threads are saved, it says "how much do these thread IDs contribute to unread counts currently?". After the write is complete it looks at the models and computes the difference between the old count impact and the new count impact, and updates the counters. I decided not to attach the unread count to the Label objects themselves because 1) they update frequently and 2) most things observing the DatabaseStore for categories do not care about counts, so they would be updating unnecessarily. The AccountSidebar now listens to the ThreadCountsStore as well as the CategoryStore, and there's a new preference in the General tab for turning off the counts. Test Plan: Tests are a work in progress, want to get feedback first! Reviewers: juan, evan Reviewed By: evan Differential Revision: https://phab.nylas.com/D2232
2015-11-24 09:12:22 +08:00
describe "when counts are persisted", ->
it "should update it's _counts cache and trigger", ->
newCounts = {
'abc': 1
}
spyOn(ThreadCountsStore, 'trigger')
ThreadCountsStore._onCountsBlobRead(newCounts)
feat(counts): Unread counts for all folders and labels across all accounts Summary: This diff replaces the UnreadCountStore with a better approach that is able to track unread counts for all folders/labels without continuous (and cripplingly slow) SELECT COUNT(*) queries. When models are written to the database, we currently don't send out notifications with the "previous" state of those objects in the database. This makes it hard to determine how to update counters. (In the future, we may need to do this for live queries). Unfortunately, getting the "previous" state is going to be very hard, because multiple windows write to the database and the "previous" state we have might be outdated. We'd almost have to run a "SELECT" right before every "REPLACE INTO". I created an API that allows you to register observers around persistModel and unpersistModel. With this API, you can run queries before and after the database changes are made and pluck just the "before" state you're interested in. The `ThreadCountsStore` uses this API to determine the impact of persisting a set of threads on the unread counts of different labels. Before the threads are saved, it says "how much do these thread IDs contribute to unread counts currently?". After the write is complete it looks at the models and computes the difference between the old count impact and the new count impact, and updates the counters. I decided not to attach the unread count to the Label objects themselves because 1) they update frequently and 2) most things observing the DatabaseStore for categories do not care about counts, so they would be updating unnecessarily. The AccountSidebar now listens to the ThreadCountsStore as well as the CategoryStore, and there's a new preference in the General tab for turning off the counts. Test Plan: Tests are a work in progress, want to get feedback first! Reviewers: juan, evan Reviewed By: evan Differential Revision: https://phab.nylas.com/D2232
2015-11-24 09:12:22 +08:00
expect(ThreadCountsStore._counts).toEqual(newCounts)
expect(ThreadCountsStore.trigger).toHaveBeenCalled()
describe "_fetchCountsMissing", ->
beforeEach ->
ThreadCountsStore._categories = [
new Label(id: "l1", name: "inbox", displayName: "Inbox", accountId: 'a1'),
new Label(id: "l2", name: "archive", displayName: "Archive", accountId: 'a1'),
new Label(id: "l3", displayName: "Happy Days", accountId: 'a1'),
new Label(id: "l4", displayName: "Sad Days", accountId: 'a1')
]
ThreadCountsStore._deltas =
l1: 10
l2: 0
l3: 3
l4: 12
feat(counts): Unread counts for all folders and labels across all accounts Summary: This diff replaces the UnreadCountStore with a better approach that is able to track unread counts for all folders/labels without continuous (and cripplingly slow) SELECT COUNT(*) queries. When models are written to the database, we currently don't send out notifications with the "previous" state of those objects in the database. This makes it hard to determine how to update counters. (In the future, we may need to do this for live queries). Unfortunately, getting the "previous" state is going to be very hard, because multiple windows write to the database and the "previous" state we have might be outdated. We'd almost have to run a "SELECT" right before every "REPLACE INTO". I created an API that allows you to register observers around persistModel and unpersistModel. With this API, you can run queries before and after the database changes are made and pluck just the "before" state you're interested in. The `ThreadCountsStore` uses this API to determine the impact of persisting a set of threads on the unread counts of different labels. Before the threads are saved, it says "how much do these thread IDs contribute to unread counts currently?". After the write is complete it looks at the models and computes the difference between the old count impact and the new count impact, and updates the counters. I decided not to attach the unread count to the Label objects themselves because 1) they update frequently and 2) most things observing the DatabaseStore for categories do not care about counts, so they would be updating unnecessarily. The AccountSidebar now listens to the ThreadCountsStore as well as the CategoryStore, and there's a new preference in the General tab for turning off the counts. Test Plan: Tests are a work in progress, want to get feedback first! Reviewers: juan, evan Reviewed By: evan Differential Revision: https://phab.nylas.com/D2232
2015-11-24 09:12:22 +08:00
ThreadCountsStore._counts =
l1: 10
l2: 0
@countResolve = null
@countReject = null
spyOn(ThreadCountsStore, '_fetchCountForCategory').andCallFake =>
new Promise (resolve, reject) =>
@countResolve = resolve
@countReject = reject
feat(counts): Unread counts for all folders and labels across all accounts Summary: This diff replaces the UnreadCountStore with a better approach that is able to track unread counts for all folders/labels without continuous (and cripplingly slow) SELECT COUNT(*) queries. When models are written to the database, we currently don't send out notifications with the "previous" state of those objects in the database. This makes it hard to determine how to update counters. (In the future, we may need to do this for live queries). Unfortunately, getting the "previous" state is going to be very hard, because multiple windows write to the database and the "previous" state we have might be outdated. We'd almost have to run a "SELECT" right before every "REPLACE INTO". I created an API that allows you to register observers around persistModel and unpersistModel. With this API, you can run queries before and after the database changes are made and pluck just the "before" state you're interested in. The `ThreadCountsStore` uses this API to determine the impact of persisting a set of threads on the unread counts of different labels. Before the threads are saved, it says "how much do these thread IDs contribute to unread counts currently?". After the write is complete it looks at the models and computes the difference between the old count impact and the new count impact, and updates the counters. I decided not to attach the unread count to the Label objects themselves because 1) they update frequently and 2) most things observing the DatabaseStore for categories do not care about counts, so they would be updating unnecessarily. The AccountSidebar now listens to the ThreadCountsStore as well as the CategoryStore, and there's a new preference in the General tab for turning off the counts. Test Plan: Tests are a work in progress, want to get feedback first! Reviewers: juan, evan Reviewed By: evan Differential Revision: https://phab.nylas.com/D2232
2015-11-24 09:12:22 +08:00
it "should call _fetchCountForCategory for the first category not already in the counts cache", ->
ThreadCountsStore._fetchCountsMissing()
calls = ThreadCountsStore._fetchCountForCategory.calls
expect(calls.length).toBe(1)
expect(calls[0].args[0]).toBe(ThreadCountsStore._categories[2])
it "should set the _deltas for the category it's counting back to zero", ->
ThreadCountsStore._fetchCountsMissing()
expect(ThreadCountsStore._deltas.l3).toBe(0)
feat(counts): Unread counts for all folders and labels across all accounts Summary: This diff replaces the UnreadCountStore with a better approach that is able to track unread counts for all folders/labels without continuous (and cripplingly slow) SELECT COUNT(*) queries. When models are written to the database, we currently don't send out notifications with the "previous" state of those objects in the database. This makes it hard to determine how to update counters. (In the future, we may need to do this for live queries). Unfortunately, getting the "previous" state is going to be very hard, because multiple windows write to the database and the "previous" state we have might be outdated. We'd almost have to run a "SELECT" right before every "REPLACE INTO". I created an API that allows you to register observers around persistModel and unpersistModel. With this API, you can run queries before and after the database changes are made and pluck just the "before" state you're interested in. The `ThreadCountsStore` uses this API to determine the impact of persisting a set of threads on the unread counts of different labels. Before the threads are saved, it says "how much do these thread IDs contribute to unread counts currently?". After the write is complete it looks at the models and computes the difference between the old count impact and the new count impact, and updates the counters. I decided not to attach the unread count to the Label objects themselves because 1) they update frequently and 2) most things observing the DatabaseStore for categories do not care about counts, so they would be updating unnecessarily. The AccountSidebar now listens to the ThreadCountsStore as well as the CategoryStore, and there's a new preference in the General tab for turning off the counts. Test Plan: Tests are a work in progress, want to get feedback first! Reviewers: juan, evan Reviewed By: evan Differential Revision: https://phab.nylas.com/D2232
2015-11-24 09:12:22 +08:00
describe "when the count promise finishes", ->
feat(counts): Unread counts for all folders and labels across all accounts Summary: This diff replaces the UnreadCountStore with a better approach that is able to track unread counts for all folders/labels without continuous (and cripplingly slow) SELECT COUNT(*) queries. When models are written to the database, we currently don't send out notifications with the "previous" state of those objects in the database. This makes it hard to determine how to update counters. (In the future, we may need to do this for live queries). Unfortunately, getting the "previous" state is going to be very hard, because multiple windows write to the database and the "previous" state we have might be outdated. We'd almost have to run a "SELECT" right before every "REPLACE INTO". I created an API that allows you to register observers around persistModel and unpersistModel. With this API, you can run queries before and after the database changes are made and pluck just the "before" state you're interested in. The `ThreadCountsStore` uses this API to determine the impact of persisting a set of threads on the unread counts of different labels. Before the threads are saved, it says "how much do these thread IDs contribute to unread counts currently?". After the write is complete it looks at the models and computes the difference between the old count impact and the new count impact, and updates the counters. I decided not to attach the unread count to the Label objects themselves because 1) they update frequently and 2) most things observing the DatabaseStore for categories do not care about counts, so they would be updating unnecessarily. The AccountSidebar now listens to the ThreadCountsStore as well as the CategoryStore, and there's a new preference in the General tab for turning off the counts. Test Plan: Tests are a work in progress, want to get feedback first! Reviewers: juan, evan Reviewed By: evan Differential Revision: https://phab.nylas.com/D2232
2015-11-24 09:12:22 +08:00
it "should add it to the count cache", ->
ThreadCountsStore._fetchCountsMissing()
advanceClock()
@countResolve(4)
advanceClock()
expect(ThreadCountsStore._counts.l3).toEqual(4)
feat(counts): Unread counts for all folders and labels across all accounts Summary: This diff replaces the UnreadCountStore with a better approach that is able to track unread counts for all folders/labels without continuous (and cripplingly slow) SELECT COUNT(*) queries. When models are written to the database, we currently don't send out notifications with the "previous" state of those objects in the database. This makes it hard to determine how to update counters. (In the future, we may need to do this for live queries). Unfortunately, getting the "previous" state is going to be very hard, because multiple windows write to the database and the "previous" state we have might be outdated. We'd almost have to run a "SELECT" right before every "REPLACE INTO". I created an API that allows you to register observers around persistModel and unpersistModel. With this API, you can run queries before and after the database changes are made and pluck just the "before" state you're interested in. The `ThreadCountsStore` uses this API to determine the impact of persisting a set of threads on the unread counts of different labels. Before the threads are saved, it says "how much do these thread IDs contribute to unread counts currently?". After the write is complete it looks at the models and computes the difference between the old count impact and the new count impact, and updates the counters. I decided not to attach the unread count to the Label objects themselves because 1) they update frequently and 2) most things observing the DatabaseStore for categories do not care about counts, so they would be updating unnecessarily. The AccountSidebar now listens to the ThreadCountsStore as well as the CategoryStore, and there's a new preference in the General tab for turning off the counts. Test Plan: Tests are a work in progress, want to get feedback first! Reviewers: juan, evan Reviewed By: evan Differential Revision: https://phab.nylas.com/D2232
2015-11-24 09:12:22 +08:00
it "should call _fetchCountsMissing again to populate the next missing count", ->
ThreadCountsStore._fetchCountsMissing()
spyOn(ThreadCountsStore, '_fetchCountsMissing')
advanceClock()
@countResolve(4)
advanceClock()
advanceClock(10001)
feat(counts): Unread counts for all folders and labels across all accounts Summary: This diff replaces the UnreadCountStore with a better approach that is able to track unread counts for all folders/labels without continuous (and cripplingly slow) SELECT COUNT(*) queries. When models are written to the database, we currently don't send out notifications with the "previous" state of those objects in the database. This makes it hard to determine how to update counters. (In the future, we may need to do this for live queries). Unfortunately, getting the "previous" state is going to be very hard, because multiple windows write to the database and the "previous" state we have might be outdated. We'd almost have to run a "SELECT" right before every "REPLACE INTO". I created an API that allows you to register observers around persistModel and unpersistModel. With this API, you can run queries before and after the database changes are made and pluck just the "before" state you're interested in. The `ThreadCountsStore` uses this API to determine the impact of persisting a set of threads on the unread counts of different labels. Before the threads are saved, it says "how much do these thread IDs contribute to unread counts currently?". After the write is complete it looks at the models and computes the difference between the old count impact and the new count impact, and updates the counters. I decided not to attach the unread count to the Label objects themselves because 1) they update frequently and 2) most things observing the DatabaseStore for categories do not care about counts, so they would be updating unnecessarily. The AccountSidebar now listens to the ThreadCountsStore as well as the CategoryStore, and there's a new preference in the General tab for turning off the counts. Test Plan: Tests are a work in progress, want to get feedback first! Reviewers: juan, evan Reviewed By: evan Differential Revision: https://phab.nylas.com/D2232
2015-11-24 09:12:22 +08:00
expect(ThreadCountsStore._fetchCountsMissing).toHaveBeenCalled()
describe "when deltas appear during a count", ->
it "should not set the count and count again in 10 seconds", ->
ThreadCountsStore._fetchCountsMissing()
spyOn(ThreadCountsStore, '_fetchCountsMissing')
advanceClock()
ThreadCountsStore._deltas.l3 = -1
@countResolve(4)
advanceClock()
expect(ThreadCountsStore._counts.l3).toBeUndefined()
expect(ThreadCountsStore._fetchCountsMissing).not.toHaveBeenCalled()
advanceClock(10001)
expect(ThreadCountsStore._fetchCountsMissing).toHaveBeenCalled()
feat(counts): Unread counts for all folders and labels across all accounts Summary: This diff replaces the UnreadCountStore with a better approach that is able to track unread counts for all folders/labels without continuous (and cripplingly slow) SELECT COUNT(*) queries. When models are written to the database, we currently don't send out notifications with the "previous" state of those objects in the database. This makes it hard to determine how to update counters. (In the future, we may need to do this for live queries). Unfortunately, getting the "previous" state is going to be very hard, because multiple windows write to the database and the "previous" state we have might be outdated. We'd almost have to run a "SELECT" right before every "REPLACE INTO". I created an API that allows you to register observers around persistModel and unpersistModel. With this API, you can run queries before and after the database changes are made and pluck just the "before" state you're interested in. The `ThreadCountsStore` uses this API to determine the impact of persisting a set of threads on the unread counts of different labels. Before the threads are saved, it says "how much do these thread IDs contribute to unread counts currently?". After the write is complete it looks at the models and computes the difference between the old count impact and the new count impact, and updates the counters. I decided not to attach the unread count to the Label objects themselves because 1) they update frequently and 2) most things observing the DatabaseStore for categories do not care about counts, so they would be updating unnecessarily. The AccountSidebar now listens to the ThreadCountsStore as well as the CategoryStore, and there's a new preference in the General tab for turning off the counts. Test Plan: Tests are a work in progress, want to get feedback first! Reviewers: juan, evan Reviewed By: evan Differential Revision: https://phab.nylas.com/D2232
2015-11-24 09:12:22 +08:00
describe "when a count fails", ->
it "should not immediately try to count any other categories", ->
ThreadCountsStore._fetchCountsMissing()
spyOn(ThreadCountsStore, '_fetchCountsMissing')
spyOn(console, 'error')
advanceClock()
@countReject(new Error("Oh man something really bad."))
advanceClock()
expect(ThreadCountsStore._fetchCountsMissing).not.toHaveBeenCalled()
describe "_fetchCountForCategory", ->
it "should make the appropriate label or folder database query", ->
spyOn(DatabaseStore, 'count')
Matcher.muid = 0
ThreadCountsStore._fetchCountForCategory(new Label(id: 'l1', accountId: 'a1'))
Matcher.muid = 0
expect(DatabaseStore.count).toHaveBeenCalledWith(Thread, [
Thread.attributes.accountId.equal('a1'),
Thread.attributes.unread.equal(true),
Thread.attributes.labels.contains('l1')
])
Matcher.muid = 0
ThreadCountsStore._fetchCountForCategory(new Folder(id: 'l1', accountId: 'a1'))
Matcher.muid = 0
expect(DatabaseStore.count).toHaveBeenCalledWith(Thread, [
Thread.attributes.accountId.equal('a1'),
Thread.attributes.unread.equal(true),
Thread.attributes.folders.contains('l1')
])
describe "_saveCounts", ->
beforeEach ->
ThreadCountsStore._counts =
'b': 3
'a': 5
ThreadCountsStore._deltas =
'a': -1
'c': 2
it "should merge the deltas into the counts and reset the deltas, ignoring any deltas for which the initial count has not been run", ->
ThreadCountsStore._saveCounts()
expect(ThreadCountsStore._counts).toEqual({
'b': 3
'a': 4
})
it "should persist the new counts to the database", ->
feat(transactions): Explicit (and faster) database transactions Summary: Until now, we've been hiding transactions beneath the surface. When you call persistModel, you're implicitly creating a transaction. You could explicitly create them with `atomically`..., but there were several critical problems that are fixed in this diff: - Calling persistModel / unpersistModel within a transaction could cause the DatabaseStore to trigger. This could result in other parts of the app making queries /during/ the transaction, potentially before the COMMIT occurred and saved the changes. The new, explicit inTransaction syntax holds all changes until after COMMIT and then triggers. - Calling atomically and then calling persistModel inside that resulted in us having to check whether a transaction was present and was gross. - Many parts of the code ran extensive logic inside a promise chained within `atomically`: BAD: ``` DatabaseStore.atomically => DatabaseStore.persistModel(draft) => GoMakeANetworkRequestThatReturnsAPromise ``` OVERWHELMINGLY BETTER: ``` DatabaseStore.inTransaction (t) => t.persistModel(draft) .then => GoMakeANetworkRequestThatReturnsAPromise ``` Having explicit transactions also puts us on equal footing with Sequelize and other ORMs. Note that you /have/ to call DatabaseStore.inTransaction (t) =>. There is no other way to access the methods that let you alter the database. :-) Other changes: - This diff removes Message.labels and the Message-Labels table. We weren't using Message-level labels anywhere, and the table could grow very large. - This diff changes the page size during initial sync from 250 => 200 in an effort to make transactions a bit faster. Test Plan: Run tests! Reviewers: juan, evan Reviewed By: juan, evan Differential Revision: https://phab.nylas.com/D2353
2015-12-18 03:46:05 +08:00
spyOn(DatabaseStore, '_query').andCallFake -> Promise.resolve([])
spyOn(DatabaseTransaction.prototype, 'persistJSONBlob')
runs =>
ThreadCountsStore._saveCounts()
waitsFor =>
DatabaseTransaction.prototype.persistJSONBlob.callCount > 0
runs =>
expect(DatabaseTransaction.prototype.persistJSONBlob).toHaveBeenCalledWith(ThreadCountsStore.JSONBlobKey, ThreadCountsStore._counts)
feat(counts): Unread counts for all folders and labels across all accounts Summary: This diff replaces the UnreadCountStore with a better approach that is able to track unread counts for all folders/labels without continuous (and cripplingly slow) SELECT COUNT(*) queries. When models are written to the database, we currently don't send out notifications with the "previous" state of those objects in the database. This makes it hard to determine how to update counters. (In the future, we may need to do this for live queries). Unfortunately, getting the "previous" state is going to be very hard, because multiple windows write to the database and the "previous" state we have might be outdated. We'd almost have to run a "SELECT" right before every "REPLACE INTO". I created an API that allows you to register observers around persistModel and unpersistModel. With this API, you can run queries before and after the database changes are made and pluck just the "before" state you're interested in. The `ThreadCountsStore` uses this API to determine the impact of persisting a set of threads on the unread counts of different labels. Before the threads are saved, it says "how much do these thread IDs contribute to unread counts currently?". After the write is complete it looks at the models and computes the difference between the old count impact and the new count impact, and updates the counters. I decided not to attach the unread count to the Label objects themselves because 1) they update frequently and 2) most things observing the DatabaseStore for categories do not care about counts, so they would be updating unnecessarily. The AccountSidebar now listens to the ThreadCountsStore as well as the CategoryStore, and there's a new preference in the General tab for turning off the counts. Test Plan: Tests are a work in progress, want to get feedback first! Reviewers: juan, evan Reviewed By: evan Differential Revision: https://phab.nylas.com/D2232
2015-11-24 09:12:22 +08:00
describe "CategoryDatabaseMutationObserver", ->
beforeEach ->
@label1 = new Label(id: "l1", name: "inbox", displayName: "Inbox")
@label2 = new Label(id: "l2", name: "archive", displayName: "Archive")
@label3 = new Label(id: "l3", displayName: "Happy Days")
@label4 = new Label(id: "l4", displayName: "Sad Days")
@threadA = new Thread
id: "A"
unread: true
labels: [@label1, @label4]
@threadB = new Thread
id: "B"
unread: true
labels: [@label3]
@threadC = new Thread
id: "C"
unread: false
labels: [@label1, @label3]
describe "given a set of modifying models", ->
scenarios = [{
type: 'persist',
expected: {
l3: -1,
l2: -1,
l4: 1
}
},{
type: 'unpersist',
expected: {
l1: -1,
l3: -2,
l2: -1
}
}]
scenarios.forEach ({type, expected}) ->
it "should call countsDidChange with the folder / label membership deltas (#{type})", ->
queryResolves = []
query = jasmine.createSpy('query').andCallFake =>
new Promise (resolve, reject) ->
queryResolves.push(resolve)
countsDidChange = jasmine.createSpy('countsDidChange')
m = new ThreadCountsStore.CategoryDatabaseMutationObserver(countsDidChange)
beforePromise = m.beforeDatabaseChange(query, {
type: type
objects: [@threadA, @threadB, @threadC],
objectIds: [@threadA.id, @threadB.id, @threadC.id]
objectClass: Thread.name
})
expect(query.callCount).toBe(2)
expect(query.calls[0].args[0]).toEqual("SELECT `Thread`.id as id, `Thread-Label`.`value` as catId FROM `Thread` INNER JOIN `Thread-Label` ON `Thread`.`id` = `Thread-Label`.`id` WHERE `Thread`.id IN ('A','B','C') AND `Thread`.unread = 1")
expect(query.calls[1].args[0]).toEqual("SELECT `Thread`.id as id, `Thread-Folder`.`value` as catId FROM `Thread` INNER JOIN `Thread-Folder` ON `Thread`.`id` = `Thread-Folder`.`id` WHERE `Thread`.id IN ('A','B','C') AND `Thread`.unread = 1")
queryResolves[0]([
{id: @threadA.id, catId: @label1.id},
{id: @threadA.id, catId: @label3.id},
{id: @threadB.id, catId: @label2.id},
{id: @threadB.id, catId: @label3.id},
])
queryResolves[1]([])
waitsForPromise =>
beforePromise.then (result) =>
expect(result).toEqual({
categories: {
l1: -1,
l3: -2,
l2: -1
}
})
m.afterDatabaseChange(query, {
type: type
objects: [@threadA, @threadB, @threadC],
objectIds: [@threadA.id, @threadB.id, @threadC.id]
objectClass: Thread.name
}, result)
expect(countsDidChange).toHaveBeenCalledWith(expected)