Mailspring/spec_integration

Integration Testing

In addition to unit tests, we run integration tests using ChromeDriver and WebdriverIO through the Spectron library.

Running Tests

script/grunt run-integration-tests

This command will, in order:

1. Run npm test in the /spec_integration directory and pass in the NYLAS_ROOT_PATH
2. Boot jasmine and load all files ending in -spec
3. Most tests in beforeAll will boot N1 via the N1Launcher. See spec_integration/helpers/n1-launcher.es6
4. This instantiates a new spectron Application which will spawn a ChromeDriver process with the appropriate N1 launch args.
5. ChromeDriver will then boot a Selenium server at http://localhost:9515
6. The ChromeDriver / Selenium server will boot N1 with testing hooks and expose an controlling API.
7. The API is made easily available through the Spectron API
8. The N1Launcher's mainWindowReady or popoutComposerWindowReady or onboardingWindowReady methods poll the app until the designated window is available and loaded. Then will resolve a Promise once everything has booted.

Writing Tests

The Spectron API is a pure extension over the Webdriver API. Reference both to write tests.

Most of the methods on app.client object apply to the "currently focused" window only. Since N1 has several windows (many of which are hidden) the N1Launcher extension will cycle through windows automatically until it finds the one you want, and then select it.

Furthermore, "loaded" in the pure Spectron sense is only once the window is booted. N1 windows take much longer to full finish loading packages and rendering the UI. The N1Launcher::windowReady method and its derivatives take this into account.

You will almost always need the minimal boilerplate for each integration test:

describe('My test', () => {
  beforeAll((done)=>{
    // Boot in dev mode with no arguments
    this.app = new N1Launcher(["--dev"]);
    this.app.mainWindowReady().finally(done);
  });

  afterAll((done)=> {
    if (this.app && this.app.isRunning()) {
      this.app.stop().finally(done);
    } else {
      done()
    }
  });

  it("is a test you'll write", () => {
  });

  it("is an async test you'll write", (done) => {
    doSomething.finally(done)
  });
});

Executing Code in N1's environment

The app.client.execute and app.client.executeAsync methods are extremely helpful when running code in N1. Those are documented slightly more on the WebdriveIO API docs page here

it("is a test you'll write", () => {
  this.app.client.execute((arg1)=>{
    // NOTE: `arg1` just got passed in over a JSON api. It can only be a
    // primitive data type

    // I'M RUNNING IN N1
    return someValue

  }, arg1).then(({value})=>{
    // NOTE: the return is stuffed in an attribute called `value`. Also it
    // passed back of a JSON API and can only be a primitive value.
  })
});

Debugging tests.

Debugging is through lots of console.loging.

There is code is spec_integration/jasmine/bootstrap.js that attempts to catch unhandled Promises and color them accordingly.

If you want to access logs from within N1 via the app.client.execute blocks, you'll have to either package it up yourself and return it, or use the new app.client.getMainProcessLogs() just added into Spectron.