React Native E2E Tests and Automatic Deploys

Archive note: This post was originally published on DEV Community in April 2021. It uses Node.js 12, Java 8, Android API 29, an older React Native CircleCI orb, and now-outdated CI setup commands. Treat it as a record of the pipeline and its tradeoffs, not as configuration to paste into a current project.

A little context

I had spent weeks trying to create a pipeline where every pull request targeting our staging branch ran end-to-end tests automatically. When one of those pull requests was merged, the same pipeline needed to create a TestFlight build and a Google Play internal-testing build.

My experience was… difficult. That didn’t mean yours had to be.

The short version: PAIN.

What did the pipeline require?

1. CircleCI with access to macOS executors. Both platforms could be tested elsewhere, but this implementation used macOS while building iOS and, at the time, while wrestling the Android environment into place.
2. App Store Connect and Google Play Console accounts. Fastlane needed these for automatic beta delivery.
3. Patience. CI systems can smell fear.

Start with Detox locally

Before touching CI, I added Detox to the React Native project and made sure the end-to-end suite ran locally with Jest. CI adds enough uncertainty by itself; debugging the tests and the environment simultaneously makes everything harder.

The project’s .detoxrc.json described the iOS simulator and Android emulator builds. Once both configurations worked on a development machine, it was time to create .circleci/config.yml.

Build the CircleCI foundation

The pipeline used version 5.1.0 of the community React Native orb:

version: 2.1

orbs:
  rn: react-native-community/react-native@5.1.0

The rn alias was arbitrary; it simply prefixed commands and executors supplied by the orb, such as rn/yarn_install.

Android also needed a Gradle task that resolved project dependencies ahead of the build so CircleCI could warm its cache:

task downloadDependencies() {
  description 'Download all dependencies to the Gradle cache'

  doLast {
    configurations.findAll().each { config ->
      if (config.name.contains("minReactNative") && config.canBeResolved) {
        print config.name
        print '\n'
        config.files
      }
    }
  }
}

Check out once, analyze on Linux

The first job checked out the repository and persisted it to a CircleCI workspace. The next attached that workspace, installed JavaScript dependencies, and ran the unit tests on a less expensive Linux executor.

jobs:
  checkout_code:
    executor:
      name: rn/linux_js
      node_version: "12"
    steps:
      - checkout
      - persist_to_workspace:
          root: .
          paths: .

  analyse_js:
    executor:
      name: rn/linux_js
      node_version: "12"
    steps:
      - attach_workspace:
          at: .
      - rn/yarn_install
      - run:
          name: Run tests
          command: yarn test

This gave every expensive native job the same tested source snapshot.

Android end-to-end tests

The orb’s normal Android build did not produce a Detox-specific build in this setup. That led to strange failures and false negatives, so I replaced the convenience job with an explicit sequence:

1. Start a macOS executor and attach the workspace.
2. Install Java 8, Android SDK tools, API 29, build tools, and the emulator.
3. Create and start a headless Pixel 2 virtual device.
4. Wait for Android to finish booting and disable system animations.
5. Build the android.emu.release Detox configuration.
6. Run the suite and preserve screenshots from failed tests as CircleCI artifacts.

The last two steps looked like this:

android_e2e_test:
  executor:
    name: rn/macos
  steps:
    - attach_workspace:
        at: .
    - rn/setup_macos_executor:
        homebrew_cache: true
        node_version: "12"
    - rn/yarn_install:
        cache: false

    # Emulator installation, creation, startup, and boot checks ran here.

    - rn/detox_build:
        configuration: "android.emu.release"
    - run:
        name: Detox test
        command: >-
          detox test -c android.emu.release -l warn --headless
          --take-screenshots failing
          --artifacts-location /tmp/detox_artifacts
    - store_artifacts:
        path: /tmp/detox_artifacts

The same work could also be moved to the orb’s Linux Android executor, saving macOS minutes once its Android toolchain was configured correctly.

iOS end-to-end tests

The built-in iOS build-and-test job worked until it didn’t. Recreating it explicitly made each failing stage easier to see and control:

ios_e2e_test:
  executor: rn/macos
  steps:
    - checkout
    - attach_workspace:
        at: .
    - rn/setup_macos_executor:
        homebrew_cache: true
        node_version: "12"
    - rn/ios_simulator_start:
        device: "iPhone 11"
    - rn/yarn_install:
        cache: false
    - rn/pod_install:
        pod_install_directory: ios
    - rn/ios_build:
        build_configuration: "Release"
        cache: false
        derived_data_path: "ios/build"
        device: "iPhone 11"
        project_path: "ios/example.xcworkspace"
        project_type: workspace
        scheme: "example"
    - run:
        name: Detox test
        command: >-
          detox test -c ios.sim.release -l warn --headless
          --take-screenshots failing
          --artifacts-location /tmp/detox_artifacts
    - store_artifacts:
        path: /tmp/detox_artifacts

At this point, both apps had end-to-end tests running in CI. Time for a break, because Fastlane was next.

Deliver beta builds with Fastlane

The native projects already had their Fastlane lanes and signing credentials configured. CircleCI’s responsibility was to install the required Ruby tools and call those lanes only after the matching platform’s tests passed.

Android internal testing

fastlane_android_internal:
  executor: rn/linux_android
  steps:
    - attach_workspace:
        at: .
    - rn/yarn_install
    - run:
        name: Install Bundler
        command: gem install bundler
    - run:
        name: Install Fastlane
        command: gem install fastlane
    - run:
        name: Upload to Google Play via Fastlane
        command: cd android && fastlane upload_to_googleplay

iOS TestFlight

fastlane_ios_testflight:
  executor:
    name: rn/macos
  steps:
    - attach_workspace:
        at: .
    - rn/yarn_install:
        cache: false
    - run:
        working_directory: ios
        command: pod install
    - run:
        name: Install Bundler
        command: gem install bundler
    - run:
        name: Install Fastlane
        command: gem install fastlane
    - run:
        name: Upload to TestFlight via Fastlane
        working_directory: ios
        command: fastlane beta

The hardest part was configuring signing and store access correctly for each project. Fastlane’s CI and code-signing documentation was essential here.

For build numbers, this pipeline generated sortable values from the current date:

• Android in build.gradle: (int)(date.getTime() / 10000)
• iOS in Fastfile: DateTime.now.strftime("%Y%m%d%H%M")

Connect the workflow

The final workflow expressed the important dependency chain: check out, analyze once, test both platforms in parallel, and release each beta only after its own end-to-end suite succeeded on main.

workflows:
  main:
    jobs:
      - checkout_code
      - analyse_js:
          requires:
            - checkout_code
      - android_e2e_test:
          requires:
            - analyse_js
      - ios_e2e_test:
          requires:
            - analyse_js
      - fastlane_android_internal:
          filters:
            branches:
              only:
                - main
          requires:
            - android_e2e_test
      - fastlane_ios_testflight:
          filters:
            branches:
              only:
                - main
          requires:
            - ios_e2e_test

The details have aged, but the shape still holds: validate the shared code once, test native platforms independently, preserve failure artifacts, and make delivery depend on the tests that protect each build.