Specify the XCTest test environment in which GitLab CI will run

When conducting XCTest testing using GitLab CI, you must define the test environment.

If the machine running Runner on the macOS app is an Apple Silicon machine, do we run it natively, or do we run the Intel binary with Rosetta2?

If you are testing an iOS app, do you use an iOS simulator or a connected device? For instance, would the simulator be an iPhone 13 Pro or an iPad Air, etc.?

This article explains how to specify them.

For more information on how to set up CI/CD in GitLab, see the following article.

See Also

Automate XCTest with GitLab’s CI GitLab has a Continuous Integration (CI) feature that allows you to perform continuous integration without deploying a separate solution dedicated to CI. The…

macOS Apps

You can specify the test environment with the platform and arch options. You can pass both options to the -destination argument of xcodebuild.

You set macOS as the platform.

The possible values for arch are the following.

Example of the GitLab CI/CD configuration file

For example, the configuration file .gitlab-ci.yml for testing both Intel and Apple Silicon versions on an Apple Silicon Mac, in that order, would look like this.

stages:
  - build
  - test
  - archive
  - deploy

build_project:
  stage: build
  script:
    - xcodebuild clean -project CITestMac.xcodeproj -scheme CITestMac | xcpretty
    - xcodebuild test -project CITestMac.xcodeproj -scheme CITestMac -destination 'platform=macOS,arch=x86_64' | xcpretty -s
    - xcodebuild test -project CITestMac.xcodeproj -scheme CITestMac -destination 'platform=macOS,arch=arm64' | xcpretty -s
  tags:
    - mac

To run on Rosetta2

To test the Intel Mac version binary by running it on Rosetta2 on an Apple Silicon Mac, set x86_64 as arch.

In case when you specifiy arm64 on an Intel Mac

If you set arm64 for xcodebuild to run on an Intel Mac, an error will ensue and signal that the GitLab CI job has failed.

xcodebuild: error: Unable to find a destination matching the provided destination specifier:
		{ platform:macOS, arch:arm64 }
	Available destinations for the "CITestMac" scheme:
		{ platform:macOS, arch:x86_64, id:906947B6-2F2A-5257-9AA3-A0AF24CDA8DE }
	Ineligible destinations for the "CITestMac" scheme:
		{ platform:macOS, name:Any Mac }

If you want to test both binaries, you will need an Apple Silicon Mac.

iOS Apps

Specify by platform, name, and OS value to pass to xcodebuild‘s -destination option.

platform determines whether to use an actual device or a simulator; when running from CI, the simulator is typically used more frequently than a physical device.

name specifies the name of the simulator or device. For example, iPhone 13.

OS specifies the version of the simulator’s OS; if it is iOS 15.5, specify 15.5.

Xcode can install additional simulators by specifying the OS version. By installing simulators for multiple versions of the OS, testing on various OSs can be performed.

When the behavior differs from OS to OS, the simulator also behaves differently, making it possible to test multiple versions, which is difficult (or, rather, time-consuming) to do manually.

How to add a simulator

Do the following: install additional simulators other than the standard built-in simulator with Xcode.

STEP

Select the “Preferences…” from the “Xcode” menu.

STEP

Open the “Components” tab.

STEP

A table of simulators that can be additionally installed is displayed. Click the down arrow button (download button) displayed to the left of the simulator you wish to install.

Example of GitLab CI/CD configuration file

For example, if you want to use the iPhone 13 simulator on iOS 15.5, the configuration file would look like this.

stages:
  - build
  - test
  - archive
  - deploy

build_project:
  stage: build
  script:
    - xcodebuild clean -project CITest.xcodeproj -scheme CITest | xcpretty
    - xcodebuild test -project CITest.xcodeproj -scheme CITest -destination 'platform=iOS Simulator,name=iPhone 13,OS=15.5' | xcpretty -s
  tags:
    - mac

When you want to test with multiple simulators

The script: is the script you want Runner to execute. Therefore, if you want to test on multiple simulators, change the value of the -destination option in the xcodebuild test statement to specify various simulators.