When conducting XCTest testing using GitLab CI, you must define the test environment.
If the machine running Runner on the macOS app is an Apple Silicon machine, do we run it natively, or do we run the Intel binary with Rosetta2?
If you are testing an iOS app, do you use an iOS simulator or a connected device? For instance, would the simulator be an iPhone 13 Pro or an iPad Air, etc.?
This article explains how to specify them.
For more information on how to set up CI/CD in GitLab, see the following article.
macOS Apps
You can specify the test environment with the platform and arch options. You can pass both options to the -destination argument of xcodebuild.
You set macOS as the platform.
The possible values for arch are the following.
| arch | Description |
|---|---|
x86_64
| 64bit, Intel Mac Binary |
arm64
| Apple Silicon Mac Native Binary |
Example of the GitLab CI/CD configuration file
For example, the configuration file .gitlab-ci.yml for testing both Intel and Apple Silicon versions on an Apple Silicon Mac, in that order, would look like this.
stages:
- build
- test
- archive
- deploy
build_project:
stage: build
script:
- xcodebuild clean -project CITestMac.xcodeproj -scheme CITestMac | xcpretty
- xcodebuild test -project CITestMac.xcodeproj -scheme CITestMac -destination 'platform=macOS,arch=x86_64' | xcpretty -s
- xcodebuild test -project CITestMac.xcodeproj -scheme CITestMac -destination 'platform=macOS,arch=arm64' | xcpretty -s
tags:
- mac
To run on Rosetta2
To test the Intel Mac version binary by running it on Rosetta2 on an Apple Silicon Mac, set x86_64 as arch.
In case when you specifiy arm64 on an Intel Mac
If you set arm64 for xcodebuild to run on an Intel Mac, an error will ensue and signal that the GitLab CI job has failed.
xcodebuild: error: Unable to find a destination matching the provided destination specifier:
{ platform:macOS, arch:arm64 }
Available destinations for the "CITestMac" scheme:
{ platform:macOS, arch:x86_64, id:906947B6-2F2A-5257-9AA3-A0AF24CDA8DE }
Ineligible destinations for the "CITestMac" scheme:
{ platform:macOS, name:Any Mac }If you want to test both binaries, you will need an Apple Silicon Mac.
iOS Apps
Specify by platform, name, and OS value to pass to xcodebuild‘s -destination option.
platform determines whether to use an actual device or a simulator; when running from CI, the simulator is typically used more frequently than a physical device.
| platform | Description |
|---|---|
iOS
| Use devices connected to the machine |
iOS Simulator
| Use iOS Simulator. |
name specifies the name of the simulator or device. For example, iPhone 13.
OS specifies the version of the simulator’s OS; if it is iOS 15.5, specify 15.5.
Xcode can install additional simulators by specifying the OS version. By installing simulators for multiple versions of the OS, testing on various OSs can be performed.
When the behavior differs from OS to OS, the simulator also behaves differently, making it possible to test multiple versions, which is difficult (or, rather, time-consuming) to do manually.
How to add a simulator
Do the following: install additional simulators other than the standard built-in simulator with Xcode.
(1) Select the “Preferences…” from the “Xcode” menu.
(2) Open the “Components” tab.
(3) A table of simulators that can be additionally installed is displayed. Click the down arrow button (download button) displayed to the left of the simulator you wish to install.
Example of GitLab CI/CD configuration file
For example, if you want to use the iPhone 13 simulator on iOS 15.5, the configuration file would look like this.
stages:
- build
- test
- archive
- deploy
build_project:
stage: build
script:
- xcodebuild clean -project CITest.xcodeproj -scheme CITest | xcpretty
- xcodebuild test -project CITest.xcodeproj -scheme CITest -destination 'platform=iOS Simulator,name=iPhone 13,OS=15.5' | xcpretty -s
tags:
- mac
When you want to test with multiple simulators
The script: is the script you want Runner to execute. Therefore, if you want to test on multiple simulators, change the value of the -destination option in the xcodebuild test statement to specify various simulators.
