Specify the XCTest test environment in which GitLab CI will run

When you have GitLab CI perform testing using XCTest, you need to specify the test environment.

If the machine running Runner on the macOS app is an Apple Silicon machine, do we run it natively or do we run the Intel binary with Rosetta2?

If you are testing an iOS app, do you use an iOS simulator or an actual device that is connected? Is the simulator an iPhone 13 Pro or an iPad Air, etc.?

This article explains how to specify them.

For more information on how to set up CI/CD in GitLab, see the following article.

TOC

macOS Apps

You can specify the test environment with the platform option and the arch option, both options can be passed to the -destination argument of xcodebuild.

You specify the macOS to the platform.

The value of the arch is a one of following.

arch 説明
x86_64 64bit, Intel Mac Binary
arm64 Apple Silicon Mac Native Binary
The value of the arch

Example of the GitLab CI/CD configuration file

For example, the configuration file .gitlab-ci.yml for testing both Intel and Apple Silicon versions on an Apple Silicon Mac, in that order, would look like this.

stages:
  - build
  - test
  - archive
  - deploy

build_project:
  stage: build
  script:
    - xcodebuild clean -project CITestMac.xcodeproj -scheme CITestMac | xcpretty
    - xcodebuild test -project CITestMac.xcodeproj -scheme CITestMac -destination 'platform=macOS,arch=x86_64' | xcpretty -s
    - xcodebuild test -project CITestMac.xcodeproj -scheme CITestMac -destination 'platform=macOS,arch=arm64' | xcpretty -s
  tags:
    - mac

To run on Rosetta2

To test the Intel Mac version binary by running it on Rosetta2 on an Apple Silicon Mac, specify x86_64 for arch.

When arm64 is specified on an Intel Mac

If you specify arm64 for xcodebuild to run on an Intel Mac, an error will occur and you will be notified that the GitLab CI job has failed.

xcodebuild: error: Unable to find a destination matching the provided destination specifier:
		{ platform:macOS, arch:arm64 }
	Available destinations for the "CITestMac" scheme:
		{ platform:macOS, arch:x86_64, id:906947B6-2F2A-5257-9AA3-A0AF24CDA8DE }
	Ineligible destinations for the "CITestMac" scheme:
		{ platform:macOS, name:Any Mac }

If you want to test both binaries, you will need an Apple Silicon Mac.

iOS Apps

Specify by platform, name, and OS value to pass to xcodebuild‘s -destination option.

platform specifies whether to use the actual machine or a simulator; when executing from CI, the simulator is probably used more often than the actual machine.

platform Description
iOS Use devices connected to the machine
iOS Simulator Use iOS Simulator.
platform value

name specifies the name of the simulator or device to be used. For example, iPhone 13.

OS specifies the version of the simulator’s OS; if it is iOS 15.5, specify 15.5.

Xcode can install additional simulators by specifying the OS version. By installing simulators for multiple versions of the OS, testing on multiple OSs can be performed.

When the behavior differs from OS to OS, the simulator also behaves differently, making it possible to test multiple versions, which is difficult (or rather, time-consuming) to do manually.

How to add a simulator

To install additional simulators other than the standard built-in simulator with Xcode, do the following.

STEP
Select the “Preferences…” from the “Xcode” menu.
STEP
Open the “Components` tab.
STEP
A table of simulators that can be additionally installed is displayed. Click the down arrow button (download button) displayed to the left of the simulator you wish to install.
All necessary simulators can be downloaded individually
All necessary simulators can be downloaded individually

Example of GitLab CI/CD configuration file

For example, if you want to use the iPhone 13 simulator on iOS 15.5, the configuration file would look like this.

stages:
  - build
  - test
  - archive
  - deploy

build_project:
  stage: build
  script:
    - xcodebuild clean -project CITest.xcodeproj -scheme CITest | xcpretty
    - xcodebuild test -project CITest.xcodeproj -scheme CITest -destination 'platform=iOS Simulator,name=iPhone 13,OS=15.5' | xcpretty -s
  tags:
    - mac

When you want to test with multiple simulators

The script: is the script you want Runner to execute. Therefore, if you want to test on multiple simulators, change the value of the -destination option in the xcodebuild test statement to specify multiple simulators.

Let's share this post !

Author of this article

Akira Hayashiのアバター Akira Hayashi Representative, Software Engineer

I am an application developer loves programming. This blog is a tech blog, its articles are learning notes. In my work, I mainly focus on desktop and mobile application development, but I also write technical books and teach seminars. The websites of my work and books are here -> RK Kaihatsu.

TOC