Test automation with Appium in white label apps

Appium test

May 2023

Test automation with Appium in white label apps

Posted at 14: 46h software delivery by

Introduction to test automation with Appium

Regression testing can become a very tedious and expensive process when tested manually.

Especially when, in addition to the functionalities, each UI component also has to be checked for its correct layout.

Experience has shown that modern projects have a very high demand for test automation, and the question regularly arises as to whether there are powerful tools that can automate this process.

The answer is yes and in the following article we will introduce test automation with the tool appium.

appium is an open source tool for test automation of native, hybrid and mobile web applications. It supports many different programming languages ​​like Java, Ruby, Python and PHP. The Appium philosophy is that writing and running tests is not tied to any particular language or framework. Appium is cross-platform, which means the same API can be used to write tests on different platforms (iOS, Android, and Windows).

In this blog article, we take a quick look at how easy it is to use Appium to write tests to test UI components in a white-labeled app on iOS and Android devices.

Use case scenario

Let's imagine we have a white label app that can be customized to many different themes and we want to automate testing that each app uses the correct themes.

With manual tests, the effort increases in proportion to the number of different app versions. Each time the apps are updated, they have to be tested again to ensure that the designs are still correct.

At a certain point, we can no longer rely 100% on manual testing. Classic UI tests focus on specific views and interactions. With this type of testing, it is difficult to verify the color or position of a UI component, or even the font used.

So we need a detailed review of what's on the screen and whether its state has changed over time.

Visualization check with Appium and OpenCV

OpenCV is a framework with algorithms for image processing and computer vision and can be used as a plugin together with Appium. Image comparison with Appium + OpenCV can be useful for many test automation tasks that are purely design-focused.

Let's see how this works with an example.

It must be checked whether the current design corresponds to the expected state. First, we need screenshots that we'll use to compare to the current design. They are automatically generated with the Appium Driver. For the image comparison we use the similarity value and if this value is smaller than the expected one, a visualization image with the differences is automatically generated.

Appium OpenCV

Expected condition

Appium OpenCV

It's on

We have now removed the logo in the right image and changed the font and text size. We now run a test with the following source code:

//Visual check fun doVisualCheck( screen: Screen, platformName: PlatformName, matchThreshold: Double, driver: AppiumWrapper) { val baselineImage = getBaselineScreenshotFileFor(platformName, screen) val similarity = driver.getImagesSimilarity(baselineImage) if (similarity.score >= matchThreshold ) { println("Visual check of '${screen.title}' passed; similarity match was result.score") } else { val failVisualization = getErrorScreenshotFileFor(platformName, screen) similarity.storeVisualization(failVisualization) throw Exception("Test FAILED check of '${screen.title}' failed; similarity match was only ${similarity.score}, and below the threshold of $matchThreshold. Visualization written to ${failVisualization.absolutePath}.") } } fun getBaselineScreenshotFileFor(platformName: PlatformName, screen: Screen): File { val imageLocation = this.getBaselineScreenshotLocationFor(platformName, screen) val imageFile = File(imageLocation) if (!imageFile.exists()) {
error("No existing baseline image file found at '$imageLocation'.") } return imageFile } fun getBaselineScreenshotLocationFor(platformName: PlatformName, screen: Screen): String = getScreenshotLocation(platformName, this.id, "baseline", screen.title) fun getImagesSimilarity(baselineImage: File): SimilarityMatchingResult { if (!baselineImage.exists()) {
error("No existing baseline image file with name '$baselineImage'.") } val options = SimilarityMatchingOptions() options.withEnabledVisualization() return driver.getImagesSimilarity( baselineImage, driver.getScreenshotAs( OutputType.FILE
) as File, options ) }

After the visualization check with Appium and OpenCV was completed, all changes were detected and illustrated on a generated image. The output is as follows: "Similarity match was only 0.9647010, and below the threshold of 0.99. The visualization was written to / /screenshots/iOS/OpenCVSample/fail/test.png”

Screenshot: Test automation result

Generated image after failed visualization check.

Summary

Appium with OpenCV can be efficiently used to test native or hybrid apps. Like any other automation tool available, it has its pros and cons.
The advantage is that we can always automatically check if there are any unexpected changes to the app design. In addition: the cross-platform approach and the fact that several platforms can be tested at the same time.

The downside is that we have to manually check at the beginning that all the screenshots used for later comparison are correct (ie fonts, colors, images, etc.).

Based on the project requirements, automating the testing process with the use of Appium can save a lot of time and detect errors early.

Ergesa Theilmann

author 
Ergesa Theilmann
Senior Developer
with a focus on Mobile (iOS)