Functional testing of mobile apps

For years I’ve been a huge fan of gadgets and most of all smartphones so I consider myself lucky that I could make it my profession. No, I’m not a smartphone but I’m a mobile app tester. I spend my days with a bunch of smartphones on my desk and I test apps on them. I also spend a lot of time creating test automation, I’ve worked with selenium Webdriver, Appium, XCUITEST, Espresso and Rest Assured. A couple of times a year I teach new colleagues about functional mobile testing and that’s something I’d like to share.

Introduction in Mobile

What makes a device a mobile device?

It’s a portable computer that’s filled with sensors, has a battery. And the thing I find interesting is that a mobile device is such a personal device. If I look at my smartphone, it has pictures of my wife and my kids. It holds all my contacts and appointments, all my text messages and I use it all day for anything.

What’s the difference between mobile and desktop?

A desktop is a desktop computer or a laptop, and a mobile is a smartphone, feature phone, tablet or a smartwatch.

How ‘big’ is mobile?

In recent years mobile became big, very big even bigger than the use of desktops.

What are the relevant platforms?

For me the relevant platforms are Android and iOS, but it’s dependent on the market you’re operating in. If you’re working with feature phones, most likely KaiOS is relevant for you, and Huawei’s platform HarmonyOS is also up and coming.

How is the market divided?

I already wrote about platforms that are relevant for me, and to figure out what is relevant for you we can look at the numbers. I use https://gs.statcounter.com/ to look at this, it shows stats based on over 10 billion page views per month to over 2 million global websites. Worldwide Android is the biggest platform, about 70% of the market, and next is iOS.

These worldwide numbers aren’t representative for the projects that I worked on, or maybe you’re working on, so have a look at the number that are important for you. For me that would be the Netherlands where Android is a bit bigger, almost 60% and iOS is about 40%

But looking at other markets like India we can see that Android holds 90% of the market.

If you’re building an app or testing an app, find out what the relevant platforms are to work on.

Also have a look at the relevant platform versions. There are a lot of version of each platform out there and maybe you need to support every version and maybe you can make the decision to support only the biggest platform versions.

What you see in these graphs is that iOS users generally update very quickly, for android it goes a bit slower but a lot quicker than a couple of years ago.

User experience

The user experience is every experience a user has with your product or service, it’s basically the entire brand experience. As a brand you need to deliver a unique experience for your user, so figure out the needs and wishes of your user group. Elke den Ouden did a research on reasons for returns, and she found out that almost 50% of the returns weren’t because the product was broken, but actually the product didn’t meet the expectations for some reason.

To prevent this, you should do proper research about your target audience, what their needs and desires are. And involve the users in this process, when you’re developing an app, introduce it to your users and find out whether it matches the experience with the brand and is your app user friendly? As a tester you shouldn’t just make sure the product is technically okay but also the usability,

Design

When it comes to design, simplicity for the user is key, everybody is able to use it. The looks used to be skeuomorphism, where items look like everyday items, currently it’s a more flat and minimalistic design. The design choices made for iOS makes it very accessible for users, but might be too limited for some users.

iOS 14 makes a clear step towards Androids state of mind, giving users access to widgets, which gives the user more room to adjust the homescreen.

Android has a different view than iOS, they give the user the possibility to adjust android completely to their wishes. The choices are great joy for users but it might be too technical of some users.

Android 11 is focused more on smart appliances, so controlling your speakers, lights et cetera.

Even though in my opinion the designs of Android and iOS are growing more toward each other there are still some clear difference, which the users are accustomed to. If you look at these examples iOS and Android users will immediately recognize the way to use the app, how to open the menu, how to navigate through the app. When you develop a new app, it’s very important to be empathetic towards the users, it really helps to have an app that is designed in a way that is recognizable for your users. E.g., to have a hamburger menu will help your Android users, while a tab bar is the way iOS users navigate through their apps.

Mobile apps

Revenue models

There are a couple of revenue models for apps, because most apps are built to create some revenue; the app could be filled with advertisements, or a freemium model, where you pay to unlock ‘pro-features’. It’s also possible to purchase the app in the App or play store. Another option is transaction fees, the app generates revenue by charging users fees for a completed transaction, like Airbnb or Uber. Some apps can even generate so much data about users, like metadata it’s very interesting for other businesses, who are willing to buy that data.

Type of Apps

Basically, there are three types of app that you can build, there are Native apps, which are written explicitly for one platform, using specific platform tools, typically for iOS is written in the Swift language in XCode, and for Android in Java or Kotlin in Android Studio. The apps are installed and updated in the App store and Play store. In general, the developers are more expensive because they’re more specialized in the specifics for that platform.

Secondly there are the webapps, which basically are websites that are fully optimized for mobile usage. In contrary to native apps the webapp is only available online. For a webapp you need web developers, which are a bit cheaper in general than a Native app developer which is a plus, and you can always show the latest version of your product because you don’t need to update via the app stores. The downside however could be that you don’t have the same access to the hardware of a device.

The third is the Hybrid app, it contains a part that is platform specific, and a part that is a web view. The pro for choosing a hybrid app is that the frame is natively made, so the users will be familiar with the navigation, also the web view is updated via the web so it’s easy to renew your content. The downside is however that the experience isn’t as good as a native app. And you still need native development knowledge for the native parts.

How would a web app look and feel compared to a normal website? To compare I’ll show you the AliExpress website;

The first is the normal website, not optimized for mobile, the texts are hard to read, and the elements hard to tap because they are so small.

This is what a mobile app would look like:

Notice the “m” in the URL, it’s optimized for mobile, the texts are readable, the elements are big enough to tap. It is much easier for users to use the webapp, it basically looks and feels like an app. The difference is that a native or hybrid app has access to the devices’ hardware, such as the camera. Which brings me to my last example, which is the native app.

Just like the web app, it’s optimized for mobile devices, this one is installed via the play store / app store. The app has access to the devices’ hardware, so it can use the camera to scan a QR-code or look for items similar to your picture, in this case.

When you’re going to build an app, you need to decide what type of app you’re going to build and you need to think about what is important to you, to make it a bit easier look below:

I mentioned that a web app has less access to the devices’ hardware, but that doesn’t mean that there is no access. A wonderful example is Google AR on the web, it uses the AR core and chrome to give you some AR goodies, just check out this example or https://immersive-web.github.io/webxr-samples/tests/ .

Architecture and backend

To properly explain the architecture, I made this visual.

On the left you see the ‘head’, the front end that is the app that interacts with the user. In the middle is the API, this uses rest calls to connect the front end to the backend, what could be a database or a server which is displayed on the right side.

This might sound a bit technical so I will give an example. When you log into an app, you enter a username and a password and you hit the login button. In the api this is one call with at least two parameters, the username and the password, and the server will responds with a HTTP status call and a token.

Usually, the API is built before the frontend is build, but that doesn’t mean you cannot test anything until the frontend is ready. You can start testing the API, a basic tool to test API’S is Postman. In the next example you see the login call (post call, with an URL), and the parameters, the username, password and API key to identify the app to the API. When you hit send the request is sent to the server and the server will respond in this case with a status 200 (OK) and a response body.

You could check the response body yourself each time you want to test the login feature. Or you can write some tests in Postman to do it for you, in the next example you see that the test checks for a status 200 in two different ways.

Challenges

The biggest challenge for mobile testing and development are the many different devices, different hardware, different form factors, screen sizes, aspect ratios, networks, platforms, OS versions or regional differences. Does this mean you need to test every feature on every device?

Testing

Installing

Before you can test an app, you will have to install the app. Users install apps from the Play store / App store, but as a tester you will test it before the app gets there. There are several ways to install the app to test, maybe the developer hands you the APK / IPA file, or you download it from a build server or you use an app distributor like HockeyApp, Xamarin, firebase or appcenter.

If you install the app via ADB, you need a couple of things,

• A phone (or emulator)
• USB debugging turned on
• ADB (part of android platform tools)

To install the app, use the following command:

ADB install <path to your APK file>

Testing

Jonathan Kohl wrote a mnemonic for mobile app testing, called I SLICED UP FUN. Which is a great way to remember a lot of different approaches for testing mobile apps.

I SLICED UP FUN stands for Input, Store, Location, Interactions/Interruptions, Communication, Ergonomics, Data, Usability, Platform, Function, User scenario, Network.

Input

Input is about interacting with the phone, so besides hardware buttons, maybe a keyboard it’s also the touchscreen and software keyboard. And don’t forget different orientations. What happens when the app launches in portrait mode? What happens when you launch the app in landscape mode? Is the view supported? Is the layout okay?

Store

Before you submit your app to the store you need to have a look at the submission guidelines.

For iOS they can be found here: https://developer.apple.com/app-store/review/guidelines/

For Android they can be found here: https://developer.android.com/distribute/best-practices/launch/launch-checklist

These checklist help you prepare for your app launch, and over at Apple your app will be tested by employee’s to see if it doesn’t break any of the rules, if it does your app will be rejected and you lose a lot of time before you can release your app.

Location

Your location can have a lot of impact on an apps’ behavior. It’s where you are on this earth and also being in movement, which might cause issues to your app. For instance, if you’re testing for locations nearby, you could use the android emulator and switch you’re location around, you can pinpoint yourself anywhere you want. You can also change the system language of your phone to see if the app supports that language

Interactions/Interruptions

Mobile phones aren’t built for multitasking the way a laptop or desktop is. There is also generally less RAM than in a PC, which could cause the OS to shut down apps in the background. Imagine you’re filling out a huge form and having a chat in another app and after the chat you realize your entire form is gone. If you spend some time to use the app you’re testing and using other apps in the meantime you might find a lot of issues, also try playing a game on the phone which usually requires a lot of resources.

Communication

Smartphones are essential in our communication, we use them to call someone, send hundreds of text messages, share pictures and updates about our lives. Your app might do the same, or gets interrupted by other apps, receiving calls, video calls, chat messages and other notifications. Using the Android emulator, you can test it! You can actually let the emulator receive and answer a call or a text message. I once tested an app that played radio over the internet, when I received a text message or a call, the radio would stop to play the notification, but afterwards didn’t pick the radio sounds back up, so you would have to start it again manually, which can be very annoying to the user, the same goes for ejecting and inserting the headphones, yes back then phones still had a headphone jack.

Ergonomics

Using smartphones for a long time can cause a lot of stress on the body. That means you need to consider this when you’re building your app. Flows shouldn’t take too long, because the screen is small you have to put your eyes to work, and your fingers as well. It’s a lot different than a flow for a desktop. If you look at a checkout flow, for mobile you need to optimize it as much as possible, make it easy to log in, fill forms and make a payment.

For a smartwatch is again a different story, the interaction time is just a couple of seconds, so you shouldn’t build a checkout flow for a smartwatch, but just notification updates about an order.

Data

When talking about data, in can be anything the app needs to process so this can be a text input but also media or documentation files. During testing you can try special characters, and process pictures, videos, documents, corrupt files etc.

Usability

I once read a quote from Charles Bukowski

It wasn’t the big things that drove people mad, it was the small stuff: little things not going well, small irritants that over time made you crazy, the leaking faucet, the stains that won’t come out of clothes, the mobile phone that won’t dial.

And that’s what usability is about, the issues in the app that frustrate users, it’s not even the big crashes in your app, but things running slow, buttons that are too small, basically anything that could frustrate you is up for discussion.

Platform

The P stands for platform, and the relevant platforms are iOS and Android. Apart from that you also need to consider platform versions, it can be a lot of work to support older platform versions, while there are less than 2% of the users on those platforms. The platforms can also have serious security issues so requiring your users to update could be the smart move. On the other side new versions of the operating systems are continuously rolling out, so you should test on the beta versions of those OSes so you know your app is running smoothly for your users once the stable update of the OS is released.

Function

The function of the app is the most obvious one to test. Is that app doing what it is supposed to be doing? Does every button have a function? Is every input field working? And does the app meet the users’ expectations?

User scenario

To test user scenarios, I always imagine at least three types of users;

• Someone who’s not familiar with technology and has a hard time using the app
• A senior citizen that takes a lot of time to read everything and works slowly
• A youth that is very impatient with the app and wants to see results quickly

This is a very basic step to test different user scenarios, if you want to take it a step further you can create personas.

Your app collects data about users, you know how many male and female users there are, what phones they use, what their age category is. If you know your user group even better you might know what they like and dislike, and what motivates them. I once worked on an app for a postal service, and we had a persona called Daan who was expecting a package, and another persona called Lieke who was sending out a lot of packages. The more personal you can make these personas the better! It also helps to create empathy through the entire organization for these personas.

Network

Most apps require a network connection, some more than others and the connection with smartphones is wireless. This can be either wi-fi or cellular data. It’s also very likely to switch between these two. Like when you’re walking out of your house, you lose your wi-fi connection and switch to 4g, and once you arrive at the office you connect to their wi-fi. Apps that are dependent on a connection need to be able to handle switching to different networks, and handling a slow and fast connection. The best way to test it is to walk out of the building, but to simulate a slow connection you can grab the emulator again and edit the connection settings.

Other types of testing

Besides I SLICED UP FUN there are more types of testing, you can test for the impact of settings, like setting the clock of the device to am/pm or 24h. This might sound alien, but I personally encountered that when testing the postal service app that showed the expected time of arrival, that once the clock switched to PM the app couldn’t show the expected arrival time and crashed. It took a long time to figure this out because none of my test devices had this setting.

Another one is time zone settings, I once tested an app that showed the programming for a radio station, and depending on the time zone, it showed the moment the show would be on in that specific time zone, this was a clear case of “it’s not a bug, it’s a feature” but wasn’t designed like this.

Power saving mode is another one to test. You can set the emulator to 1% battery and you’ll see the devices will cut all resources to save power. I really suggest to use the emulator for this because it’s very hard to keep a real device around 1% battery power. And whilst you’re on power saving, make sure to check that your apps power consumption is on an acceptable level, maybe it’s burning resources.

If your app uses notifications, make sure you test the do not disturb mode, if your app manages to bug the user while he or she doesn’t want to be disturbed, your app is likely to be deleted.

If your app uses a server it’s smart to perform some stress testing to make sure that you can handle the expected load and more.

Depending on your user group and location you might need to perform compliance testing, like how to handle private user data.

One thing that is often overlooked is accessibility, your app should be built to be accessible for the blind, the color blind etc. And as a tester you can test the voice over, but for the sake of your colleagues use headphones while testing voice over.

Some other testing types are;

• Database testing
• Internationalization
• Security
• Limited resources, storage, RAM, battery, network
• Field testing (Go out of the building)

And last but not least there is exploratory testing, in the past this was looked at as unstructured testing, but actually when you do it right, it’s structured and very valuable. It does require some planning, plan what you want to see in the app, and write down what you did and what catches your eye. Also, timebox each session and make sure to stay on track. If something catches your eye you can return later and explore this some more.

You can organize a bug hunting session with the team, the whole team takes a phone and starts going through the app looking for issues.

Test strategy

All of these methods of testing are wonderful, but testing is more than the test execution. As a tester you need a complete strategy to work effectively and efficiently.

The strategy consists of the following items for me:

• Scope
• Device labs
• Priority and risks
• Test design
• Test execution
• Reporting
• Release
• Maintenance

Scope

The scope answers the question about what to test, is your focus solely the app (frontend) or does it include the API and backend? I’ve worked on many projects where besides the app I also tested the API’s, mostly using postman or Rest assured. But once I worked on a project where I was only testing the app, and I wasn’t allowed to spend time during the sprints to test the API because that was a separate team, that didn’t have a tester. Stubborn as I am, I did spend time to teach these developers how to set up testing for themselves and the quality of the API’s improved so much! Even though it wasn’t my direct responsibility I feel that everyone who’s working on a part of the product should feel responsible for the entire product. In the end, when the users can’t log in due to an API error, the app receives 1-star reviews because “it’s” not working.

Device labs

I recognize three different types of devices labs which are, simulators/emulators, real devices that are present ad-hoc and cloud / remote labs.

First of the difference between the emulator and simulator, an emulator imitates the hardware while a simulator looks like the real device but uses the hardware of the host.

For instance, iOS Simulator uses apps that are built specifically for intel chips because MacBook’s (used to) use intel chips while iPhones use ARM chips.

It’s important to note that it’s not sufficient to only test on emulators and simulators, because you do not proof that the app is working on a real device.

The real ad hoc devices can be based on a Bring Your Own Device strategy, where everyone tests on their own device, but this might not represent your user group. Another possibility is to purchase or rent a bunch of devices for testing. Having real devices does ask for maintenance of the devices and to add and remove devices from time to time.

The cloud / remote device lab has a ton of devices available that you access via the browser or API, you can also hook your automation test set up to these devices. I often use Saucelabs as a remote device lab, and the possibilities are growing over time. Nowadays it’s possible to prepare images that are scanned or photographed by the device for testing, and you can actually remote debug the devices.

It’s clear that there are a lot of devices to choose from, but for testing it’s impossible to test everything on every device out there. You will have to make choices. If your app uses analytics it gives you great insight to what your current user base is using, you can see the device vendor, model and software version. In that light it’s not just important what your users are using right now, but what they used to use and will use in the future. To give you a general idea of vendor separation I’ll share some number with you.

In this chart you see the device vendors over time for The Netherlands, which is where I’m from, the biggest and thus most important vendors are Apple and Samsung. The next in line is Huawei, which was growing steadily and would have grown even bigger if it weren’t for the US trade limitations. For me it sounds logical to add a Huawei device to the test set, but bear in mind that today’s Huawei devices come without the Play Store installed. If you want to support Huawei devices you will have to release your app to Huawei’s App Gallery as well. Another vendor that is steadily growing in The Netherlands is Xiaomi, this company that started out with the MIUI custom firmware Rom, which I remember installing on my Galaxy S back in the day. Nowadays it builds not only smartphones but actually any smart device for your home, and their smartphones sometimes surprisingly reach the top of the ranks for DXOMARK smartphone camera reviews. Given the stable growth of the vendor I would surely consider adding a Xiaomi device to my test set. As I said before the market for The Netherlands might be very different from what’s relevant for you, for instance India has a very different market.

The market for India is much more divided than The Netherlands, nowadays Xiaomi is the biggest phone vendor, followed by Samsung. But other vendors like Oppo and Mobicel are big as well. A famous worldwide vendor like Apple is quite small in India and has been for years. It might not be worth the effort to build an app for iOS in India but instead just focus on Android.

It comes down to making decisions, what platforms will I support? What OS versions? On which devices will I test, and which browsers will I use?

How was the market divided half a year ago? And how will the market be divided in half a year?

Test design

Figuring out what to test and with what gravity it’s good to discuss it with your team. I suggest you organize a risk assessment session. Where you talk about what the risks for the project are and how to cover them. I will give you some pointers to set it up, the credits for this method go to Angie Jones. First, I distinguish four main subjects, risk, value, cost-efficiency and history.

Risk — probability vs impact

Value — Distinctness vs Induction to action

Cost-efficiency — quickness vs ease

History — Similar to weak area’s vs frequency of breaks

Next each subject is put on this graph, on a scale from 1 to 5, probability, how often is the feature used? And Impact, how pissed off is the user when the feature doesn’t work.

For example, a log in feature that is required to use the rest of the app, it’s used every time that the app is used so it’s a 5, and if users can’t do anything in the app they’ll be really pissed off, so also a 5. But a FAQ page is probably not visited that often so a 1 or a 2, and if it doesn’t work the user might find their answers somewhere else so also a 1 or a 2.

The next subject is Value, distinctness and induction to action. Distinctness is about the question, does testing this cover parts of the app that aren’t covered in other tests. In this example you need to log in to access other parts of the app, but testing it doesn’t cover anything else, so distinctness is low. To access the FAQ page, you would need to log in, so it’s a little higher than log in.

The induction to action answers the question how quickly this feature would be fixed if somehow it broke. If it would require an immediate hotfix it would be a five, to fix it in the next release a 3 or a 4 and If the fix is going to the backlog it could be a 1 or 2. If I look at the Log in feature, I think It would be in a hot fix thus it’s a 5. The FAQ page would probably end up on the backlog so a 2.

The third subject is Cost-efficiency, in general testing is expensive (not testing is even more expensive but that’s another story) it’s important to do it efficiently and also cost-efficient. In terms of automation the questions are, how quickly can I automate this and how easy is it to automate. In our chart the quicker and easier it is to automate the higher the score will be. Our examples are the log in feature, which should be easy, like filling in a username, a password, hit login and validate the flow. I expect this to be really easy and really quick so both get a 5. For the FAQ page, it sounds easy as well but the content of the FAQ could change every now and then, so it might be a bit harder and take more time than the log in feature, so I give it two 4’s.

The last subject is history, the project could be going on for a while and when a new feature is introduced it might be similar to some of the more existing features especially the weaker areas. If it’s similar to other weak areas you can expect that the new feature is going to need some extra attention as well, so the more similar it is to other weak areas the higher the score for ‘similar to other weak areas’. The new feature might be part of a bigger feature or it’s a change to an existing feature. In that case you can have a good estimation of the frequency of bugs, how often do bugs occur in the feature, the more bugs in the feature the higher the score.

In light of the example the Log in feature might be a pretty unique feature so it could be a 2, and since I don’t know the frequency of bugs, I’ll give it a 3. The FAQ might be a bit more similar to other features so I’ll give it a four but for fun I’ll put the frequency of bugs to a 1.

I gave all these scores to the two features, and while it was a lot of fun, we should do something useful with it too. First, we’ll sum up the scores to see their final scores, for the log in feature that is 25 + 5 + 25 + 6 = 61 and the FAQ, 1 + 4 + 16 + 4 = 25.

The FAQ page has a low score of 25 so testing should resort to unit testing and a basic manual test. The log in feature has a medium score which means it gets unit testing, manual testing and a happy flow of UI automation.

What happens when you get to a score but you’re not happy with the result? Well, if you have a very good reason you can decide to do something different than what the score says. Maybe a feature has a medium score but you’re sure it will need extra testing, then by all means do that.

Test execution

The biggest part of testing is the test execution, when it comes to mobile testing there are two requirements, the test object or system under test and the tools for testing.

The test object is the app, so you’ll need a test environment, either mocks, production or test environment. If you scope is broader, you’ll also need access to the API and access to the backend.

The test tools that you’ll most likely need for Android are;

• ADB (Android debug bridge) a tool for debugging, you can use it to install an app, read logs, send and receive files from a device.
• Android Studio, the IDE for developing android apps. It also manages emulators, and is the place to create and run the automated espresso tests.
• Charles proxy, a tool to read traffic from your app to the servers, it has lots of features to change the traffic so you can test your app with different responses from the backend
• Vysor, a tool to mirror your Android device to your desktop, it’s very useful to make screenshots, screen recordings, demoing the app and even controlling the device with your desktop
• Postman, a tool to fire API calls, you can use it to test the API’s related to your app

The test tools that you’ll most likely need for iOS are;

• XCode, the IDE for developing iOS apps. It also manages simulators and is the place to build and run XCUITests.
• Apple configurator 2, a tool to install IPA files from your desktop on your iPhone, and it’s also possible to maintain devices with specific profiles.
• QuickTime can be used to create screen recordings of your iPhones.
• Charles proxy
• Postman

Reporting

After test execution you will need to report about your progress, this means you probably need to log bugs, communicate about bugs and advice about the launch of a release. It’s important to agree with your team about how you report bugs and report about a release, it could be a full scale 15 pages report or a short sentence in the chat application of your team, as long as everybody agrees it’s fine.

For bug reporting a couple of things are important to mention;

• App version (System under test)
• Test device, model, OS version
• Steps to reproduce
• Expected and actual behavior
• Screenshots / screen recordings to reproduce

Release

Before you can release the app, you need to have a look at the submission guidelines.

For iOS they can be found here: https://developer.apple.com/app-store/review/guidelines/

For Android they can be found here: https://developer.android.com/distribute/best-practices/launch/launch-checklist

Your app will be reviewed by Apple before it is available in the store, if it doesn’t follow the guidelines the app will be rejected and your release is delayed.

Besides the guidelines, you should also perform a regression test, to see that all the features that you haven’t worked on are still working as you would expect. There might be stuff that needs to be taken care of before you can go live with a new app or new version of your app, things like screenshots and copy for the stores, environment variables, feature toggles, all these things should go on your release checklist.

Both stores provide a Beta program, iOS TestFlight and Google Play Alpha and Beta, here you can invite real users to receive early feedback about your app before sending the app to all of your users.

Maintenance

Once your app is live you come into the maintenance stage, users will provide you with feedback and app ratings, it’s important to monitor the feedback and ratings to make sure your users are happy with your app and it matches their expectations. It’s also wise to implement monitoring about crashes and errors that occur, you really do not want the app to crash for your users so always make time to investigate and fix crashes in production.

Test automation

Test automation is not really the topic of this story, but I’d still like to tell you a bit about it. Test automation consist mainly on three levels, unit testing, service level (API) testing and UI testing. Unit tests are a part of programming and code that test the code, these are executed very quickly. API tests focus on the service layer between the head (app) and the backend. The UI tests are like a robot that interacts with the UI, it takes longer to create and maintain these tests, they are usually flaky and it also takes longer to execute them.

Unit tests are robust and executed quickly, so it makes sense to have a lot them, in sense of code coverage, the amount of code that is covered with unit tests it’s a good practice to keep it at 80%, much more would most likely mean that you’re writing tests just to get the coverage than actually adding value to the product. The API tests can cover a lot of the calls that are used by your app, for both platforms! Most of the time the API is available before the front end is, so you can already automate the API tests before looking at the UI, you could use Postman as I’ve shown earlier, or build a test framework using REST Assured in Java.

For UI automation the choice for a framework depends on your system under test, generally I would pick XCUITEST for a native iOS app, Espresso for a native Android app and Appium for a webapp, hybrid or apps build with platforms like React Native. Whichever you choose, choose wisely, look for tools that have broad community support and don’t be afraid to try something you’re not familiar with yet.

Closure

This concludes my story about functional mobile app testing. I hope you had even more fun reading it than I had writing, and hope it taught you some valuable knowledge.