Mobile web automation testing with Appium
Nowadays the market share of mobile usage is more and more important. In recent decades it has resulted in a steady rise of 48% growth (source). As TUI/Musement is a customer-centric company, the importance of quality in terms of mobile testing cannot be ignored. This can be achieved in different ways such as manual tests. Manual testing can help ensure the quality of a product however it isn’t scalable and most importantly it doesn’t allow a good ROI (see this interesting study about the impacts of test automation over cost, quality and time to market).
Times have changed, and today’s users are on a multitude of devices ranging from desktop computers, laptops, mobile phones, tablets, and there are too many browsers and browser versions to count at this point. This means that the interoperability in our test strategy could not be overlooked. For this reason, starting with an existing defect found on a mobile phone we decided to try Appium. We experimented with Appium as a testing framework to understand the possibilities of introducing it as the very first automated mobile testing system at TUI/Musement and also to share our impressions with the testing community.
How does Appium work?
Appium is a server which receives commands from automated script (our test suite) and translates it in a “language” understandable by the mobile devices. Simultaneously it is able to receive responses from the mobile devices and translates it to the automated scripts. In layman’s terms it acts as an anti-corruption layer.
Our problem in a nutshell
We were informed of the presence of a bug in one of our venue pages reproducible with an iPhone on Chrome. While scrolling down and clicking many times on the “View more” button to see more experiences, some of them were shown duplicated.
Our goal with Appium was to reproduce the defect with an automate by simulating the customer’s actions on that page with both iPhone/Android by introducing it as a regression test prior fix from the development team.
Why Appium and not another tool?
To answer this question we could of course fall into the trap of the subjective answer (like why did you choose to buy a Ferrari and not a Porsche?) but in a very pragmatic approach these are the specifics that caught our attention:
- Open source
Although there’s also a paid version “for those with cash to burn” you can definitely leverage many things in terms of testing automation just by using the Open Source version, plus it has a huge community support. From what we understood a drawback is that its configuration can sometimes be complex and tricky to implement, so a bit challenging but free (for a further benchmark of the mobile testing framework check this out). - Plethora of programming language supported
It can indeed support a lot of webdriver compatible programming languages: C#, Clojure, Java, JavaScript with Node.js, Objective-C, Perl, PHP, Python, and Ruby - Cross platforms
It supports automated testing on iOS, Android, and Windows - Great potentialities in terms of automation of mobile gestures
Most of the actions that can be done with a mobile device phone (both physical or virtual) can be automated.
For example, you can go and automate changes on the phone settings or you can open other apps during the testing phase. It therefore works with:
• Native iOS app (Swift or Objective C)
• Native Android app (Kotlin or Java)
• Cross platform app (Xamarin, React Native, …)
• Mobile Browser (our current use test case) - Great flexibility
Appium is just a server that forwards commands to the mobile phone, so it leaves complete flexibility on how to implement the test suite, see examples:
• Cucumber Ruby Browserstack
• Robot framework - We had a veteran at home
My quality assurance travelling companion Thomas used it for many years for medical mobile applications.
What approach did we use to solve the problem?
As fervent disciples of the Agile methodologies we followed an iterative and incremental approach. Basically, make it work then make it better was our motto.
- Make it work
We configured Appium on our local environments then we configured and installed the mobile device emulator (note that with Appium you can of course plug in real devices). We chose the most suitable programming language. Which in our case was Javascript as we’re using it with TestCafe in different projects and in some other monitoring as code projects. As a last step we wrote the algorithm to reproduce the problem. In that step we came out with some issues, luckily solvable (see next chapter). - Make it better
Once we solved all the set-up problems and once the algorithm worked we wanted to introduce a BDD based framework for our test because we wanted to have a test described in a readable and understandable format for the 3 amigos: Cucumber JS
It is quite straightforward to use plus it has a good documentation and it is very similar to Behat, which I personally used in other PHP projects (both use Gherkin language). We decided to follow Martin Fowler’s paradigm by creating a Venue Page object with the methods needed for our test (code can be found here).
What difficulties did we meet in its implementation?
Look before you leap
We used Chrome 83.0.4103 on the iOS emulator (as the original defect) and we discovered that in order to allow Appium to be able to interact with your browser you need a correct compatible Chromedriver version installed locally. This was not taken for granted; indeed, we had this issue:
Luckily Appium is quite clear when an error occurs, so we read the documentation about Chromedriver’s integration and the following command allowed us to pick the correct Chromedriver for the version of Chrome under test.
Shadow DOM what the heck is this?
You may be familiar with the shadow in a DOM document but we weren’t and since we had to close the cookie consent banner on the Venue page’s opening, and as that banner was indeed in a shadow root we had to understand how webdriver could interact with such elements. Luckily there’s an ad-hoc API to use.
CPU usage
If you do decide to use Appium you need to be aware that running it locally with a mobile emulator requires high CPU load (it also can sometimes lead to possible memory leak issue). We did not do an in-depth analysis to solve these issues; we just ran it with the basic set-up but looking at Appium official forums it seems we’re not the first to point this out, even though solutions may exist to bypass it.
Async and await in loop can hurt
As every action we did were asynchronous, and modestly none of us are Javascript experts we had to struggle a bit with async/await especially when implementing the algorithm to find duplicate experiences. This nice article really helped.
Demo time
Thinking about the future with Appium
Appium is a really nice framework which offers great prospects though it isn’t easy for everyone and requires you to be a real “computer geek”. It is indeed just one of the many pieces of the puzzle hence I recommend you to consider the overall picture before adopting it. That said, I think that with a little experience, a little bit of elbow grease and a major step forward you can very quickly set-up a mobile automation testing suite. In our case the following step will consist in its containerization in order to introduce a continuous delivery pipeline. We know that while doing it we’ll need to decide if we want to handle and maintain the devices emulators installation or if we want to rely on services such as saucelabs which provides real devices automation tests with Appium. Last but not least we will likely want to extend the mobile app automation testing (such as Musement app for Android or iOS) or even to hybrid apps. It won’t be easy, but not impossible either.
A huge thank you to Thomas for his support!
Interesting links:
- a nice video about Automating Hybrid Applications with Appium
- Appium boilerplate
