iOS apps Testing
Testing native iOS apps has had a bad rap in the past, especially when it comes to test automation. It has proven to be difficult, time-consuming, and the tools were not as capable as they are on other platforms (most notably Java). Positive note, the round trip from making an edit in the code, to compiling, to deploying and debugging is extremely fast when using the iOS Simulator. This helps cut test execution time down considerable compared to Android. Great strides have been made since iOS 8 and Xcode 6 have been out, so if you have not looked into automated testing of your iOS apps in a while, you may want to take a second look.
XCTest – Out of the box, iOS developers have access to a respectable unit testing framework know as XCTest. Think of unit testing as testing the code from the inside out. Looking at the internal operations under the covers, a well written suite of XCTests can quickly point to where problems are in your code base as you update to newer iOS versions. By default, all projects created from within Xcode generate an initial test suite that developers can expand upon.
OCMock/OCMockito – Mocking is a testing technique used to mimic situations that are difficult to reliable create in a repeatable and predictable fashion during the execution of a test. An example would be to replicate a trade on Wall Street or a bank transaction, something you may not want to do for real over and over again all day long. OCMock and OCMokito are two popular testing frameworks that allow developers to create more reliable testing environments that do not depend on the availability of complex back-end systems in order to provide an assessment of the overall health of the code.
Xcode Service and Continuous Integration – Continuous Integration refers to a technique in development that all code changes that are saved to the system (referred to as the code repository) are compiled and tested on an automated and continuous basis. Monitoring the overall health of your code can alert developers when a potential problem is first introduced to the code. There is typically a very visible and accessible report that allows developers to know exactly when something has gone wrong during development. That way there is nothing that any one developer can do without every developer instantly knowing about it; good or bad. XCode Service has made monitoring the health of iOS project as easy as possible.
Automating UI Testing with Instruments – Apple has always provided a rich set of instruments that helped developers know what was going on inside its apps. From memory leaks, random crashes, and battery drains, instruments have proven to be an invaluable tool when it comes to writing high quality and dependable apps. Automating the testing of the user interface is no exception. Where unit tests test the app from the inside out, user interface tests test the app from the outside in. Now developers can create test scripts that will execute tests over and over again against the user interface to ensure that changes they make do not break anything they thought they were done developing.
Android App Testing
The biggest challenge to automated testing on Android has been test execution time. Even when leveraging the maturity of testing technologies that have evolved on the Java platform, it can still take hours for a suite of tests to complete. This is due primarily to the use of emulators in testing: emulators can slow development to a crawl at times.
Tethering a device to an automated test server can take a test device away from a developer or tester, but does prove to run things a lot faster. There are faster emulators out there like GenyMotion, but even then, looking for an even faster solution can make the difference in how often the tests get executed by members of your development team.
Another difference you will find when comparing iOS to Android is that for iOS, there is typically one and only one way to do things: the Apple way. With Android there are almost always more than two or three ways to accomplish about the same thing; unit testing, mocking, continuous integration and user interface testing.
JUnit/JUnit 4 – While the default Android testing API for unit testing, AndroidTestCase, is based on the JUnit 3 API, you can now consider switching to JUnit 4 to unit test your Android apps by using AndroidJUnitRunner. Like XCTest on iOS, JUnit is a unit testing framework meant to test how your app works on the inside. This is a developer’s first level of defense when trying to keep the quality of code that worked perfectly well when it was first written, from breaking some time down the road.
Mockito/Robolectric – While Mokito is more of a traditional mocking framework that can be used on the device or within an emulator, Robolectric is more of a replacement of the emulator or device entirely. Almost an “uber mock” if you will, Robolectric allows developers to execute tests almost anywhere. The advantage that Robolectric brings to the table is speed, you will find that your tests perform much faster, sometimes an order of magnitude faster. You can almost consider it more of a simulator replacement for Android’s emulator that provides just enough support to create a testing environment stable enough to provide value to your testing activity. Value that typically degrades when tests take way too long to execute.
Jenkins/Bamboo – Continuous integration servers that check the health of your Java code have been around for years. Technologies and servers have come and gone many times over. Two of the more popular choices for introducing a Continuous Integration environment to your Android projects are Jenkins and Bamboo. Both provide a means of monitoring your code base, automatically executing tests, and alerting developers when something has gone terribly wrong.
Calabash/Appium – While not entirely exclusive to Android as a User Interface testing harness, both Calabash and Appium provide a means of testing apps from the user interface. The ability to test the app from the outside in complements a unit testing strategy providing a more comprehensive strategy when it comes to monitoring the health of your apps code. One without the other will prove to leave gaps in your automated testing strategy that you will need to compensate for with your manual testing strategy.
Testing native iOS apps has had a bad rap in the past, especially when it comes to test automation. It has proven to be difficult, time-consuming, and the tools were not as capable as they are on other platforms (most notably Java). Positive note, the round trip from making an edit in the code, to compiling, to deploying and debugging is extremely fast when using the iOS Simulator. This helps cut test execution time down considerable compared to Android. Great strides have been made since iOS 8 and Xcode 6 have been out, so if you have not looked into automated testing of your iOS apps in a while, you may want to take a second look.
XCTest – Out of the box, iOS developers have access to a respectable unit testing framework know as XCTest. Think of unit testing as testing the code from the inside out. Looking at the internal operations under the covers, a well written suite of XCTests can quickly point to where problems are in your code base as you update to newer iOS versions. By default, all projects created from within Xcode generate an initial test suite that developers can expand upon.
OCMock/OCMockito – Mocking is a testing technique used to mimic situations that are difficult to reliable create in a repeatable and predictable fashion during the execution of a test. An example would be to replicate a trade on Wall Street or a bank transaction, something you may not want to do for real over and over again all day long. OCMock and OCMokito are two popular testing frameworks that allow developers to create more reliable testing environments that do not depend on the availability of complex back-end systems in order to provide an assessment of the overall health of the code.
Xcode Service and Continuous Integration – Continuous Integration refers to a technique in development that all code changes that are saved to the system (referred to as the code repository) are compiled and tested on an automated and continuous basis. Monitoring the overall health of your code can alert developers when a potential problem is first introduced to the code. There is typically a very visible and accessible report that allows developers to know exactly when something has gone wrong during development. That way there is nothing that any one developer can do without every developer instantly knowing about it; good or bad. XCode Service has made monitoring the health of iOS project as easy as possible.
Automating UI Testing with Instruments – Apple has always provided a rich set of instruments that helped developers know what was going on inside its apps. From memory leaks, random crashes, and battery drains, instruments have proven to be an invaluable tool when it comes to writing high quality and dependable apps. Automating the testing of the user interface is no exception. Where unit tests test the app from the inside out, user interface tests test the app from the outside in. Now developers can create test scripts that will execute tests over and over again against the user interface to ensure that changes they make do not break anything they thought they were done developing.
Android App Testing
The biggest challenge to automated testing on Android has been test execution time. Even when leveraging the maturity of testing technologies that have evolved on the Java platform, it can still take hours for a suite of tests to complete. This is due primarily to the use of emulators in testing: emulators can slow development to a crawl at times.
Tethering a device to an automated test server can take a test device away from a developer or tester, but does prove to run things a lot faster. There are faster emulators out there like GenyMotion, but even then, looking for an even faster solution can make the difference in how often the tests get executed by members of your development team.
Another difference you will find when comparing iOS to Android is that for iOS, there is typically one and only one way to do things: the Apple way. With Android there are almost always more than two or three ways to accomplish about the same thing; unit testing, mocking, continuous integration and user interface testing.
JUnit/JUnit 4 – While the default Android testing API for unit testing, AndroidTestCase, is based on the JUnit 3 API, you can now consider switching to JUnit 4 to unit test your Android apps by using AndroidJUnitRunner. Like XCTest on iOS, JUnit is a unit testing framework meant to test how your app works on the inside. This is a developer’s first level of defense when trying to keep the quality of code that worked perfectly well when it was first written, from breaking some time down the road.
Mockito/Robolectric – While Mokito is more of a traditional mocking framework that can be used on the device or within an emulator, Robolectric is more of a replacement of the emulator or device entirely. Almost an “uber mock” if you will, Robolectric allows developers to execute tests almost anywhere. The advantage that Robolectric brings to the table is speed, you will find that your tests perform much faster, sometimes an order of magnitude faster. You can almost consider it more of a simulator replacement for Android’s emulator that provides just enough support to create a testing environment stable enough to provide value to your testing activity. Value that typically degrades when tests take way too long to execute.
Jenkins/Bamboo – Continuous integration servers that check the health of your Java code have been around for years. Technologies and servers have come and gone many times over. Two of the more popular choices for introducing a Continuous Integration environment to your Android projects are Jenkins and Bamboo. Both provide a means of monitoring your code base, automatically executing tests, and alerting developers when something has gone terribly wrong.
Calabash/Appium – While not entirely exclusive to Android as a User Interface testing harness, both Calabash and Appium provide a means of testing apps from the user interface. The ability to test the app from the outside in complements a unit testing strategy providing a more comprehensive strategy when it comes to monitoring the health of your apps code. One without the other will prove to leave gaps in your automated testing strategy that you will need to compensate for with your manual testing strategy.
References:
No comments:
Post a Comment