Continuous Integration and Code Coverage in Android using Travis CI

Testing has always been a trending topic in the world of Android. It becomes a time consuming task in large android projects where multiple developers are working together, maintaining the code quality and resolving bugs; which is a very difficult task.

This is a very serious issue in the open source projects where multiple developers from different environment and different level of expertise contribute to a single code base. Everyone is not the perfect and someone can commit changes that break the application code and we should have something that can alert us if someone is doing something wrong as soon as possible.

It would be great if we can find out that whether the code we are going to push onto our master branch of repository contains any possible bugs that can break the system. Here, continuous integration comes into play.


What is Continuous Integration?

Continuous Integration (CI) is a development practice that requires developers to integrate code into a shared repository several times a day. Each check-in is then verified by an automated build, allowing teams to detect problems early.

In simple words, each time you commit and push your code to the remote repository, continuous integration will check if everything is working correctly in the code before you merge it into your master (assuming you are using git for version control).

With continuous integration, you can:

  •  Automated build process. Every commit/push to a git repo will automatically trigger a new build.
  • Find out Android Lint errors.
  • Coding styles to ensure that the new code is matching the code format that your team has decided.
  • Get the code coverage percentage. This will be really helpful to figure out if any part of the code is remaining to test.
  • Run Unit tests located on your /test directory of project.
  • Run Instrumentation tests located in your /androidTest project.

How to configure Continuous Integration?

As the title of this post suggests, here I am going to use Travis CI. You can use other services available to the market like CircleCI, Codeship, etc. But here, I will stick to Travis CI as it is most used CI service for GitHub projects and also it is free for Open Source projects.

Prerequisites:

  • An account on GitHub. I believe nowadays every developer has it.
  • An account in Travis CI. If you don’t have it yet you can create it from here.

Let’s start with the integration. For the demo purpose, I am going to use API 25 as the targeted Android version in our Gradle and we are going to test our build on API 21 and API 19.

We have skipped the description of some lines as we don’t want to make this article too long and boring. We will concentrate more on the lines which are more important and difficult to understand.

Step-1 : Creating .yml file

First, create a .travis.yml file into your project’s root directory. This file will contain the configuration script for the automated build testing.

1fshca7ehyvosixq8sczwca

Step-2 : Basic setup:

Add above lines at the starting of the file. This basically tells Travis that this repository is an android application source code and it has to use Java 8 for compilation. The notification section will provide email notifications on the email you provided whenever a build gets completed successfully or fails due to some error.

Step-3 : Defining android targets:

Next, add above lines to your .travis.yml file. The line under the matrix section defines the Android API version and the processor architecture, on which the code will be tested. These lines re-executes the script for each variable. So if I write another line with: “– ANDROID_TARGET=android-16 ANDROID_ABI=armeabi-v7a”, it will execute the whole script with the 21, 19 and 16 APIs. We can say it is really useful to test different API levels. So, each commit you push to the repository will start two separate builds and test prepossess. You can see this in Travis dashboard like below.

1xrxjad5hawop4pgcypscpa

In the global section, you can declare any global environment variable that is required to successfully complete the build process. This field is optional. Here, we defined “ADB_INSTALL_TIMEOUT =10”. This will increase the ADB connection timeout to 10 mins. Normally, it is 2 minutes by default.

Step-4 : Setting up the build configs

The android section contains the build related configurations. Specify the build tools version, targeted android API and extra repository you required to complete the build process like play services, m2repository, etc. At the end, specify emulator system images on which application testing will be performed. If you are going to use more than one API levels like this example, remember to define all the emulators needed. In our case, we are going to perform a test on API 21 and 19. So “sys-img-armeabi-v7a-android-25”, “sys-img-armeabi-v7a-android-18” are required emulator images.

Step-5: Enabling caching of resources:

Travis can cache directories that you need to speed up the subsequent builds by caching the Gradle dependency folders. To enable the caching, you have to specify which directories you want to cache under the cache section. Here, I am caching some of the .gradle folders.

Step-6: Writing build script:

In every project, we need to run the instrumentation tests. So we will run the emulator using “emulator -avd test -no-skin -no-audio -no-window &”. “ android-wait-for-emulator” script will wait until emulator is up and running. This will print “Waiting for emulator” in Travis log until emulator starts. The next two lines are printing the available devices for our information and sending the key “menu” to the emulator to unlock it.

Make sure you are using supported android targets and android ABI. You can list out supported android targets by running “android list targets”. You will get list of the supported android targets for given build environments in Travis log window like below:

1*he10rbUQvNTLQQ1IRAUeDQ.png

Step-7: Enabling code coverage:

Code coverage is the best way to motivate you and your team to write more tests. Once we launched the tests and connected tests on Travis, we can generate the code coverage report with all the results. Once the jacoco (Java Code Coverage library) report is generated, it uploads all the reports tothe codecov server. You can see these reports online. (If you don’t know how to integrate jacoco in your android project, we found this article very useful. Or you can see the sample project, created by the codecov team.)

To run the code coverage after every successful build, you can integrate codecov by adding below lines in .travis.yml.

Congratulations! We completed all the required changes in our repository.

Your final .travis.yml file should look like this:

Step-8:

We are almost there. Stay with us. Now, sign in to Travis CI website and go to “Add new repository”. Select the repository in which you integrated .travis.yml file and enabled the Travis integration.

1ul3byu0joekdm2f68wqn7w
Enable travis integration by clicking the switch

That’s it. You are ready to push your commit with .travis.yml file.


Run your first build:

As soon as you push the commit to your GitHub repository, Travis will start building the project automatically. You can see the past build history by going to the “Build history” tab.

1utqdiqolts5wgkelt8ua-g

You can see your build status (whether it is running, passed or failed) right behind your GitHub repository commits list.

1*f4OxepYumczZgVvwV1cmqw.png

At this stage, if your build fails, try to find out what went wrong. You can find answers on how to fix those bugs on StackOverflow or paste your error in the comments below. We will try our best to resolve those the errors.

Here is one sample project to help you:

You can also find the test project that we used for the demonstration purpose in this tutorial in the below link. Go ahead and take a look into it.

https://github.com/multidots/Android-Automated-Test-and-CI-Demo

This is an sample application which validates email address you entered in edit text. Here, we integrated Travis CI and CodeCov using Jacoco. You can see all the build statuses in Travis CI from here. If you want to see the latest code coverage report for the project, you can find it over here. This will give you an idea of how code coverage report looks like.


Conclusion:

Continuous Integration (CI) is one of the good practice while developing the code. This lets you and your team concentrate on development and reduces the efforts required for maintaining the code. By outsourcing the maintenance of your test infrastructure, you can remove a whole bunch of work from your engineering team that can then be used to build your product. On another hand, code coverage gives you better idea of the amount of code remaining to test and inspires your team to write better test cases. As an overall effect, they both together help you and your team to develop robust product and allows you to deploy your code with confidence.

Code your next android app using RxJava

RxJava is new hot topic in the world of Android Development. The only problem is that it is very difficult to understand. Especially, Functional reactive programming is very hard to understand when you come from Object Oriented Programming. So, I am developing a series of articles to help you understand the basic concept of Reactive Programming.

In the previous Part 1we get the basic concept of the reactive programming. If you did not read yet, I will encourage you to first go through Part 1 and get the basic idea about reactive programming.

As we discussed in Part 1, Rx is made up of three key component.

RX = OBSERVABLE + OBSERVER + SCHEDULERS

Let’s create each component one by one. But, first let’s start with integration.


Integrate RxAndroid in project

RxAndroid is basically an android specific wrapper that provides some functions explicit to android and those are not available in RxJava. So, if you want to integrate Rx into your java project only, you don’t need to implement RxAndroid library into your project.

Here is the gradle dependency of RxJava and RxAndroid.

The version of both the library may change, depends on when you are reading this article. You can find latest version from here.


Marble Diagram

Rx uses marble diagram to explain how any operator works. Marble diagrams are very convenient and easy to understand.

  • The line above the box indicates the raw data. This data needs to be emitted by the observable. Different types of objects are display with different shapes.
  • Box in the middle indicates operator. There are many other operators available, that basically controls when and how the observable should emit the data. We are going to look into those operators in future.
  • The line below the box indicates the data that is emitted by the observable. Observer in Rx receives this emitted data.

filter


Let’s create Observable

As you know observable is nothing but a function that emits the data stream. Below is the observable that emits 1 to 5 one after another.

Here just() is called as an operator. It emits the values provided in arguments. (And that’s why they named as just.)

0bhyoyg_o-glxlikv

Sometimes we want to refine specific event only to be emitted by observable. Let’s say in our above example we only want to emit only odd numbers out of the observable. We can achieve this thing using another operator called filter(). As the name suggest filter operator filters items emitted by an Observable.

06oy3ienx8ysblul


Creating observer

Observers consume the data stream emitted by the observable. Wheneverthe observable emits the data all the registered observer receives the data.

In RxJava there are three callbacks you are going to receive in an observer.

  • onNext() : This method will be called when there is any new data emitted by the observer. The object that is emitted by the observable can be found in argument parameters of this callback.
  • onError() : You will receive this callback whenever there is any error occurred on observable. (After all the world is not perfect.)
  • onComplete() : Whenever observable is done with emitting the data streams, you will receive this callback. This indicates that there is no more data to emit.

In many cases you don’t care about onCompleted() or onError(). So instead of using Observer<T> we can use a simpler class to define what to do during onNext() using Action1 class.

Here, onCall() is equivalent to onNext() in our first approach.


Manage concurrency using Scheduler

As you know scheduler in the reactive programming manages the concurrency.

In Android the most common operation when dealing with asynchronous tasks is to observe the task’s result or outcome on the main thread because you want to update the UI. Using vanilla Android, this would typically be accomplished with an AsyncTask. But with Rx you can achieve this by using the schedulers.

There are two methods, which controls thread management.

  • subscribeOn() : By using this method you can define on which thread the observable should run.
  • observeOn() : By using this method you can define on which thread the observer should run.

RxJava and RxAndroid library provides some predefined schedulers. Like, Schedulers.io() indicates the IO thread. While Schedulers.newThread() will create a new thread to run observer/observable. You can find other various types of schedulers here.

Finally, we will use subscribe() to subscribe the observer to receive the data issued by the observable. This will return Subscription object that holds the reference to that specific connection between observer and observable.

Let’s see the output of above program.

16zdedxewtwsrasybxipjvw

We can see from the output that only odd numbers were emitted by the observer. At the end, when all the data are emitted, onComplete() got execute.

Unsubscribe:

If you want to unsubscribe the observer from observable, you can call unsubscribe.

In android this is essential that you call unsubscribe in onDesrtoy() of your activity/fragment to release the connection between observer and observable. Otherwise, it may call memory leak.

If you have multiple subscriptions in your class, then you can use CompositeSubscription to unregister all the subscriptions at once. Below is an example on how you can do that:


In the next part I am going to explain different operators used in RxJava. Meanwhile, If you liked the article, click the 💚 below so more people can see it!

twitter-follow

What is Reactive Programming?

 

Nowadays everybody is talking about Reactive Programming and you’re curious in learning this new thing called Reactive Programming. Maybe you’ve seen it used a few places but you’re still a little confused and would like some clarifications.

In this article, we are going to learn the basic concepts of the Reactive Programming. Starting from the next article we are going to do some real programming and learn how to use RxJava in the Android application development.

So, first let’s understand what are the problem we are facing? Why do we require Reactive Programming? Because if there is no problem, then we don’t need a solution right??


Why do we need Asynchronous work?

The simple answer is we want to improve the user experience. We want to make our application more responsive. We want to deliver a smooth user experience to our users without freezing the main thread, slowing them down and we don’t want to provide the jenky performance to our users.

To keep the main thread free we need to do a lot of heavy and time-consuming work we want to do in the background. We also want to do heavy work and complex calculations on our servers as mobile devices are not very powerful to do the heavy lifting. So we need asynchronous work for network operations.


The evaluation matrix:

evaluation-matrix

Let’s see what do we need from the library that handles all the asynchronous work. You can imagine below 4 points as the evaluation matrix for the asynchronous library.

  • Explicit execution: If we start the execution of a bunch of work on a new thread, we should be able to control it. If you are going to perform some background task, you gather the information and prepare them. As soon as you are ready, you can kick-off the background task.
  • Easy thread managementIn asynchronous work, thread management is the key. We often need to update the UI on the main thread from the background thread in the middle of the task or at the end of the task. For that, we need to pass our work from one thread (background thread) to another thread (here main thread). So you should be able to switch the thread easily and pass the work to another thread when needed.
  • Easily composable: Ideally, It would be great if we can create an asynchronous work and as we start spinning background thread, it just do it’s work without depending any other thread (especially on UI thread) and stays independent from the other thread until it finishes its job. But in the real world, we need to update the UI, make database changes and many more things that make threading interdependent. So the asynchronous library should be easily composable and provide less room for the error.
  • Minimum the side effects: While working with multiple threads, the other thread should experience minimum side effects from the other thread. That makes your code easily readable and understandable to a new person and it also makes error easily traceable.

What is Reactive Programming?

According to wikipedia:

Reactive programming is a programming paradigm oriented around data flows and the propagation of change. This means that it should be possible to express static or dynamic data flows with ease in the programming languages used, and that the underlying execution model will automatically propagate changes through the data flow.

In simple words, In Rx programming data flows emitted by one component and the underlying structure provided by the Rx libraries will propagate those changes to another component those are registered to receive those data changes. Long story short: Rx is made up of three key points.

RX = OBSERVABLE + OBSERVER + SCHEDULERS

We are going to discuss these points in detail one by one.

  • Observable: Observable are nothing but the data streams. Observable packs the data that can be passed around from one thread to another thread. They basically emit the data periodically or only once in their life cycle based on their configurations. There are various operators that can help observer to emit some specific data based on certain events, but we will look into them in upcoming parts. For now, you can think observers as suppliers. They process and supply the data to other components.
  • Observers: Observers consumes the data stream emitted by the observable. Observers subscribe to the observable using subscribeOn()method to receive the data emitted by the observable. Whenever the observable emits the data all the registered observer receives the data in onNext() callback. Here they can perform various operations like parsing the JSON response or updating the UI. If there is an error thrown from observable, the observer will receive it in onError().
  • Schedulers: Remember that Rx is for asynchronous programming and we need a thread management. There is where schedules come into the picture. Schedulers are the component in Rx that tells observable and observers, on which thread they should run. You can use observeOn()method to tell observers, on which thread you should observe. Also, you can use scheduleOn() to tell the observable, on which thread you should run. There are main default threads are provided in RxJava like Schedulers.newThread() will create new background that. Schedulers.io() will execute the code on IO thread.

3 simple steps to use Rx in your application

filter

Let’s look into the basic example. This will explain 3 simple steps to use Reactive programming in your application.

Step-1 Create observable that emits the data:

Here database is an observable which emits the data. In our case, it emits the strings. just() is an operator. Which basically emits the data provided in the argument one by one. (We are going to look into the operators in detail in our upcoming articles. So, don’t worry about them.)

Step -2 Create observer that consumes data:

In above code snippet observer is an observer that consumes the data emitted by the database observable. It processes the data received and also handles error inside it.

Step-3 Manage concurrency :

At the last step, we define our schedulers that manage the concurrency. subscribeOn(Schedulers.newThread()) tells database observable to run on background thread. observeOn(AndroidSchedulers.mainThread()) tells observer to run on the main thread. This is basic code for reactive programming.

So by now you should be able to understand, why we need reactive programming, why we need them and how we can implement them. In the upcoming articles, we are going to learn how to use RxJava and it’s operators in detail.


What next? (Part 2)

Now, visit the next part to start some programming example in RxJava and how to use RxJava in your Android/Java project.

If you liked the article, click the 💚 below so more people can see it!

Twitter Follow.png

Why you should learn Go?

Go will be the server language of the future.” — Tobias Lütke, Shopify

In past couple of years, there is a rise of new programming language: Go or GoLang. Nothing makes a developer crazy than a new programming language, right? So, I started learning Go before 4 to 5 months and here I am going to tell you about why you should also learn this new language.

I am not going to teach you, how you can write “Hello World!!” in this article. There are lots of other articles online for that. I am going the explain current stage of computer hardware-software and why we need new language like Go? Because if there is no problems, then we don’t need solution, right?


Hardware limitations:

Moore’s law is failing.

First Pentium 4 processor with 3.0GHz clock speed was introduced back in 2004 by Intel. Today, my Mackbook Pro 2016 has the clock speed of 2.9GHz. So, nearly in one decade, there is no too much gain in the raw processing power. You can see the comparison of increasing the processing power with the time in below chart.

DennardScaling.png

From the above chart yo can see that the single-thread performance and the frequency of the processor remained steady for almost a decade. If you are thinking that adding more transistor is the solution, than you are wrong. This is because at smaller scale some quantum properties starts to emerge (like tunneling and because it actually costs more to put more transistors (why?) and the number of transistors you can add per dollar starts to fall.

So, for the solution of above problem,

  • Manufacturers started adding more and more cores to the processors. Now a days we have quad-core and octa-core CPUs available.
  • We also introduced hyper threading.
  • Added more cache to the processors to increase the performance.

But above solutions has it’s limitations too. We cannot add more and more cache to the processor to increase performance as cache have physical limits: the bigger the cache, the slower it gets. Adding more core to the processor has its cost too. Also, that cannot scale to indefinitely. These multi-core processors can run multiple threads simultaneously and that brings concurrency to the picture. We’ll discuss it later.

So, if we cannot rely on the hardware improvements, the only way to go is more efficient software to increase the performance. But sadly, modern programming language are not much efficient.

“Modern processors are a like nitro fueled funny cars, they excel at the quarter mile. Unfortunately, modern programming languages are like Monte Carlo, they are full of twists and turns.” — David Ungar


Go has goroutines !!

As we discussed above, hardware manufacturers are adding more and more cores to the processors to increase the performance. All the data centers running on those processors and we should expect an increase in the number of cores in upcoming years. More to that, today’s applications using multiple micro-services for maintaining database connections, message queues and maintain caches. So, the software we develop and the programming languages should support concurrency easily and they should be scalable with increased number of cores.

But, most of the modern programming languages(like Java, Python etc.) are from the ’90s single threaded environment. That doesn’t mean they do not support multi-threading. Most of those programming languages supports multi-threading. But the real problem comes with concurrent execution and threading-locking, race conditions and deadlocks. Those things makes it hard to create a multi-threading application on those languages.

For an example, Creating new threads in Java is not memory efficient. As every thread consumes approx 1MB of the memory heap size and eventually if you start spinning thousands of threads, it will put tremendous pressure on the heap and will cause shut down due to out of memory. Also, if you want to communicate between two or more threads, it’s very difficult.

On the other hand, Go was released in 2009 when multi-core processors were already available. That’s why GoLang is built with keeping concurrency in mind. Go has goroutines instead of threads. They consume almost 2KB memory from the heap and you can spin millions of goroutines at any time.

parallel-execution

Other benefits are :

  • Goroutines have growable segmented stacks. That means they will use more memory only when needed.
  • Goroutines have a faster startup time than threads.
  • Goroutines come with built-in primitives to communicate safely between themselves (channels).
  • Goroutines allow you to avoid having to resort to mutex locking when sharing data structures.
  • Also, goroutines and OS thread are not 1:1 mapping. A single goroutine can run on multiple threads. Goroutines are multiplexed onto a small number of OS threads.

You can see Rob Pike’s excellent talk concurrency is not parallelism to get more deep understanding on this.

All the above points, make Go very powerful to handle concurrency like Java, C and C++ while keeping concurrency code straight and beautiful like Earlang.

concurancy


Go runs directly on underlying hardware.

One most considerable benefit of using C, C++ is their performance over other modern higher level languages like Java/Python. Because C/C++ are compiled and not interpreted.

Processors understand binaries. Generally, when you build an application using Java or other JVM-based languages when you compile your project, it compiles the human readable code to byte-code which can be understood by JVM or other virtual machines that run on top of underlying OS. While execution, those VM interprets those bytecodes and convert them to binaries that can be understood by processors.

VM.png
Execution on VM based languages

While on the other side, C, C++ does not execute on VMs and that removes one step from the execution cycle and increases the performance. It directly compiles the human readable code to binaries.

non-vm
Execution on non-VM based languages

But, freeing and allocating variable in those languages is a huge pain. While most of the programming languages handle object allocation and removing using Garbage Collector or Reference Counting algorithms.

Go brings best of both the worlds. Unlike lower level languages like C/C++, Go is compiled language. That means performance is almost nearer to lower level languages. It also uses garbage collection to allocation and removal of the object. So, more malloc() and free() statements!!! Cool!!!


Code written in Go is easy to maintain.

Let me tell you one thing. Go does not have crazy programming syntax unlike other languages. It has very neat and clean syntax.

The designers of the Go at google had this thing in mind when creating the language. As google has the very large code base and thousands of developers were working on that same codebase, code should be simple to understand to other developers and one segment of code should have minimum side effects on another segment of the code. That will make code easily maintainable and easy to modify.

Go intentionally leaves out many features of many features of modern OOP languages.

  • No classes. Every thing is divided into packages only. Go has only structs instead of classes.
  • Does not support inheritance. That will make code easy to modify. In other languages like Java/Python, if the class ABC inherits class XYZ and you make some changes in class XYZ, then that may produce some side effects in other classes that inherit XYZ. By removing inheritance, Go makes it easy to understand the code also (as there is no super class to look at while looking at a piece of code).
  • No have constructors.
  • No annotations.
  • No generics.
  • No exceptions.

Above changes make Go very different from other languages and it makes programming in Go different than other. You may not like some of the points from above. But, it is not like you can not code without above features. All you have to do is write 2–3 more lines. But on the positive side, it will make your code cleaner and add more clarity to your code.

Code fun.PNG

Above graph displays that Go is almost as efficient as C/C++, while keeping the code syntax simple as Ruby, Python and other languages. That is a win-win situation for both humans and processors!!!

Unlike other new languages like Swift, it’s syntax of Go is very stable. It remained same since the initial public release 1.0, back in year 2012. That makes it backward compatible.


Go is backed by Google.

  • I know this is not any direct technical advantage. But, Go is designed and supported by Google. Google has one of the largest cloud infrastructures in the world and it is scaled massively. Go is designed by Google to solve their problems of supporting scalability and effectiveness. Those are the same issues you will face while creating your own servers.
  • More to that Go is also used by some big companies like Adobe, BBC, IBM, Intel and even Medium.(Source: https://github.com/golang/go/wiki/GoUsers)

Conclusion:

  • Even though Go is very different than other object-oriented languages, but it is still the same beast. Go provides you high performance like C/C++, super efficient concurrency handling like Java and fun to code like Python/Perl.
  • If you don’t have any plans to learn Go, I will still say hardware limit puts pressure to us, software developers to write super efficient code. The developer needs to understand the hardware and make the programs optimize accordingly. The optimized software can run on cheaper and slower hardware (like IOT devices) and overall batter impact on end user experience.

~If you liked the article, click the 💚 below so more people can see it! Also, you can follow me on by clicking follow button below, so you get updates regarding future articles on Go!!~

follow me on twitter.jpg