Slack ships its client application on many different platforms — we currently support Mac, Windows, Linux, iOS, Android, and Windows Phone. Initially these different platform applications were developed independently at Slack, with separate teams and codebases for each of the mobile platforms and the desktop. About a year ago, Slack formed a new team, LibSlack, to develop a cross-platform C++ library to encapsulate common functionality and integrate it into client apps. LibSlack is currently in use in the Slack iOS and Android apps, with a plan to expand to other platforms.
Sharing common code between different platforms has many benefits but also presents challenges. There have been some bumps in the road and changes of direction as we developed the early versions of LibSlack, and as we tried to figure out ways to make it most useful to the client applications. This post covers the problems we are trying to solve, some challenges we have faced, and the tools and techniques we have used to solve them.
Why LibSlack?
Slack’s initial approach of developing each platform client independently made sense at the time — it allowed platform engineers to quickly develop in the native languages for each platform, without any dependencies on other projects or platforms. But it also meant that any time a new feature was added it had to be implemented multiple times, once for each platform. In addition, the same bug could arise on different platforms and have to be fixed multiple times, and there was inconsistent behavior of the same feature between the platforms — for example, lists of users had a different sort order in different clients. Moving shared code into a platform independent library can greatly reduce development effort and improve consistency and quality across platforms.
The LibSlack Sandwich
Which parts of the application code should be shared across platforms? Where is native code necessary, and where can common platform-independent code be used? We still want Slack to have a great native look and feel, so the UI layer should remain platform code and take advantage of platform APIs. In addition, we want a base platform layer to take advantage of native functionality such as networking support, background tasks, and efficient threading. For example, LibSlack’s threading library is implemented using GCD on iOS and thread pools on Android, allowing us to better integrate with platform thread management. The code best suited to move into the cross-platform library is the business logic and data handling. These areas can be managed in platform-independent code, and keeping this complicated logic in a separate library enables us to have consistent behavior across platforms. Thus, our plan is to have a delicious LibSlack sandwich — a platform UI layer on top, LibSlack logic in the middle, and a platform layer on the bottom to access native capabilities.
Cache Flow
Data caching is one of the main focuses of LibSlack. Initially, Slack clients assumed they could download a complete picture of all team information when the client was launched. This worked when Slack teams were small and most users connected through reliable desktop network connections, but as the teams using Slack grew in size, and more people wanted to use their Slack clients in conditions with less reliable network connections, it became untenable to download all information about the team when the app launched.
To handle large teams, Slack clients are switching to lazy loading as much information as possible, getting only the data that is necessary for the current view, and saving a partial cache of information. This improves startup times and behavior on poor networks, but it leads to more complex logic to determine what data will be needed and when. Unreliable connections make cached data even more valuable, since the cache allows the app to retain functionality when it is offline. This means Slack clients now need to determine the most important information that the user will need to use the app, and fetch it proactively. As the logic for what data is fetched and cached becomes more complicated, LibSlack will play an important role by taking over this responsibility for the clients.
LibSlack aims to unify the logic to determine what information needs to be fetched and saved, when cached information is out of date, what data should be evicted from the cache, and who needs to be notified when updated information is received. In the future, LibSlack will power improvements to the offline behavior of Slack clients by proactively fetching data, maintaining a queue of offline actions, and executing them when the network becomes available. Our goal is to make Slack clients function as well in spotty network conditions as they do in reliable ones, and ensure that the user’s actions are never lost because their network dropped.
Language Choice
One of the first decisions we faced in creating a cross-platform library to use with all Slack clients was which programming language to build it with. After much discussion and experimentation, we chose modern C++ as the core language for LibSlack development. C++ has support across compilers and toolchains for all major platforms. In addition, modern C++ has added performance enhancing features like move semantics, lambdas, return value optimization, and smart pointers to aid in memory management. There is an ecosystem of libraries like Boost that make C++ more powerful and easier to use. We’re currently using C++ 11 due to the need to support the compiler used for our Windows Mobile client, but we intend to update to C++ 14 shortly, and keep updating to make use of the most recent language features.
LibSlack APIs
In order to incorporate LibSlack into Slack client applications, we need to generate headers for the LibSlack APIs in the languages used on each platform. To create those headers we currently use Djinni, an open source project from Dropbox. Djinni takes an interface description file and creates C++, Java and Objective-C headers based on that description file. C++ types like unordered_set<T> are translated into NSSet in Objective-C and HashSet in Java. Djinni creates interfaces that are implemented in LibSlack and called by the clients. It also creates interfaces that are implemented in the clients to provide native functionality needed by LibSlack, like threading and networking. Once Djinni has generated the headers, either the C++ or client-side code implements the interfaces.
Aside from creating the headers, Djinni also handles memory management across the boundary between C++ and other languages, and handles string and other data translation to and from C++. But there are some drawbacks as well — marshalling calls across the boundary between languages can have performance costs, and Djinni does impose some restrictions on API design (like no support for lamdas or blocks). Also, Djinni does not currently support Javascript, which will be needed to integrate Libslack with the desktop client. To generate those headers we may need to extend Djinni or write our own header generation scripts.
Development & Debugging
Creating a cross-platform C++ library for native apps can lead to development and debugging challenges, such as difficulty debugging when combining C++ with native code, and issues of integration and syncing releases of the library to releases of the clients.
For LibSlack, we addressed these issues in a few ways. We initially built a reference client application for easier development, using React Native for the UI and Objective-C or Java to connect to LibSlack. This simple app exercised basic LibSlack functionality and allowed us to do quick prototyping and testing of features in development. Eventually we improved our integration with the client applications to the point that we were able to drop the use of this app and debug features directly in the Slack clients.
Our initial approach to syncing releases with the Slack client applications involved shipping a binary of LibSlack each week, which clients then integrated. But we found that this system created a lot of work for Libslack engineers to validate releases each week, and often required hot-fixes to update the binary and address bugs found after clients integrated it. We have since moved to a continuous integration system where LibSlack code is updated daily, so we can catch and address issues earlier. The mobile platforms also now build LibSlack from source. This enables client and LibSlack engineers to easily set breakpoints and step into the LibSlack code to debug issues, which was not possible with binaries. To ensure that LibSlack does not increase client build times, we are using ccache, a cross-platform compiler cache that stores previous compilation output per file and allows it to be used again in future builds.
To debug customer issues, we have also added logging and analytics to LibSlack. The combination of logging, performance metrics, and stack traces from Fabric/Crashlytics help us to track down issues in the wild.
Unit and Integration Testing
One advantage of LibSlack is the ability to provide a highly tested codebase that can be more reliable than separate platform implementations. Because LibSlack does not contain any UI code, it is highly testable. While we are not practicing strict TDD, we do require unit tests when checking in new functionality and we currently have around 70% code coverage with unit tests. These unit tests are run on 5 platforms for every PR that is opened, and PR’s are gated so they cannot be merged until all unit tests have passed.
In addition, we have added integration tests to exercise the APIs LibSlack provides to clients, to make sure our tests are representative of how the code is actually used by our Slack client applications. Once again, we are gating LibSlack PR’s on the passing of these integration tests on all of our client platforms.
Finally, we are working towards having automated performance testing of LibSlack on-device, to catch any performance regressions before they get out in the wild.
exit()
There will always be a need for native development in the client Slack apps. Not only for UI and native capabilities, but prototyping can often be done more quickly and easily in the clients directly. However, by creating a highly reliable, scalable, cross-platform library to share as much common code as possible, we are able to increase efficiency of development, increase quality, and unify Slack client behavior to make it more dependable for customers on all platforms.
If you are a developer with experience creating cross-platform applications, come join us!
