I made the same app 15 times, here are the results Part 2— Experiments & Results

Ali Taha Dinçer on 2024-01-23

Mastering the Mobile Dev Maze: UIKit vs. SwiftUI vs. XML vs. Compose — Part 2

The Previous Part

Welcome, dear reader! If you’re joining us again after exploring the initial segment of this series, a hearty welcome back. This installment is where things get really exciting. I’ll be delving into the nitty-gritty of various experiments and their outcomes, all centered around the 15 applications I’ve developed using KMP & CMP. There’s a mix of professional insights and candid commentary waiting for you. For those who might have missed the first part, fear not; you can catch up using the link provided below.

I made the same app 15 times, here are the results Part 1 — Introduction & Methodology TLDR: I have implemented the same coin app 15 times using UIKit, SwiftUI, XML, Compose and Compose Multiplatform by…medium.com

Also, throughout reading the series, you can follow up the repo below, where I published all the apps along with libraries and resources:

GitHub - Subfly/the_compose_experiment: A repository that holds a continuous experimentation with… A repository that holds a continuous experimentation with Compose Multiplatform until it reaches stable - GitHub …github.com

Now, let’s dive right in!

In the Methodology section of the preceding article, I mentioned our dual testing approach: automation via Maestro and the more user-centric method of fast scrolling. These were applied to Android apps, while for iOS, due to specific reasons explained earlier, we stuck with just the fast scroll test. We conducted tests on physical devices, namely a Galaxy S22 for Android apps and an iPhone 15 Pro Max Emulator for iOS. Remember, all these apps were tested in their release mode. Each test is clearly labeled in the detailed charts I’ve prepared for you. In the Android test names, you’ll find tags like “automated” (indicating Maestro tests) and “user scroll” for the manual method.

Before we jump into the results, a quick note on the SwiftUI implementations: they all use LazyVStack within a ScrollView. Despite hearing that ListView might be more efficient, albeit more restrictive, this was our chosen setup, something to keep in mind while reviewing the SwiftUI results.

One last technical detail: the FPS and Memory Usage tests were conducted separately. First, I ran the FPS test, then deleted the app, tweaked the code to switch from FPS to Memory Usage measurement, reinstalled the app, and finally conducted the Memory Usage test. A little heads-up for you to consider as you peruse the findings.

Alright, that’s enough preamble. Let’s start with the libraries☺️

Part 1: Libraries

Part 1.1: KMPKtorCoinbase

As already mentioned in the previous part of this article series, this library holds the business logic of all the KMP apps I have implemented. For this part, I only give you the size of the library, in order to make explaining the incoming parts easier for you. I have calculated the size of the library directly looking at the sizes of files built by running Gradle tasks of maven:publishToLocal and podPublishXCFramework (yes, that library uses Cocoapods, I maybe wrote a seperate article about that).

• KMPKtorCoinbaseSharedAndroidReleaseSourcesJar: 20.9 KB
• KMPKtorCoinbaseSharedKotlinReleaseSourcesJar: 21.74 KB
• KMPKtorCoinbaseFramework(Cocoa-Device): 11.9 MB
• KMPKtorCoinbaseFramework(Cocoa-Simulator): 23.25 MB

Same as the above part, but that library utilizes XCFramework and built by using assembleXCFramework Gradle task. Additionally, again, I have to mention that this library has all the code in KMPKtorCoinbase library, and I will give further information in Acknowledgements part of that article.

• CMPCoinbaseSharedAndroidReleaseSourcesJar: 30.98 KB
• CMPCoinbaseSharedKotlinReleaseSourcesJar: 34.39 KB
• CMPCoinbaseFramework(XCF-Device): 139.37 MB
• CMPCoinbaseFramework(XCF-Simulator): 228.63 MB

However, from the details shared earlier, it’s clear that incorporating KMP and CMP into your app is likely to bump up its size. This increase can be attributed to the nature of Compose due to code generation and the generated Objective-C headers. So, as we move forward, keep this in mind — it’s a crucial piece of the puzzle in understanding the overall impact of these libraries on app sizes.

Part 2: Android

Part 2.1: BaseAndroidXMLKtor

This app is implemented without any KMP or CMP dependency and utilizes XML for UI. The networking in the app utilizes Ktor. The app size is 17.44 MB.

At first glance, you might be wondering, “What the heck is that?”. How can XML be so non-performant? And why are there FPS drops? The simple answer lies in the performance of Maestro on scrolls. It’s so slow that the Choreographer enters to the standby mode, stabilizing itself at 20 FPS. This adjustment is made to reduce GPU usage and preserve battery life. However, this isn’t immediately obvious from the graph, is it? Let’s take a closer look at the results of the user scroll test:

Voilà! A perfect 120 FPS, matching the Galaxy S22’s 120 Hz screen. Now, observe how the Choreographer activates standby mode after 30 seconds without scrolls. The initial drop visible at the 3rd second occurs because the data loads at the 2nd second, and I began scrolling at the 4th second. Thus, this graph represents excellent performance in terms of FPS. You can compare it with the first graph above, noting the points where it hits 120 FPS during the scroll in Maestro.

Since this is a Maestro test graph, it is unreliable in my opinion to say something. At least, GC points are pretty visible and memory usage averages at 200 MB’s.

Compared to Maestro results, user scroll test spikes much faster and GC entry points are pretty visible. The test ended in almost 23 seconds (I was like Sonic at that moment probably). From what I understand, the increase and stabilization between 250–300 MB’s of memory usage is due to pooling of RecyclerView. But needs more investigation to prove it. Also, I can not explain that huge spike in 3rd second, but guessing that it is the time when data is loaded and mapped to UI Models, after that probably GC worked and collected the DTO’s, as it reduced the memory usage half the size.

Part 2.2: BaseAndroidComposeKtor

This app is implemented without any KMP or CMP dependency and utilizes Jetpack Compose for UI. The networking in the app utilizes Ktor. The app size is 23.62 MB, due to generated code during building Composables.

We observe a graph quite similar to the XML one. However, there are noticeable direct increases and decreases — let’s call them oscillations — between seconds. At first glance, it may be hard to understand, but it’s important to first comprehend how LazyColumn works in Compose. Similar to RecyclerView in XML, LazyColumn attempts to recycle its items, but with a different mechanism. Imagine you have x number of items visible on your screen, with the first visible item being the y-th element. LazyColumn in Compose keeps both the y-1-th and the x+1-th elements in reserve. This means that both the previous and next items are available during scrolling, leading to a smoother experience for the user.

In the case of Maestro, where scrolling is considerably slow, the x+1-th element attempts to load itself, resulting in good FPS. In the following second, the Choreographer tries to stabilize at lower FPS rates. However, the now-visible x+1-th element prompts the rendering of the x+2-th element, elevating the FPS once more. Since it involves just one element, it renders quickly and then returns to standby mode, resulting in the oscillating pattern observed in the graph. This is particularly evident between the 9th and 12th seconds.

Voila! Again! A much better and easier chart to understand. However, one can see the small oscillations on the graph during scroll test. Unfortunately, this is the curse of Compose due to it’s calculations before rendering. Even though, we still have a good performance between 110–120 FPS throughout the test, but slightly worse then XML 🙁

Above chart is not that much informative without looking at the chart of user scroll results in my opinion. At least, one can see the GC entry points. Let’s directly jump to user scroll test.

Compared to the Maestro chart, this one offers much more information. We see a wider version of the Maestro chart above, where the garbage collection (GC) entry points are more visible. This is due to the fast scrolling of Composables, along with the loading of data and the cleaning of unused DTOs at the 4th second. It’s apparent that the memory stabilizes at almost 240 MB.

Part 2.3: KMPAndroidXMLKtor

This app is implemented with KMP dependency in order to get the business logic needed and utilizes XML for UI. The networking in the app utilizes Ktor. The app size is 17.42 MB, which is 200 KB’s less than the Base version, because of the removal of Android’s Lifecycle ViewModel. “KMPKtorCoinbase” has it’s own handling mechanism of lifecycle, which requires a manual cleanup after the Activity for Android or UIViewController of the iOS is finished.

Compared to the Base version, we see a pretty similar chart above for Maestro test.

Perfection, same as Base version. But I have to mention that I have started testing on 9th second, the coffee was good and I was sipping it, sorry ☺️

Compared to the Base version, memory usage is much more stable in KMP app, but there is no differences between business logic parts that affects that type of difference. Additionally, one can see that app stabilizes between 150–175 MB’s of memory usage, which is better than the Base version.

This version is very similar to the Base version, but with a slight improvement — there’s a reduction of about 50 MB in memory usage. Interestingly, there are no significant differences in the business logic, except for the removal of Android’s Lifecycle ViewModel. Yet, the KMP versions of the Android apps performed better.

Additionally, it’s worth noting that the use of Kotlin flow in the “KMPKtorCoinbase” library is different. However, the logic applied to make Kotlin flow multi-platform is essentially just a wrapper around the typical StateFlow and MutableStateFlow. This shouldn’t have any significant effect, especially on memory usage on the Android side.

Anyway, let’s move on to the next app and see how they compare.

Part 2.4: KMPAndroidComposeKtor

This app is implemented with KMP dependency in order to get the business logic needed and utilizes Jetpack Compose for UI. The networking in the app utilizes Ktor. The app size is 23.6 MB, due to removal of Android’s Lifecycle ViewModel, but pretty close to the Base version. Even though it is not enough information to conclude that, we can say that output of generated Composables are pretty same.

Yes, results are pretty similar to Base version, but slightly differs, and in fact a bit less performant. But this can be due to the Maestro. Let’s see user scroll test to understand the issue.

Yep, as I said, the result is pretty similar to Base version. The oscillations are expected because of the Jetpack Compose’s nature. I believe that the issue relies on Maestro, anyway.

Like XML version of the KMP app, memory is much stable, but as I said, I changed nothing in the code except the removal of Android’s Lifecycle ViewModel. However, this requires a further investigation.

Still the same, but closer to the Base version. Memory stabilizes at 250 MB. Oscillations constantly occur due to GC.

Part 2.5: KMPAndroidCMPKtor

This app is implemented with CMP dependency in order to get the business logic needed and utilizes Compose Multiplatform for UI. The networking in the app utilizes Ktor. The app size is 25.36 MB, which is almost 1.5 MB’s more than the “BaseAndroidComposeKtor”. The reason is, not only the library introduces smaller composables in order to be both used in Android and iOS side easily, but also has same sources twice, one in XML and one in directly Kotlin code as an ImageVector. I will give further information about that in the Acknowledgements section of this article.

Pretty similar to the previous automated Maestro tests, however, I have to admit that I have started testing in the 10th second. On the other hand, charts are pretty same.

Well, a pretty standard chart for the Compose, like the previous ones. Oscillations are still visible due to Compose’s nature.

Again, a typical automated Maestro memory usage chart. Memory is still stable at 150–175 MB’s like the previous ones. Let’s jump to the user scroll test and finalize the outcomes.

Finally, we arrive at a graph that closely resembles the memory usage results from the previous user scroll test, at least for the Compose implementations. In my opinion, this provides sufficient information to conclude that, most of the time, whether it’s KMP, CMP, or fully native, the results for Android are quite similar.

Part 3: iOS

Part 3.1: BaseiOSUIKit

This app is implemented without any KMP or CMP dependency and utilizes UIKit for UI. The networking in the app utilizes URLSession. The app size is 631 KB.

Simply perfect. Nothing much to say, good ol’ UIKit is a performance monster.

With the powerful recycling mechanism of TableView, we have a very stable and minimal usage of memory, around 40–50 MB’s.

Part 3.2: BaseiOSSwiftUI

This app is implemented without any KMP or CMP dependency and utilizes SwiftUI for UI. The networking in the app utilizes URLSession. The app size is 950 KB.

As already mentioned in the article, I have used LazyVStack inside of a ScrollView for all SwiftUI implementations. This chart would be much better if I used a ListView. However, as you can see, after almost a few hundreds of elements scrolled, performance of the SwiftUI decreases with that type of implementation, and when there is nothing to draw (since there are no scrolls), the view stabilizes itself.

After seeing that chart, one can simply understand that, each item in the LazyVStack that inside of a ScrollView is preserved in the memory, and this is the reason of the FPS drop in the previous chart.

Part 3.3: KMPiOSUIKit

This app is implemented with KMP dependency in order to get the business logic needed and utilizes UIKit for UI. The networking in the app utilizes Ktor. The app size is 7.2 MB, which means adding KMP dependency to the iOS app immensely increased the ipa size (at least 11 times).

Compared to the Base version, there’s a noticeable drop in FPS performance, despite this being a UIKit implementation. Initially, I thought the data might not be loading quickly enough from the KMP side to the UI. However, my approach involved loading the entire coin dataset from the Coinpaprika endpoint and transferring it to the UI via a Kotlin flow. Since all the data is available during the UI’s drawing phase, the issue likely lies elsewhere. Finally, my hypothesis is that Kotlin’s Data Classes may not be as performant as Swift’s Structs, but this requires further investigation to be concluded. Perhaps that’s a topic for another article, eh?

Comparing the memory usage chart with FPS chart, it is still very hard to understand the exact reason of the FPS drops, since there are no patterns that actually give any information. The only thing that we can infer from that memory usage chart is, there is a huge probability that KMP’s Kotlin/Native GC is trying to collect unused DTO’s in around 10th second and doing another collection in around 20th second. After that, considering the UI Model’s are still kept in KMP side, in memory wise, performance is similar to the Base version, even though it stabilizes between 110–120 MB’s. This can be said, as I already mentioned, UI Model’s are kept in KMP side, which increases the memory usage from 40–50 MB’s in Base version, but it is pretty stable because of the TableView’s recycling mechanism.

Part 3.4: KMPiOSSwiftUI

This app is implemented with KMP dependency in order to get the business logic needed and utilizes SwiftUI for UI. The networking in the app utilizes Ktor. The app size is 7.2 MB, which means adding KMP dependency to the iOS app immensely increased the ipa size (at least 7 times).

The graph is pretty similar to the Base version, and the same issue of using LazyVStack inside of a ScrollView can be seen easily. Nothing much to say, but since it is pretty similar to Base version, one can infer that SwiftUI did a better job with KMP compared to the UIKit version, however, since the performance issue of SwiftUI is available due to implementation reasons, it will be wrong to say that and requires investigation by changing the UI implementation using ListView.

Compared to the Base version, one can see that the chart is much more stable, however this can be pretty fallacious. Comparing the first seconds of this chart and the “KMPiOSUIKit” version, we can see that the starting values of memory usages are pretty close, and throughout the test, memory usage increases to up to 350 MB’s, which is pretty close to the Base version of SwiftUI. Considering the memory usages in the GC entry points, it can be said that Apple’s GC and Kotlin/Native GC worked well together to balance the SwiftUI’s over-use of memory, even the implementation was not the best. However, this requires a further investigation.

Part 3.5: KMPiOSCMPUIKit

This app is implemented with CMP dependency in order to both get the business logic needed and Compose Multiplatform for UI. In this case, whole list is rendered using CMP and navigation is handled via navigation controller. The networking in the app utilizes Ktor. The app size is 29.4 MB, which means adding CMP dependency to the iOS app immensely increased the ipa size (at least 46 times).

The chart above is quite revealing, clearly indicating that the app experiences occasional freezes during fast scrolling through the list. On the flip side, there are minor oscillations in terms of FPS performance, which can be attributed to the pre-calculations that Compose performs before drawing. This is quite similar to what we observed in “BaseAndroidComposeKtor”. Disregarding the freezes, the performance is relatively stable, hovering around 55–60 FPS. However, these freezing issues are a significant drawback for the app’s overall performance.

Compared to the Base version, we see a much worse chart. Usage of memory is drastically increased, and final stabilization occur on 375–400 MB’s. On the other hand, Kotlin/Native GC entry points are directly visible, where reduces the memory usage by almost 200 MB’s each time.

Part 3.6: KMPiOSCMPSwiftUI

This app is implemented with CMP dependency in order to both get the business logic needed and Compose Multiplatform for UI. In this case, whole list is rendered using CMP and navigation is handled via NavigationStack and NavigationLink. The networking in the app utilizes Ktor. The app size is 29.4 MB, which means adding CMP dependency to the iOS app immensely increased the ipa size (at least 30 times).

Compared to the previous UIKit version (KMPiOSCMPUIKit), we can still see immediate drops in the FPS, which is the result of the same issue that the UIKit version had. However, it can be seen that SwiftUI handled the FPS drops due to freezes much better than the UIKit version. I can say that, during testing, the freezes took less time compared to the UIKit version. Like the UIKit version, small oscillations on the FPS can be visible, which is due to aforementioned issues, similar to the “BaseAndroidComposeKtor” version.

Pretty similar to the UIKit version above. Entry points are clearly visible, and both the Apple’s GC and Kotlin/Native’s GC (mostly Kotlin/Native GC) tried their best to stabilize the memory usage, which is around 300 MB’s. However, this is still worse than the “BaseiOSUIKit” version.

Part 3.7: KMPiOSCMPUIKitListItem

This app is implemented with CMP dependency in order to both get the business logic needed and Compose Multiplatform for UI. In this case, all list elements are rendered using CMP inside of a TableView and navigation is handled via navigation controller. The networking in the app utilizes Ktor. The app size is 29.3 MB, which means adding CMP dependency to the iOS app immensely increased the ipa size (at least 46 times). Even though the TableView code is preserved during building the app, I can not explain why there is a 100KB decrease. This requires further investigation.

Up to this point, you’ve seen fancy charts following the introductory paragraph in each 3.* subsection of this article. However, this part doesn’t include any charts, and let me explain why.

During the testing of this app, it consistently crashed after the first 7 to 10 scrolls. At first glance, one might think it’s probably an Out of Memory (OOM) issue. However, the actual problem turned out to be quite different. XCode didn’t always provide a clear cause for the crash. But when I managed to capture it at a critical moment, instead of just a bunch of memory locations, it mentioned something like an IllegalStateException and Metal. Notably, when the crash occurred, I could clearly see empty black spaces on the screen where the CoinTile should have been. This led me to realize that loading CoinTile Composables was a challenging process for the system.

If you open the directory composeBase/src/iosMain/kotlin/dev/subfly/cmpcoinbase/view/ in the “CmpCoinbase” library, and look at any file, you’ll see that any composable that can be called on the iOS side has been wrapped with ComposeUIViewController. This means that every composable has a lifecycle tied to a custom UIViewController. Essentially, every CoinTile used in my TableView as a TableViewCell is basically a UIViewController. For the Android developers reading this article, it means to having every CoinTile in a RecyclerView being an Activity. With this insight, I believe that the CMP side is not quick enough to initialize these UIViewControllers within a frame. As a result, iOS skips this frame (possibly several of them), and the Kotlin/Native GC clears the pointers to these undrawn views, which Metal can’t access and draw. However, this hypothesis requires further investigation and might be a significant issue for CMP. Hence, instead of charts, you’re getting this lengthy explanation ☺️.

Part 3.8: KMPiOSCMPSwiftUIListItem

This app is implemented with CMP dependency in order to both get the business logic needed and Compose Multiplatform for UI. In this case, all list elements are rendered using CMP inside of a LazyVStack inside of a ScrollView and navigation is handled via NavigationStack and NavigationLink. The networking in the app utilizes Ktor. The app size is 29.4 MB, which means adding CMP dependency to the iOS app immensely increased the ipa size (at least 46 times).

Unlike the UIKit version mentioned earlier (KMPiOSCMPUIKitListItem), I was actually able to run and test this app. However, as you can see from the chart, the app was quite laggy and experienced periodic freezing. Compared to the Base version, it was somewhat less laggy towards the end of the test. Still, the persistent lag and freezes made the app practically unusable. Considering the same explanation for the previous app also applies here, it appears that SwiftUI handled the UIViewController tiles better than UIKit. This observation also suggests that rendering a large number of UIViewControllers from the CMP side to the iOS side is a challenging task.

At first sight it seems like a pretty stable chart. The only drastic clean-up happens in 5th second and other GC points are not that much powerful, although each ComposeUIViewController added to the stack, and the implementation includes LazyVStack inside of a ScrollView. However, the stability of the chart requires further investigation due to the aforementioned reasons.

Part 3.9: CMPUIKit

This app is implemented with CMP dependency in order to both get the business logic needed and Compose Multiplatform for UI. In this case, both screens are rendered using CMP and navigation is handled via navigation controller. The networking in the app utilizes Ktor. The app size is 29.4 MB, which means adding CMP dependency to the iOS app immensely increased the ipa size (at least 46 times).

Similar to the “KMPiOSCMPUIKit” version, we see some drastic drops in FPS, which is the results of freezes during scroll. However, a full screen CMP implementation did a better job compared to using CMP part by part, as freezes took less time to melt down, and user experience were not that bad. In fact, the app was pretty usable. Still, one can see that there are oscillations during fast scroll, which means the same issue still exists, which is the pre-calculations that done before drawing any composable, which is the fate of Compose.

Chart is pretty similar to “KMPiOSCMPUIKit” version. GC entry points are pretty visible. Most of the time, only the Kotlin/Native’s GC worked, and tried to stabilize between 275–300 MB’s, which is close to the most Android versions. Even though it seems a good thing, compared to the “BaseiOSUIKit” version, the performance is pretty worse.

Part 3.10: CMPSwiftUI

This app is implemented with CMP dependency in order to both get the business logic needed and Compose Multiplatform for UI. In this case, both screens are rendered using CMP and navigation is handled via NavigationStack and NavigationLink. The networking in the app utilizes Ktor. The app size is 29.4 MB, which means adding CMP dependency to the iOS app immensely increased the ipa size (at least 46 times).

Compared to previous UIKit version above (CMPUIKit), performance is pretty same in terms of FPS, excluding what happened in 28th second. I remember that, during testing, app froze for 2 seconds and was unusable in that time interval. Other than that, performance was pretty good and app was usable.

The chart here is almost identical to the previous UIKit version, leading to similar conclusions. Additionally, we can infer that CMP’s performance is consistent regardless of the base used (UIKit or SwiftUI), particularly in terms of memory usage. This is an expected outcome, as the primary difference is in the navigation part, while both screens and the entire business logic are rendered directly in Compose Multiplatform.

The Next Part

When I initially embarked on writing this article, my intention was to compile everything into a single page. However, as the content grew, I found myself with over 9000 words, equivalent to nearly 35 minutes of reading time on Medium. In the interest of simplicity and to enhance readability, I’ve opted to split this article into three parts. The next installment will delve into comparison between the predictions in the previous part with the outcomes on this part, acknowledgements gathered throughout this experiment and finalizing with a conclusion part. You can explore these details in the upcoming section below:

I made the same app 15 times, here are the results Part 3— Outcomes & Acknowledgements Welcome back, readers! If you’re joining us again for the final installment of this series, it’s great to have you on…medium.com

I want to extend my gratitude to you for accompanying me on this journey. Feel free to share and utilize any part of this article series and project, as long as proper credits are attributed. I value your feedback and welcome any questions you may have. You can reach out to me at:

• My LinkedIn: Ali Taha Dinçer
• My E-Mail: alitahasubfly@gmail.com