All About Proto DataStore. in This Post, We Will Learn About Proto - by Simona Milanović - Android Developers - Medium

31/03/2024, 11:53 All about Proto DataStore.
aStore. In this post, we will learn about Proto… | by Simona Milanović | Android Develop…
Open in app Sign up Sign in
Search
All about Proto DataStore

Simona Milanović · Follow
Published in Android Developers
9 min read · Jan 31, 2022
Listen Share
In this post, we will learn about Proto DataStore, one of two DataStore
implementations. We will discuss how to create it, read and write data and how to
handle exceptions, to better understand the scenarios that make Proto a great
choice.
Proto DataStore uses typed objects backed by Protocol Buffers, to store smaller
datasets while providing type safety. It removes the need for using key-value pairs,
making it structurally different from its SharedPreferences predecessor and its
sibling implementation, Preferences DataStore. However, that’s not all — DataStore
brings many other improvements over SharedPreferences . Feel free to quickly jump
back to our first post in the series and take a look at the detailed comparison we’ve
made there. Going forward, we will refer to Proto DataStore as just Proto , unless
specified otherwise.
To sum up:
https://medium.com/androiddevelopers/all-about-proto-datastore-1b1af6cd2879 1/22
31/03/2024, 11:53 All about Proto DataStore. In this post, we will learn about Proto… | by Simona Milanović | Android Develop…
Provides a fully asynchronous API for retrieving and saving data, using the
power of Kotlin coroutines
Does not offer ready-to-use synchronous support — it directly avoids doing any
work that blocks the UI thread
Relies on Flow’s inner error signalling mechanism, allowing you to safely catch
and handle exceptions when reading or writing data
Handles data updates safely in an atomic read-modify-write operation,

providing strong ACID guarantees
Allows easy and simple data migrations
Need full type safety and your data requires working with more complex
classes, like enums or lists? This isn’t possible with Preferences, so choose Proto
instead
Intro to Protocol Buffers

To use Proto DataStore, you need to get familiar with Protocol Buffers — a language-
neutral, platform-neutral mechanism for serializing structured data. It is faster,
smaller, simpler and less ambiguous than XML and easier to read than other similar
data formats.
You define a schema of how you want your data to be structured and specify
options such as which language to use for code generation. The compiler then
generates classes according to your specifications. This allows you to easily write
and read the structured data to and from a variety of data streams, share between
different platforms, using a number of different languages, like Kotlin.
Example schema of some data in a .proto file:
1 // Copyright 2022 Google LLC.

2 // SPDX-License-Identifier: Apache-2.0
3
4 message Person {
5 required string name = 1;
6 required int32 id = 2;
7 optional string email = 3;
8 }
person.proto hosted with ❤ by GitHub view raw
How to use the generated Kotlin code for constructing your data model:
1 /* Copyright 2022 Google LLC.

2 SPDX-License-Identifier: Apache-2.0 */
3
4 val person: Person =
5 Person.newBuilder()
6 .setId(1234)
7 .setName("John Doe")
8 .setEmail("jdoe@example.com")
9 .build()
Person.kt hosted with ❤ by GitHub view raw
Or you can try out the newly announced Kotlin DSL support for protocol buffers for
a more idiomatic way of building your data model:

3
4 val person = person {
5 id = 1234
6 name = "John Doe"
7 email = "jdoe@example.com"
8 }
Person.kt hosted with ❤ by GitHub view raw
Investing a bit more time into learning this new serialization mechanism is
definitely worth it as it brings type safety, improved readability and overall code
simplicity.
Proto DataStore dependency setup

Now let’s look at some code and learn how Proto works.
We will use the Proto DataStore codelab sample. If you’re interested in a more
hands-on approach with implementation, we really encourage you to go through the
Working with Proto DataStore codelab on your own.
This sample app displays a list of tasks and the user can choose to filter them by
their completed status or sort by priority and deadline. We want to store their
selection — a boolean for displaying completed tasks and a sort order enum in
Proto.
We will firstly add Proto dependencies and some of the basic protobuf settings to
your module’s build.gradle . If you’re interested in a more advanced customisation
of the protobufs compilation, check out the Protobuf Plugin for Gradle notes:

3
4 plugins {
5 …
6 id "com.google.protobuf" version "0.8.17"
7 }
8
9 dependencies {
10 implementation "androidx.datastore:datastore:1.0.0"
11 // Starting from Protobuf 3.8.0, use the lite runtime library
12 implementation "com.google.protobuf:protobuf-javalite:3.18.0"
13 ...
14 }
15
16 protobuf {
17 // Configures the Protobuf compilation and the protoc executable
18 protoc {
19 // Downloads from the repositories
20 artifact = "com.google.protobuf:protoc:3.14.0"
21 }
22
23 // Generates the java Protobuf-lite code for the Protobufs in this project
24 generateProtoTasks {
25 all().each { task ->
26 task.builtins {
27 // Configures the task output type
28 java {
29 // Java Lite has smaller code size and is recommended for Android
30 option 'lite'
31 }
32 }
33 }
34 }
35 }
36
build.gradle hosted with ❤ by GitHub view raw
💡 Quick tip — if you want to minify your build, make sure to add an additional rule
to your proguard-rules.pro file to prevent your fields from being deleted:

3
4 -keepclassmembers class * extends androidx.datastore.preferences.protobuf.GeneratedMessa
5 <fields>;
6 }
proguard-rules.pro hosted with ❤ by GitHub view raw
Protobuf setup for Proto DataStore

Our journey with Proto starts by defining the structure of your persisted data in a
.proto file. Think of it as a readable schema for you and a blueprint for the
compiler. We will name ours user_prefs.proto and add it to the app/src/main/proto
directory.
Following the Protobuf language guide, in this file we will add a message for each
data structure we want to serialize, then specify a name and a type for each field in
the message. To help visualize this, let’s look at both a Kotlin data class and a
corresponding protobuf schema.
UserPreferences — Kotlin data class:

3
4 data class UserPreferences(
5 val showCompleted: Boolean,
6 val sortOrder: SortOrder
7 )
8
9 enum class SortOrder {
10 UNSPECIFIED,
11 NONE,
12 BY_DEADLINE,
13 BY_PRIORITY,
14 BY_DEADLINE_AND_PRIORITY
15 }
UserPreferences.kt hosted with ❤ by GitHub view raw
UserPreferences — .proto schema:
1 // Copyright 2022 Google LLC.

2 // SPDX-License-Identifier: Apache-2.0
3
4 syntax = "proto3";
5 option java_package = "com.codelab.android.datastore";
6 option java_multiple_files = true;
7
8 message UserPreferences {
9
10 bool show_completed = 1;
11 SortOrder sort_order = 2;
12
13 enum SortOrder {
14 UNSPECIFIED = 0;
15 NONE = 1;
16 BY_DEADLINE = 2;
17 BY_PRIORITY = 3;
18 BY_DEADLINE_AND_PRIORITY = 4;
19 }
20 }
user_prefs.proto hosted with ❤ by GitHub view raw
If you haven’t used protobufs before, you might also be curious about the first few
lines in the schema. Let’s break them down:
syntax — specifies that you’re using proto3 syntax
java_package — file option that specifies package declaration for your generated
classes, which helps prevent naming conflicts between different projects
java_multiple_files — file option that specifies whether only a single file with
nested subclasses will be generated for this .proto (when set to false) or if
separate files will be generated for each top-level message type (when set to
true); it is false by default
Next is our message definition. A message is an aggregate containing a set of typed

fields. Many standard simple data types are available as field types, including bool ,
int32 , float , double, and string . You can also add further structure to your
messages by using other message types as field types, like we did with SortOrder .
The = 1, = 2 markers on each element identify the unique “tag” that the field uses
in the binary encoding — like an ID of sort. Once your message type is in use, these
numbers should not be changed.
When you run the protocol buffer compiler on a .proto, the compiler generates the
code in your chosen language. In our specific case, when the compiler is run, this
leads to the generation of the UserPreferences class, found in your app’s
build/generated/source/proto … directory:

3
4 public final class UserPreferences extends GeneratedMessageLite<...>
5 implements UserPreferencesOrBuilder {
6 private UserPreferences() {}
7 …
8 }
UserPreferences.java hosted with ❤ by GitHub view raw
💡 Quick tip — You can also try out the newly announced Kotlin DSL support for
protocol buffers to use a more idiomatic way of building your data model.
Now that we have UserPreferences , we need to specify the guidelines for how Proto
should read and write them. We do this via the DataStore Serializer that
determines the final format of your data when stored and how to properly access it.
This requires overriding:
defaultValue — what to return if no data is found
writeTo — how to transform the memory representation of our data object into
a format fit for storage
readFrom — inverse from the above, how to transform from a storage format into
a corresponding memory representation

3
4 object UserPreferencesSerializer : Serializer<UserPreferences> {
5
6 override val defaultValue: UserPreferences = UserPreferences.getDefaultInstance()
7
8 override suspend fun readFrom(input: InputStream): UserPreferences {
9 try {
10 return UserPreferences.parseFrom(input)
11 } catch (exception: InvalidProtocolBufferException) {
12 throw CorruptionException("Cannot read proto.", exception)
13 }
14 }
15
16 override suspend fun writeTo(t: UserPreferences, output: OutputStream) =
17 t.writeTo(output)
18 }
UserPreferencesSerializer.kt hosted with ❤ by GitHub view raw
To keep your code as safe as possible, handle the CorruptionException to avoid

unpleasant surprises when a file cannot be de-serialized due to format corruption.
💡 Quick tip — If at any point your AS is unable to find anything UserPreferences
related, clean and rebuild your project to initiate the generation of the protobuf
classes.
Creating a Proto DataStore

You interact with Proto through an instance of DataStore<UserPreferences> .
DataStore is an interface that grants access to the persisted information, in our

case in the form of the generated UserPreferences .
To create this instance, it is recommended to use the delegate dataStore and pass
mandatory fileName and serializer arguments:

3
4 private val Context.userPreferencesStore: DataStore<UserPreferences> by dataStore(
5 fileName = DATA_STORE_FILE_NAME,
6 serializer = UserPreferencesSerializer,
7 …
8 )
TasksActivity.kt hosted with ❤ by GitHub view raw
fileName is used to create a File used to store the data. This is why the dataStore
delegate is a Kotlin extension property whose receiver type must be an instance of

Context , as this is needed for the File creation via applicationContext.filesDir .
Avoid using this file in any way outside of Proto, as it would break the consistency
of your data.
In the dataStore delegate, you can pass one more optional argument —
corruptionHandler . This handler is invoked if a CorruptionException is thrown by the
serializer when the data cannot be de-serialized. corruptionHandler would then
instruct Proto how to replace the corrupted data:

3
4 corruptionHandler = ReplaceFileCorruptionHandler(
5 produceNewData = { UserPreferences.getDefaultInstance() }
6 )
DataStore.kt hosted with ❤ by GitHub view raw
You shouldn’t create more than one instance of DataStore for a given file, as doing
so can break all DataStore functionality. Therefore, you can add the delegate
construction once at the top level of your Kotlin file and use it throughout your
application, in order to pass it as a singleton. In later posts, we will see how to do
this with dependency injection.
Reading data
To read the stored data, in UserPreferencesRepository we expose a
Flow<UserPreferences> from userPreferencesStore.data . This provides efficient
access to the latest saved state and emits with every change. This is one of the
biggest strengths of Proto — your Flow ’s values already come in the shape of the
generated UserPreferences . This means you don’t have to do any additional

transformations from the saved data into a Kotlin data class model, like you would
with SharedPreferences or Preferences DataStore :

3
4 val userPreferencesFlow: Flow<UserPreferences> = userPreferencesStore.data
UserPreferencesRepository.kt hosted with ❤ by GitHub view raw
The Flow will always either emit a value or throw an exception when attempting to
read from disk. We will look at exception handling in later sections. DataStore also
ensures that work is always performed on Dispatchers.IO so your UI thread isn’t
blocked.
🚨 Do not create any cache repositories to mirror the current state of your Proto
data. Doing so would invalidate DataStore’s guarantee of data consistency. If you
require a single snapshot of your data without subscribing to further Flow
emissions, prefer using userPreferencesStore.data.first() :

3
4 // Don't
5 suspend fun fetchCachedPrefs(scope: CoroutineScope): StateFlow<UserPreferences> =
6 userPreferencesStore.data.stateIn(scope)
7
8 // Do
9 suspend fun fetchInitialPreferences() = userPreferencesStore.data.first()
Writing data
For writing data, we will use a suspend
DataStore<UserPreferences>.updateData(transform: suspend (t: T) -> T) function.
Let’s break that down:
DataStore<UserPreferences> interface — we’re currently using

userPreferencesStore as the concrete Proto implementation
transform: suspend (t: T) -> T) — a suspend block used to apply the specified
changes to our persisted data of type T
Again, you might notice a difference to Preferences DataStore which relies on using
Preferences and MutablePreferences , similar to Map and MutableMap , as the default
data representation.
We can now use this to change our showCompleted boolean. Protocol buffers simplify
this as well, removing the need for any manual transformation from and to data
classes:

3
4 suspend fun updateShowCompleted(showCompleted: Boolean) {
5 userPreferencesStore.updateData { currentPreferences ->
6 currentPreferences.toBuilder().setShowCompleted(completed).build()
7 }
8 }
There’s a few steps to analyze:
toBuilder() — gets the Builder version of our currentPreferences which

“unlocks” it for changes
.setShowCompleted(completed) — sets the new value
.build() — finishes the update process by converting it back to UserPreferences
Updating data is done transactionally in an atomic read-modify-write operation.

This means that the specific order of data processing operations, during which the
data is locked for other threads, guarantees consistency and prevents race
conditions. Only after the transform and updateData coroutines complete
successfully, the data will be persisted durably to disk and
userPreferencesStore.data Flow will be reflecting the update.
🚨 Keep in mind that this is the only way of making changes to the DataStore state.
Keeping a UserPreferences reference and mutating it manually after transform
completes will not change the persisted data in Proto, so you shouldn’t attempt to
modify UserPreferences outside of the transform block.
If the writing operation fails for any reason, the transaction is aborted and an
exception is thrown.
Migrate from SharedPreferences

If you’ve previously used SharedPreferences in your app and would like to safely
transfer its data to Proto, you can use SharedPreferencesMigration . It requires a
context, SharedPreferences name and an instruction on how to transform your
SharedPreferences key-value pairs to UserPreferences within the migrate parameter.
Pass this via the produceMigrations parameter of the dataStore delegate to migrate
easily:

3
4 produceMigrations = { context ->
5 listOf(
6 SharedPreferencesMigration(
7 context,
8 USER_PREFERENCES_NAME
9 ) { sharedPrefs: SharedPreferencesView, currentData: UserPreferences ->
10 ...
11 currentData.toBuilder().setSortOrder(
12 SortOrder.valueOf(
13 sharedPrefs.getString(SORT_ORDER_KEY, SortOrder.NONE.name)
14 )
15 ).build()
16 ...
17 }
18 )
19 }
TasksActivity.kt hosted with ❤ by GitHub view raw
In this example, we go through the process of building the UserPreferences and

setting its sortOrder to what was previously stored in the corresponding
SharedPreferences key-value pair, or simply defaulting to NONE.
produceMigrations will ensure that the migrate() is run before any potential data
access to DataStore. This means your migration must have succeeded before
DataStore emits any further values and before it begins making any new changes to
the data. Once you’ve successfully migrated, it’s safe to stop using
SharedPreferences , as the keys are migrated only once and then removed from
SharedPreferences .
The produceMigrations accepts a list of DataMigration . We will see in later episodes

how we can use this for other types of data migrations. If you don’t need to migrate,
you can ignore this as it has a default listOf() provided already.
Exception handling
One of the main advantages of DataStore over SharedPreferences is its neat
mechanism for catching and handling exceptions. While SharedPreferences throws
parsing errors as runtime exceptions, leaving room for unexpected, uncaught
crashes, DataStore throws an IOException when an error occurs with
reading/writing data.
We can safely handle this by using the catch() Flow operator and emitting
getDefaultInstance() :

3
4 val userPreferencesFlow: Flow<UserPreferences> = userPreferencesStore.data
5 .catch { exception ->
6 if (exception is IOException) {
7 Log.e(TAG, "Error reading sort order preferences.", exception)
8 emit(UserPreferences.getDefaultInstance())
9 } else {
10 throw exception
11 }
12 }
Or with a simple try-catch block on writing:

3
4 suspend fun updateShowCompleted(completed: Boolean) {
5 try {
6 userPreferencesStore.updateData { currentPreferences ->
7 currentPreferences.toBuilder().setShowCompleted(completed).build()
8 }
9 } catch (e: IOException) {
10 // Handle error
11 }
12 }
If a different type of exception is thrown, prefer re-throwing it.
To be continued
We’ve covered Protocol Buffers and DataStore’s Proto implementation — when and
how to use it for reading and writing data, how to handle errors and how to migrate
from SharedPreferences . In the next and final post, we will go a step further and
look at how DataStore fits in your app’s architecture, how to inject it with Hilt and
of course, how to test it. See you soon!
You can find all posts from our Jetpack DataStore series here:
Introduction to Jetpack DataStore
All about Preferences DataStore
All about Proto DataStore
DataStore and dependency injection
DataStore and Kotlin serialization
DataStore and synchronous work
DataStore and data migration
DataStore and testing
Mad Skills Android Featured Latest
Follow
Written by Simona Milanović

1.3K Followers · Writer for Android Developers
Android Developer Relations Engineer @Google, working on Jetpack Compose
More from Simona Milanović and Android Developers
Simona Milanović in Android Developers
All about Preferences DataStore

In this post, we will take a look at Preferences DataStore, one of two DataStore
implementations. We will go over how to create it, read…
383 4
Ben Trengrove in Android Developers
Jetpack Compose: Strong Skipping Mode Explained

Strong skipping mode changes the rules for what composables can skip recomposition and
should greatly reduce recomposition.
13 min read · Feb 28, 2024
1K 7
Rebecca Franks in Android Developers
Fun with shapes in Compose

Create a progress bar that transitions from a squiggly “star” shaped rounded polygon to a
circle while performing the regular progress…
6 min read · Feb 27, 2024
951 3
Simona Milanović in Android Developers
DataStore and dependency injection

In the following posts from our Jetpack DataStore series, we will cover several additional
concepts to understand how DataStore interacts…
271 4
See all from Simona Milanović
See all from Android Developers
Recommended from Medium
Manu Aravind
Proto Datastore implementation in Android

Datastore is a replacement for SharedPreferences to overcome its shortcomings. Datastore is
an advanced data storage solution that was…
11 1
Mayur Waghmare in Mobile Innovation Network
SharedPreferences vs DataStore: Choosing the Right One

For years, developers have relied on SharedPreferences as a lightweight solution for storing
key-value pairs. However, with the evolution…
Lists
Staff Picks
609 stories · 872 saves
Michihiro Iwasaki
Persistent Data Storage Using DataStore (Preferences) in Jetpack

Compose
❚ ℹ: Sometimes, I may make mistakes or share information that isn’t completely accurate, as
I’m not perfect. I strive to minimize errors…
11 min read · Feb 27, 2024
66 1
Anna Karenina Jusuf
The Simplest Way to Use Preferences DataStore

As per August 2021, Android has release the new library called Jetpack DataStore, it is a new
and improved data storage solution aimed at…
3 min read · Nov 12, 2023
Jorge Luis Castro Medina
Modern Android Development in 2024

Set of good practices, guides, and tools to create modern Android apps in 2024.
18 min read · Feb 12, 2024
2.7K 14
Tippu Fisal Sheriff
Creating a Timer Screen with Kotlin and Jetpack Compose in Android

“A Step-by-Step Guide to Building a Feature-Rich Timer Screen with Jetpack Compose and
Kotlin”
3 min read · Nov 4, 2023
122
See more recommendations

All About Proto DataStore. in This Post, We Will Learn About Proto - by Simona Milanović - Android Developers - Medium

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

All About Proto DataStore. in This Post, We Will Learn About Proto - by Simona Milanović - Android Developers - Medium

Uploaded by

Copyright:

Available Formats

31/03/2024, 11:53 All about Proto DataStore.

Open in app Sign up Sign in

All about Proto DataStore

Handles data updates safely in an atomic read-modify-write operation,

Allows easy and simple data migrations

Intro to Protocol Buffers

Example schema of some data in a .proto file:

1 // Copyright 2022 Google LLC.

person.proto hosted with ❤ by GitHub view raw

1 /* Copyright 2022 Google LLC.

Person.kt hosted with ❤ by GitHub view raw

1 /* Copyright 2022 Google LLC.

Person.kt hosted with ❤ by GitHub view raw

Proto DataStore dependency setup

1 /* Copyright 2022 Google LLC.

build.gradle hosted with ❤ by GitHub view raw

1 /* Copyright 2022 Google LLC.

proguard-rules.pro hosted with ❤ by GitHub view raw

Protobuf setup for Proto DataStore

UserPreferences — Kotlin data class:

1 /* Copyright 2022 Google LLC.

UserPreferences.kt hosted with ❤ by GitHub view raw

UserPreferences — .proto schema:

1 // Copyright 2022 Google LLC.

user_prefs.proto hosted with ❤ by GitHub view raw

syntax — specifies that you’re using proto3 syntax

Next is our message definition. A message is an aggregate containing a set of typed

1 /* Copyright 2022 Google LLC.

UserPreferences.java hosted with ❤ by GitHub view raw

defaultValue — what to return if no data is found

1 /* Copyright 2022 Google LLC.

UserPreferencesSerializer.kt hosted with ❤ by GitHub view raw

To keep your code as safe as possible, handle the CorruptionException to avoid

💡 Quick tip — If at any point your AS is unable to find anything UserPreferences

Creating a Proto DataStore

DataStore is an interface that grants access to the persisted information, in our

1 /* Copyright 2022 Google LLC.

TasksActivity.kt hosted with ❤ by GitHub view raw

delegate is a Kotlin extension property whose receiver type must be an instance of

1 /* Copyright 2022 Google LLC.

DataStore.kt hosted with ❤ by GitHub view raw

generated UserPreferences . This means you don’t have to do any additional

1 /* Copyright 2022 Google LLC.

UserPreferencesRepository.kt hosted with ❤ by GitHub view raw

1 /* Copyright 2022 Google LLC.

UserPreferencesRepository.kt hosted with ❤ by GitHub view raw

Let’s break that down:

DataStore<UserPreferences> interface — we’re currently using

1 /* Copyright 2022 Google LLC.

UserPreferencesRepository.kt hosted with ❤ by GitHub view raw

There’s a few steps to analyze:

toBuilder() — gets the Builder version of our currentPreferences which

.setShowCompleted(completed) — sets the new value

.build() — finishes the update process by converting it back to UserPreferences

Updating data is done transactionally in an atomic read-modify-write operation.

Migrate from SharedPreferences

1 /* Copyright 2022 Google LLC.

TasksActivity.kt hosted with ❤ by GitHub view raw

In this example, we go through the process of building the UserPreferences and

The produceMigrations accepts a list of DataMigration . We will see in later episodes

1 /* Copyright 2022 Google LLC.

UserPreferencesRepository.kt hosted with ❤ by GitHub view raw

Or with a simple try-catch block on writing: