Data Model

Ditto is a NoSQL database, that can store JSON-like Documents organized by Collections. However, unlike JSON, Ditto allows you to apply updates to the document which will be synchronized with any other copy on other devices. In addition, it supports additional data types.

collections and documents

Collections

You can think of collections as a table in a traditional database. However, unlike traditional SQL databases, Ditto's collections are far simpler and more flexible. A collection merely referenced by its string value, there is no need to "create" a collection. While it is typically common for all documents in a collection to have the same structure, it is not a technical requirement. For example, all documents referencing cars can go in the "cars" collection and boat documents in the "boats" collection. You can create any number of collections that best represent your data model.

Collection names are case sensitive. Meaning that a collection with the name cars is different from Cars. We highly advise that you keep collection names casing consistent across your application.

To get a reference to a collection:

let carsCollection = ditto.store["cars"]
// or
let carsCollection = ditto.store.collection("cars")

DITCollection *carsCollection  = [ditto.store collection:@"cars"];

val carsCollection = ditto.store["cars"]
// or
val carsCollection = ditto.store.collection("cars")

DittoCollection carsCollection = ditto.store.collection("cars");

var carsCollection = ditto.Store.Collection("cars");

Documents

A DittoSyncKit document is a schema flexible unit of data in Ditto. If collections are similar to tables, the a document is similar to a row. A document, at its highest level, is a JSON map value that can contain arbitrarily nested keys and values. Each document has a primary key

Unique Identifier

In order for documents to sync, each document must have a unique identifier which we refer to as the id. This is the primary key of the document and is not optional.

You can supply your own unique identifier when creating a document:

let docId = ditto.store["people"].insert([
    "name": "Susan",
    "age": 31], withID: "123")
print(docId) // => "123"

NSString *docId = [[ditto.store collection:@"people"]
                    insert:@{ @"name": @"Susan", @"age": @31 }
                    isDefault:false
                    withID:@"123"
                    error:nil];
NSLog(@"%@", docId); // => "123"

val docId = ditto.store["people"].insert(mapOf(
    "name" to "Susan",
    "age" to 31
    ), "123")
docId // => "123"

Map<String, Object> content = new HashMap<>();
content.put("name", "Susan");
content.put("age", 31);
String docId = ditto.store.collection("people").insert(content, "123");
docId; // => "123"

// Insert JSON-compatible data into Ditto
var content = new Dictionary<string, object>
{
    { "name", "Susan" },
    { "age", 31 }
};
ditto.Store.Collection("people").Insert(content, "123");
docId; // => "123"

The id parameter is optional during insetion. If you do not supply a document id, Ditto will automatically generate a random unique identifier string in its place like so:

let docId = ditto.store["people"].insert([
    "name": "Susan",
    "age": 31
])
print(docId) // => "507f191e810c19729de860ea"

NSString *docId = [[ditto.store collection:@"people"]
                    insert:@{ @"name": @"Susan", @"age": @31 }
                    isDefault:false
                    error:nil];
NSLog(@"%@", docId); // => "507f191e810c19729de860ea"

val doc = mapOf(
    "name" to "Susan",
    "age" to 31
)
val docId = ditto.store["people"].insert(doc)
Log.d("Debug", docId) // => "507f191e810c19729de860ea"

Map<String, Object> content = new HashMap<>();
content.put("name", "Susan");
content.put("age", 31);
String docId = ditto.store.collection("people").insert(content);
Log.d("Debug", docId); // => "507f191e810c19729de860ea"

var content = new Dictionary<string, object>
{
    { "name", "Susan" },
    { "age", 31 }
};
var docId = ditto.Store.Collection("people").Insert(content);
Console.WriteLine($"{docId}"); // => "507f191e810c19729de860ea"

The document id field must be unique. Attempting to insert a document with the same value will throw an error.

Document Data

Like JSON, Ditto only supports strings as keys in documents. That means attempting to insert a document like the following will throw an error.

Supported Data Types and Values

Document values support all JSON compatible values like string, boolean, number, null and arrays or nested maps. In addition, document values can also support special types like binary or counter types. These special types will be discussed below.

Data Type	Allowed Values
Boolean	`false` or `true`
String	A utf-8 encodable string value
Number	A 64-bit floating point value.
Array	Arrays are an ordered list of values. Arrays can contain all primitive values as well as nested collection types like other Arrays or Maps
Maps (sometimes referred to as dictionary)	This represents a nested object within the overall document. Comparing values at the map level for equality first checks that each key and each value match.
`null`	This represents an absence of value
Binary	A byte string of binary data. Can be used to store images, files etc... We highly recommend keeping the size of the binary to be as small as possible so that syncing stays fast. We highly recommend that you use the `Attachment` type for most use cases.
Attachment	A file to sync. This is different from the `Binary` type where it is meant to hold larger payloads of data. This is best use for multimedia data like images, sounds, pdfs etc...
Counter	A special 64 bit floating point value that has the ability to be incremented and decremented. This is highly valuable for building applications like an inventory application where multiple devices need to concurrently increment or decrement values.

Here's an example of how to put all the different supported types together:

// Create a DittoSyncKit instance
let ditto = DittoSyncKit()

// Insert JSON-compatible data into Ditto
ditto.store["foo"].insert([
    "boolean": true,
    "string": "Hello World",
    "number": 10,
    "map": ["key": "value"],
    "array": [1,2,3],
    "null": nil
])

// Create a DittoSyncKit instance
DITDittoSyncKit *ditto = [[DITDittoSyncKit alloc] init];

// Insert JSON-compatible data into Ditto
[[ditto.store collection:@"foo"]
  insert:@{
      @"boolean": @true,
      @"string": @"Hello World",
      @"number": @10,
      @"map": @{ @"key": @"value" },
      @"array": @[ @1, @2, @3 ],
      @"null": [NSNull null]
  }
  isDefault:false
  error:nil];

ditto.store["foo".insert(mapOf(
    "boolean" to true,
    "string" to "Hello World",
    "number" to 10,
    "map" to mapOf("key" to "value"),
    "array" to listOf(1,2,3),
    "null" to null
))

// Insert JSON-compatible data into Ditto
Map<String, Object> content = new HashMap<>();
content.put("boolean", true);
content.put("string", "Hello World");
content.put("number", 10);
Map<String, String> innerMap = new HashMap<>();
innerMap.put("key", "value");
content.put("map", innerMap);
content.put("array", Arrays.asList(1, 2, 3));
content.put("null", null);
ditto.store.collection("docs").insert(content);

// Insert JSON-compatible data into Ditto
var fooDocument = new Dictionary<string, object>
{
    { "boolean", true },
    { "string", "Hello World" },
    { "number", 10 },
    { "map", new Dictionary<string, string>{{ "key", "value"}} },
    { "array", new int[] {1, 2, 3} },
    { "null", null }
};
ditto.Store.Collection("foo").Insert(fooDocument);

Binary

You can store any type of binary files as a byte array or byte string.

Note while DittoSyncKit supports binary data, we do not recommend storing very large blobs directly in a document. A good rule of thumb is 1MB or less. We have support for larger binary data with an optimized synchronization mechanism, which we refer to as attachments (see below).

let length = 2048
let bytes = [UInt32](repeating: 0, count: length).map { _ in arc4random() }
let data = Data(bytes: bytes, count: length)

ditto.store["people"].insert([
    "name": "Frank",
    "data": data
])

NSUInteger length = 2048;
uint8_t bytes[length];
SecRandomCopyBytes(kSecRandomDefault, length, bytes);
NSData *data = [[NSData alloc] initWithBytes:bytes length:length];

[[ditto.store collection:@"people"]
 insert:@{ @"model": @"Frank", @"data": data }
 isDefault:false
 error:nil];

val data = ByteArray(2048)
Random.nextBytes(data)

ditto.store["people"].insert(mapOf(
    "name" to "Frank",
    "data" to data
))

byte[] data = new byte[2048];
new Random().nextBytes(data);

Map<String, Object> content = new HashMap<>();
content.put("name", "Frank");
content.put("data", data);
ditto.store.collection("people").insert(content);

byte[] data = new byte[2048];
new Random().NextBytes(data);
var content = new Dictionary<string, object>
{
    { "name", "Frank" },
    { "data", data }
};
ditto.Store.Collection("people").Insert(content);

Attachment

If you have a large amount of binary data, or perhaps just a large file, that you want to sync between devices then instead of inserting this into a document as bytes then you should make use of the attachments feature.

Attachments do not get synced between devices by default, even if they are part of a document that is being synced between devices. This is because they could be very large files that a given device doesn't need. Instead an attachment must be explicitly fetched using an attachment token that will be present in the document that the attachment is linked to.

let collection = ditto.store["foo"]

let myImageURL = bundle.url(forResource: "image", withExtension: "png")!

let metadata = ["name": "my_image.png"]
let attachment = collection.newAttachment(
    path: myImageURL.path,
    metadata: metadata
)!

try! collection.insert(["some": "string", "my_attachment": attachment])

// Later, find the document and the fetch the attachment

let doc = collection.findByID(docID).exec()
let attachmentToken = doc!["my_attachment"].attachmentToken!

let fetcher = collection.fetchAttachment(token: attachmentToken) { status in
    switch status {
    case .completed(let fetchedAttachment):
        // Do something with attachment
        break
    default:
        print("Unable to fetch attachment")
        break
    }
}

NSURL *myImageURL = [bundle URLForResource:@"image" withExtension:@"png"];

NSDictionary<NSString *, NSString *> *metadata = @{@"name": @"my_image.png"};
DITAttachment *attachment = [collection newAttachment:myImageURL.path
                                             metadata:metadata];

[collection insert:@{@"some": @"string", @"my_attachment": attachment} error:nil];

// Later, find the document and the fetch the attachment

DITDocument *doc = [[collection findByID:docID] exec];
DITAttachmentToken *attachmentToken = doc[@"my_attachment"].attachmentToken;

DITAttachmentFetcher *fetcher = [collection fetchAttachment:attachmentToken
                                            onStatusChanged:^(DITAttachmentStatus *status) {
    switch (status.type) {
        case DITAttachmentStatusTypeCompleted: {
            DITAttachment *fetchedAttachment = completed.attachment;
            // Do something with attachment
            break;
        }
        default:
            NSLog(@"Unable to fetch attachment")
            break;
    }
}];

val attachmentStream =  context.assets.open("image.png")

val metadata = mapOf("name" to "my_image.png")
val attachment = coll.newAttachment(attachmentStream, metadata)

val docID =  coll.insert(mapOf("some" to "string", "my_attachment" to attachment))

// Later, find the document and the fetch the attachment

val doc = coll.findByID(docID).exec()
val attachmentToken = doc!!["my_attachment"].attachmentToken

val fetcher =  coll.fetchAttachment(attachmentToken!!) {
    when (it) {
        is Completed -> {
            let fetchedAttachment = it.attachment
            // Do something with attachment
        }
        else -> println("Unable to fetch attachment")
    }
}

InputStream attachmentStream = context.getAssets().open("image.png");

Map<String, String> metadata = new HashMap<>();
metadata.put("name", "my_image.png");
DittoAttachment attachment = coll.newAttachment(attachmentStream, metadata);

Map<String, Object> content = new HashMap<>();
content.put("some", "string");
content.put("my_attachment", attachment);
String docID = coll.insert(content);

// Later, find the document and the fetch the attachment

DittoDocument doc = coll.findByID(docID).exec();
DittoAttachmentToken attachmentToken = doc.get("my_attachment").getAttachmentToken();

class AttachmentStatusHandler implements DittoAttachmentStatusChangeHandler {

    @Override
    public void onStatusChanged(@NotNull DittoAttachmentStatus status) {
        if (status instanceof DittoAttachmentStatus.Completed) {
            DittoAttachment att = ((DittoAttachmentStatus.Completed) status).getAttachment();
            // Do something with attachment
        } else {
            System.out.println("Unable to fetch attachment");
        }
    }
}

AttachmentStatusHandler attachmentStatusHandler = new AttachmentStatusHandler();
DittoAttachmentFetcher fetcher = coll.fetchAttachment(attachmentToken, attachmentStatusHandler);

var coll = ditto.Store.Collection("people");
var path = "path/to/image.png";
var metadata = new Dictionary<string, string> { { "name", "my_image.png" } };
var attachment = coll.NewAttachment(path, metadata);
var docID = coll.Insert(new Dictionary<string, object> { { "some", "string" }, { "my_attachment", attachment } });
DittoDocument doc = coll.FindById(docID).Exec();
DittoAttachmentToken attachmentToken = doc["my_attachment"].AttachmentToken;
var fetcher = carsCollection.FetchAttachment(token: attachmentToken, (status) =>
{
    if (status is DittoAttachmentFetchEvent.Completed)
    {
        // Do something with attachment
    }
    else
    {
        Console.WriteLine($"Unable to fetch attachment");
    }
});

Counter

Counter is a very special type that is specific to Ditto. While they look like the number type, they are geared towards building applications where various different devices need to increment or decrement at the same time while preserving consistency. The most common use is to build a voting system or an inventory application. Building applications that needs a consistent count with only using the default number type will not be appropriate. This is where the counter comes in.

Counters can be edited through a special method called increment which can take number value to increment or decrement. To decrement, supply a negative value.

To create a counter, first insert a document with number value. You must then call an update function to convert the number into a counter with the replaceWithCounter method. This will convert the number into a counter.

Once the value in the document is a counter, you can proceed to increment or decrement the value. This will preserve the accurate value once devices sync

let docId = ditto.store["people"].insert([
    "name": "Frank",
    "ownedCars": 0 // here 0 is a number
])

ditto.store.["people"].findByID(docId).update({ mutableDoc in
    mutableDoc?["ownedCars"].replaceWithCounter()
    mutableDoc?["ownedCars"].increment(amount: 1)
})

NSString *docId = [[ditto.store collection:@"people"]
                    insert:@{ @"model": @"Frank", @"ownedCars": @0 }
                    isDefault:false
                    error:nil];

[[[ditto.store collection:@"people"] findByID:docId] update:^(DITMutableDocument * mutableDoc) {
    [mutableDoc[@"ownedCars"] replaceWithCounterAndReturnError:nil];
    [mutableDoc[@"ownedCars"] incrementWithAmount:1 error:nil];
}];

val docID = ditto.store["people"].insert(mapOf(
    "name" to "Frank",
    "ownedCars" to 0
))

ditto.store.collection("people").findByID(docID).update { mutableDoc ->
    mutableDoc["ownedCars"].replaceWithCounter()
    mutableDoc["ownedCars"].increment(1)
}

Map<String, Object> content = new HashMap<>();
content.put("name", "Frank");
content.put("ownedCars", 0);
String docID = ditto.store.collection("people").insert(content);

class DocumentUpdater implements DittoSingleMutableDocumentUpdater {
    @Override
    public void update(@NotNull DittoMutableDocument doc) {
        DittoMutableDocument<Map<String, Object>> document = (DittoMutableDocument<Map<String, Object>>) doc;
        document.get("ownedCars").replaceWithCounter();
        document.get("ownedCars").increment(1);
    }
}

ditto.store.collection("people").findByID(docID).update(new DocumentUpdater());

var content = new Dictionary<string, object>
{
    { "name", "Frank" },
    { "ownedCars", 0 }
};
ditto.Store.Collection("people").Insert(content);
ditto.Store.Collection("people").FindById(docId).Update((mutableDoc) => {
    mutableDoc["ownedCars"].ReplaceWithCounter();
    mutableDoc["ownedCars"].Increment(1);
});

Data Model
Collections
Documents
Unique Identifier
Document Data
Supported Data Types and Values
Binary
Attachment
Counter