TODAQ/ADOT Data Model
(3 min read)
A major goal of Engineering this quarter is to enhance how TODA can be used as an application protocol, and not merely a low-level guarantee of scarcity or identity.
TODA already provides us some data structure primitives. (These primitives form an acyclic graph of assets if updates to assets are viewed as new structures...) The structures offered to us at this level include: a pointer list, a list of pairs of pointers, a trie of pointers, or arbitrary binary data.
At the application level, ADOT, we’re building structure atop these primitives to be able to display the data associated with an asset in a meaningful way. For example, while TODA defines the storage of arbitrary data, it’s helpful for us to indicate at the application level whether that data is UTF-8 encoded text (perhaps with an ISO language code), or whether it’s one of the many TODAkitty animated gifs floating around the office.
To this end, the application level will view assets as a hypergraph structure, where property-value pairs are stored as pointer pairs. The property pointer will point to an asset representing the definition of a Field, while the value pointer points to an asset, or datagram, representing the value. This opens up a very rich system of UI hinting and behavioural and data inheritance down a type hierarchy.
Further, each amendment to a TODA asset can contain this rich Field-Value structure, raising the exciting prospect of how a given asset could be composed to give a coherent “current view” - (or what some of the mathies around here insist on referring to as a “projection”).
A simple composition strategy many developers might be familiar with is a “diff”. In Unix, there’s a well-defined standard for how differences in a text file should be summed up to get the final state. There have been efforts to extend this idea to structured data, such as JSON, resulting in proposals like JSONdiff. While we have a hypergraph, which is slightly more exciting than JSON, we could still certainly define an asset type to use this sort of update composition.
But, we are not restricted to this single mode of operation. One advantage of our append-only style of assets is that data modellers may elect to use one of several such composition strategies when designing how to model their application. In fact, we expect that individual Fields will be able to indicate which of several built-in modes of composition apply to them.
A simple example is a “read-only” field. Perhaps a Customer asset has a field called CustomerID. While users could continue to append updates to the Customer, updating their address or phone number, it might be essential that the CustomerID should never change. In this case then, while we cannot ban the ability of someone to add a new value to the CustomerID field, we can define the projection of that field to always reflect the first - and only the first - value that was set for that field.
There are motivating use cases for several other modes including merge, max, min, and perhaps even entirely calculated fields - but those will need to wait for another post.
If it sounds that this “interpretation” of data inside of TODA assets is all a bit loosey-goosey, that’s correct. The ground truth of what data has actually been recorded in a TODA asset has no room for any interpretive differences: it’s locked in with crypto - but enabling a layer of flexibility for an application to view that data in different ways (just like a database view, or even a PivotTable), and providing a bit of a standard to facilitate that process, will make building applications on these core primitives a much more natural and intuitive process for our developers.
We can’t wait to share this with you!
Adam
If you enjoyed this content and would like to see more, please subscribe to the TODAQ Press here:
Thank you!