In the previous post we discussed the features that we would like to have in a modern day MVC framework. In this post let us discuss some of the key problems in designing such a framework and how we have approached it in TAME.
Data is stored in storage devices as bits. However humans don’t quite understand this, so have come up with mechanisms to store and retrieve data from a storage device using a high level interface. This high level interface is the notion of files and on top of files, we have built the notion of databases.
A database makes this storage and retrieval more user friendly rather than working with raw files. Databases are stores of “entities”. Entities may be related to other entities. Interaction with the database involves performing CRUD operations on the entities.
Entities contain fields. Fields have data types. Data types can be broadly categorized as “primitive data types” and “composite data types“. When considering relational databases, we normalize data into primitive data types. Composite data types are handled using primary and foreign key relationships across multiple tables. However, in a NoSQL store, we may store composite data as a sub-record within a single record itself and save an additional join.
So we start our discussion by asking, what are the minimum set of primitive and composite data types we need? If we do a comparative analysis of different programming languages and database technologies, we will soon realize that almost all languages have some way of representing strings, numbers and booleans. There are many other primitive types, but these are the minimum. The reasoning behind why this is so, is beyond the scope of this post.
Composites are ways of grouping data together. Composites are made up of primitives and other composites. At a minimum, we need one composite data type that has sequential properties (list like) and at least one that can be used to store non-sequential (key/value) form of data (map like).