CeresDB was originally created to act as a semi-structured database system for the Whitetail logger. The original prototype was written in Python and was then vastly improved and ported to Go in preparation to be used as the backing database for Velocimodel. This database supports the following datatypes:
STRINGINTFLOATBOOlLIST (not searchable by filters or able to be ordered)DICT (not searchable by filters or able to be ordered)ANY (not searchable by filters or able to be ordered)CeresDB stores data on disk with the ID of each record corresponding to its location. For example, if the data is stored like so in the database foo and collection bar:
foo
|-- bar
|-- 968b38d3-42a1-4b8b-a44a-bb34b1d218ec
|-- 43811f87-bb22-4084-a6c8-d0bb98c1265a
|-- 4a95fd00-39e4-4318-9260-b4c13a365b7f
We have \(n\) number of records in each UUID-named file (up to some maximum \(N\) which is defined in the configuration). Therefore, if I want to query for the 7th record that exists in 968b38d3-42a1-4b8b-a44a-bb34b1d218ec I would query for the ID 968b38d3-42a1-4b8b-a44a-bb34b1d218ec.7.
In addition, indices are stored on disk with the following structure (this example assumes that the collection has the fields hello and world which are both integers containing values between 0 and 9):
foo
|-- bar
|-- hello
|-- 0
|-- 1
|-- 2
|-- 3
|-- 4
|-- 5
|-- 6
|-- 7
|-- 8
|-- 9
|-- world
|-- 0
|-- 1
|-- 2
|-- 3
|-- 4
|-- 5
|-- 6
|-- 7
|-- 8
|-- 9
each file within the hello directory contains a list of IDs which correspond to those records which contain the value equalling the filename in the hello field. The same pattern applies to the world directory. This allows for quick selection of records based on value in that we can select those records which match hello = 2 by reading in the contents of foo/bar/hello/2.