Building something as audacious as Macrometa’s geo-distributed fast data cloud would not have been possible without the extraordinary contributions of many people. We are deeply indebted to the following people for the extraordinary body of research work and publications that have enabled us to build Macrometa.
Christopher Meiklejohn (Consulting Scientist to Macrometa)
Christopher’s research into state and operational Conflict Free Replicated Data Types (CRDTs see next section) & implementations such as LASP, Partisan and work on AntidoteDB, enabled us to bring the best ideas from CRDTs to our vision for a geo distributed database. We lucked out when Chris agreed to help us with formally verifying our approach and provided lots of valuable input on our operational CRDT and garbage collection strategies. In addition, Chris is just one of the most humble and great people we have had the good fortune of collaborating with.
Conflict Free Replicated Data Types (CRDTs)
Marc Shapiro’s research at INRIA, France lead to the creation of CRDTs, Strong Eventual Consistency, and its applications in wide area and disconnected applications. His papers and videos helped us several years back with our initial investigation of replicated data types.
Carlos Baquero’s “Pure Operation-Based Replicated Data Types” paper enabled us to think about the right approach to multi master replication and causal broadcast.
Martin Klepmann’s ground breaking book “Designing Data-Intensive Applications” opened up many exciting approaches and techniques for us to consider when building our geo-distributed cloud service at scale. We continues to source inspiration from his work, presentations and papers on streams and consistency.
Leslie Lamport invented Latex, but even more importantly gave the world a way to think about time keeping that results in correctness, performance and reliability in geo distributed systems. His paper “Time, Clocks, and the Ordering of Events in a Distributed System” helped us create our logical clock and causal ordering implementations.
We leverage following open sources in our geo-distributed platform.
Macrometa uses ArangoDB’s query engine and AQL syntax as the foundation for its database query functionality. ArangoDB is an apache licensed NoSQL database that supports multi-model polyglot persistence.
Macrometa uses Apache Bookkeeper & ZooKeeper for a scalable, fault-tolerant, and low-latency storage service optimized for real-time workloads
Macrometa uses Benthos for internal event processing and connectors.