Abstract
BG is a benchmark that rates a data store for processing interactive social networking actions such as view a member profile and extend a friend invitation to a member. It elevates the amount of stale, inconsistent, and erroneous (termed unpredictable) data produced by a data store to a first class metric, quantifying it as a part of the benchmarking phase. It summarizes the performance of a data store in one metric, Social Action Rating (SOAR). SoAR is defined as the highest throughput provided by a data store while satisfying a pre-specified service level agreement, SLA.
To rate the fastest data stores, BG scales both vertically and horizontally, generating a higher number of requests per second as a function of additional CPU cores and nodes. This is realized using a shared-nothing architecture in combination with two multi-node execution paradigms named Integrated DataBase (IDB) and Disjoint DataBase ((DB)-B-2). An evaluation of these paradigms shows the following tradeoffs. While the (DB)-B-2 scales superlinearly as a function of nodes, it may not evaluate data stores that employ client-side caching objectively. IDB provides two alternative execution paradigms, Retain and Delegate, that might be more appropriate. However, they fail to scale as effectively as (DB)-B-2. We show elements of these two paradigms can be combined to realize a hybrid framework that scales almost as effectively as (DB)-B-2 while exercising the capabilities of certain classes of data stores as objectively as IDB. (C) 2018 Elsevier B.V. All rights reserved.