About Druid - A Real-time Analytical Data Store

简介:
Druid是一个为流式数据设计的实时分析数据存储系统,包含4个组件,采用了冷热分离的结构:
Real-time Nodes
Historical Nodes
Broker Nodes
Coordinator Nodes
About Druid - A Real-time Analytical Data Store - 德哥@Digoal - PostgreSQL research

infoq里有一篇介绍druid的文章。

Druid is similiar to C-Store [38] and LazyBase [8] in that it has
two subsystems, a read-optimized subsystem in the historical nodes
and a write-optimized subsystem in real-time nodes. Real-time nodes
are designed to ingest a high volume of append heavy data, and do
not support data updates. Unlike the two aforementioned systems,
Druid is meant for OLAP transactions and not OLTP transactions.
Druid’s low latency data ingestion features share some similarities
with Trident/Storm [27] and Spark Streaming [45], however,
both systems are focused on stream processing whereas Druid is
focused on ingestion and aggregation. Stream processors are great
complements to Druid as a means of pre-processing the data before
the data enters Druid.
There are a class of systems that specialize in queries on top of
cluster computing frameworks. Shark [13] is such a system for
queries on top of Spark, and Cloudera’s Impala [9] is another system
focused on optimizing query performance on top of HDFS. Druid
historical nodes download data locally and only work with native
Druid indexes. We believe this setup allows for faster query latencies.
Druid leverages a unique combination of algorithms in its architecture.
Although we believe no other data store has the same set
of functionality as Druid, some of Druid’s optimization techniques
such as using inverted indices to perform fast filters are also used in
other data stores [26].

[参考]
目录
相关文章
|
5月前
|
Oracle 关系型数据库 数据库
Active Data Guard Real-Time Cascade
12c 的 Cascaded Standby 数据库
51 7
|
分布式计算 Apache Spark
《How to Integrate Spark MLlib and Apache Solr to Build Real-Time Entity Type Recognition System for Better Query Understanding》电子版地址
How to Integrate Spark MLlib and Apache Solr to Build Real-Time Entity Type Recognition System for Better Query Understanding
89 0
《How to Integrate Spark MLlib and Apache Solr to Build Real-Time Entity Type Recognition System for Better Query Understanding》电子版地址
《SPEED MATTERSHOW TO PROCESS BIG DATA SECURELY FOR REAL-TIME APPLICATIONS》电子版地址
SPEED MATTERS:HOW TO PROCESS BIG DATA SECURELY FOR REAL-TIME APPLICATIONS
93 0
《SPEED MATTERSHOW TO PROCESS BIG DATA SECURELY FOR REAL-TIME APPLICATIONS》电子版地址
《Performance Characterization of In-Memory Data Analytics on a Scale-up Server》电子版地址
Performance Characterization of In-Memory Data Analytics on a Scale-up Server
79 0
《Performance Characterization of In-Memory Data Analytics on a Scale-up Server》电子版地址
《Fighting Cybercrime A Joint Task Force of Real-Time Data and Human Analytics》电子版地址
Fighting Cybercrime: A Joint Task Force of Real-Time Data and Human Analytics
86 0
《Fighting Cybercrime A Joint Task Force of Real-Time Data and Human Analytics》电子版地址
|
Apache 流计算
《Large-scale near-real-time (NRT) data analytics platform empowered by Apache Flink - Ying Xu & Kailash Hassan Dayanand》电子版地址
3. Large-scale near-real-time (NRT) data analytics platform empowered by Apache Flink - Ying Xu & Kailash Hassan Dayanand, Lyft的副本
104 0
《Large-scale near-real-time (NRT) data analytics platform empowered by Apache Flink - Ying Xu & Kailash Hassan Dayanand》电子版地址
Sap Ds Data is not available. Increase the time-out interval values in Debug | Options
Sap Ds Data is not available. Increase the time-out interval values in Debug | Options
137 0
PAT (Advanced Level) Practice - 1145 Hashing - Average Search Time(25 分)
PAT (Advanced Level) Practice - 1145 Hashing - Average Search Time(25 分)
120 0
|
Shell
History displays the time information
For those of you who use terminals a lot, one of the most common commands is probably history, which allows you to view the history of terminal commands executed
117 0
|
SQL 存储 算法
The MemSQL Query Optimizer: A modern optimizer for real-time analytics in a distributed database
今天我们要介绍的MemSQL就采用这样一种新的形态(Oracle也变为了这种方式 ):即在做transformation时,要基于cost确定其是否可应用。 当然,本篇paper不止讲解了CBQT,还包括一些MemSQL优化器其他方面的介绍,包括一个有意思的heurstic based bushy join的方案。
407 0
The MemSQL Query Optimizer: A modern optimizer for real-time analytics in a distributed database