博客
关于我
强烈建议你试试无所不能的chatGPT,快点击我
存储系统的分类
阅读量:6301 次
发布时间:2019-06-22

本文共 8043 字,大约阅读时间需要 26 分钟。

之前收集了一些存储产品,最近又重新整理了一下,对他们进行了简单的分类。每个对存储的分类可能不仅相同,我的分类完全按照自己的喜好来分,如和您的分类不同,仅供参考。只是做了搜集和分类,少量产品加了写介绍,希望以后有时间,加更多更详细的介绍。

[TOC]

1.存储引擎

1.1 Hash Table

1.1.1 dbm (database manager)

The dbm library stores arbitrary data by use of a single key (a primary key) in fixed-size buckets and uses hashing techniques to enable fast retrieval of the data by key.

1.2 btree

1.2.1 berkerlydb

Key/value数据模型

Berkeley DB最初开发的目的是以新的HASH访问算法来代替旧的hsearch函数和大量的dbm实现(如AT&T的dbm,Berkeley的 ndbm,GNU项目的gdbm)

oracle

Written in C, java

BTREE, HASH, QUEUE, RECNO storage

1.2.2 LMDB (Lightning Memory-Mapped Database)

LMDB is a Btree-based database management library modeled loosely on the BerkeleyDB API, but much simplified. The entire database is exposed in a memory map, and all data fetches return data directly from the mapped memory, so no malloc's or memcpy's occur during data fetches.

1.2.3 BoltDB

Bolt is a pure Go key/value store inspired by Howard Chu's LMDB project.

1.3 LSM

1.3.1 LevelDB

C++, google

1.3.2 RocksDB

C++/java

1.3.3GoLevelDB

1.3.4gorocksdb

Go wrapper for RocksDB

1.3.5 levigo

Go wrapper for LevelDB

1.3.6 mongo-rocks

RocksDB Storage Engine Module for MongoDB

C++

1.4 LSH

1.4.1 bitcask

采用bitcask模型的有:beandb, Riak

erlang

日志结构的key/value存储系统Bitcas

Bitcask存储模型

1.5 FractalTree

1.5.1 PerconaFT

Mysql存储引擎之TokuDB以及它的数据结构Fractal tree(分形树)

TokuDB的索引结构–分形树的实现

1.6 dbm系列

1.6.1 QDBM (Quick DataBase Manager)

used by nmdb

1.6.2 ndbm (New Database Manager)

ndbm (standing for New Database Manager) is a Berkeley produced version from 1986 of the AT&T dbm database.

ndbm stores arbitrary data by use of a single key in fixed-size buckets.

1.6.3 SDBM (Substitute Database Manager)

Substitute Database Manager

1.6.4 GDBM (GNU Database Manager)

GNU Database Manager

1.6.5 tdb (Trivial Database)

Trivial Database

1.6.6 CDB

1.6.7 TinyCDB

Tiny Constant Database

1.7 双类型

1.7.1 Wiredtiger (btree, LSM)

C语言的

WiredTiger 本身也支持 LSM option (默认是 btree )

mangodb

1.7.2 Tokyo Cabinet and Kyoto Cabinet (B+tree,hash table)

Tokyo Cabinet and Kyoto Cabinet are two libraries of routines for managing key-value databases.

Kyoto Cabinet is the designated successor of Tokyo Cabinet

Tokyo Cabinet 是一个DBM的实现

Tokyo Cabinet features on-disk B+ trees and hash tables for key-value storage

Used by nmdb,Kyoto TreeDB

1.7.3 RaptorDB key value store (B+ 树 或者 MurMur 哈希索引)

一个很小的、快速的嵌入式 NoSQL 存储模块,使用 B+ 树 或者 MurMur 哈希索引

Implemented in .NET

1.8 SQL引擎类

1.8.1 InnoDB

1.9 document类

1.9.1 RaptorDB document store

2 嵌入式

2.1 SQLite

2.2 UnQLite

盘点移动开发中最流行的5个数据库

3 单机存储

3.1 单值KV存储

3.1.1 Memcache

3.1.2 nmdb

use qdbm, berkeley db, tokyo cabinet or tdb as database backends

C语言

3.1.3 Memcachedb

C语言,新浪

a distributed key-value storage system designed for persistent

It conforms to memcache protocol
uses Berkeley DB as a storing backend

write 18868 w/s

read 44444 r/s

using replication for master/slave

6 policy for replication:

3.1.4 Kyoto Tycoon

C/C++,FAL Labs

Kyoto Tycoon is a lightweight database server with auto expiration mechanism, which is useful to handle cache data and persistent data of various applications. Kyoto Tycoon is also a package of network interface to the DBM called Kyoto Cabinet.

3.1.5 ThruDB

建立在Apache Thrift framework下的简单服务

支持多个数据存储后端,包括BerkeleyDB、Disk、MySQL,还拥有Memcache和Spread集成

Thrudb is a set of simple services built on top of the Apache Thrift framework that provides indexing and document storage services

3.2 结构化KV存储

3.2.1 Redis

3.2.2 ssdb

采用ssd,使用leveldb作为存储引擎,兼容redis接口

C/C++

3.2.3 ssdb-rocks

ssdb的另一个版本,采用ssd,使用rocksdb作为存储引擎,兼容redis接口

3.3.4 ardb

redis-protocol compatible persistent nosql, it support multiple storage engines as backend like Google's LevelDB, Facebook's RocksDB, OpenLDAP's LMDB, WiredTiger, the default backend is Facebook's RocksDB.

C++

3.2.5 (reborndb)QDB

兼容redis协议

Rocksdb and LevelDB

3.2.6 Pika

Pika 的存储引擎, 基于Rocksdb 修改. 封装Hash, List, Set, Zset等数据结构

首发丨360开源的类Redis存储系统:Pika

首发丨360开源的类Redis存储系统:Pika

3.2.7 LedisDB

A high performance NoSQL like Redis powered by Go

LevelDB, goleveldb, LMDB, RocksDB, BoltDB or Memory

3.3 文档型

3.3.1 SisoDB

C#编写的,专门提供给SQL Server面向文档的db-provider

3.4 SQL

3.4.1 MySQL

3.4.2 innostore

Innostore is a simple Erlang API to Embedded InnoDB

4 单机存储的proxy集群方案

4.1 KV/Redis类

4.1.1 Twenproxy

静态的分布式Redis方案

4.1.2 Reborndb

4.1.3 Codis

4.1.4 Netflix Dynomite

基于dynamo的思想

Dynomite: NetFlix对dynamo的开源通用实现

Netflix open sources Dynomite to make any datastore distributed

A generic dynamo implementation for different k-v storage engines

inspired by Dynamo whitepaper

4.1.5 dbcached

a distributed key-value memory caching system for QDBM or Berkeley DB base on Memcached and NMDB

4.2 SQL类

4.2.1 Mycat

MyCat:开源分布式数据库中间件

4.2.2 MySQL Fabric

4.2.3 TDDL

4.2.4 Cobar

4.2.5 Atlas

4.2.6 Heisenberg

4.2.7 Vitess

5 KV存储

5.1 riak

采用bitcask模型

Riak的实现是基于Amazon的Dynamo论文

erlang

Riak是以 Erlang 编写的一个高度可扩展的分布式数据存储,Riak的实现是基于Amazon的Dynamo论文,Riak的设计目标之一就是高可用。

5.2 beandb

采用bitcask模型

distributed key-value storage system

took the ideas from Amazon's Dynamo

5.3 Project Voldemort

Voldemort is a distributed key-value storage system

BDB-JE, MySQL, Read-Only

LinkedIn

5.4 Scalaris

分布式key value

erlang

supported the ACID properties for multi-key transactions

5.5 Aeospike

www.aerospike.com

5.6 Tair

c/c++, 自研的mdb,fdb

5.7 dynomite

6 文档型存储

6.1 MongoDB

6.2 CouchDB

CouchDB是文档型存储

6.3 Membase / Couchbase

面向文档的 NoSQL 数据库管理系统

6.4 SequoiaDB

一个类mongodb的文档型存储

6.5 RavenDB

a .net document database built on the Windows ESENT storage system

支持Linq,可以使用C#的Linq语法查询数据

6.6 OrientDB

虽然是文档型数据库,但是它的关系管理方式却和图形数据库相类似

Distributed Graph Database with the flexibility of Documents

是兼具文挡数据库的灵活性和图形数据库管理链接 能力的可深层次扩展的文档-图形数据库管理系统。可选无模式、全模式或混合模式下。支持许 多高级特性,诸如ACID事务、快速索引,原生和SQL查询功能。可以JSON格式导入、导出文档。若不执行昂贵的JOIN操作的话,如同关系数据库可在 几毫秒内可检索数以百记的链接文档图

Even if it is a document-based database, the relationships are managed as in graph databases with direct connections between records.

It supports schema-less, schema-full and schema-mixed modes.

6.7 RethinkDB

RethinkDB 1.14 (Brazil) 发布,分布式数据库

7 列式存储

7.1 HBase

7.2 Cassandra

7.3 Accumulo

Apache Accumulo is based on Google's BigTable design and is built on top of Apache Hadoop, Zookeeper, and Thrift.

7.4 Hypertable

an open source database system inspired by publications on the design of Google's BigTable.
Hypertable runs on top of a distributed file system such as the Apache HDFS, GlusterFS or the Kosmos File System (KFS).

7.5 Scylla

8 NewSQL

8.1 Actordb

8.2 Cockroachdb

Go语言

Design Documents

8.3 FoundationDB

8.4 Oceanbase

OceanBase架构介绍

8.5 SnappyData

8.6 TiDB

Go 语言

Rust语言

从零开始写分布式数据库

9 图数据库

9.1 Neo4j

9.2 Infinite Graph

an enterprise distributed graph database

10 File存储

10.1 Ceph

Ceph:一个 Linux PB 级分布式文件系统

10.2 FastDFS

10.3 HDFS

10.4 MogileFs

10.5 MooseFS

a fault tolerant, network distributed file system

10.6 TFS

10.7 GlusterFS

GlusterFS:异地备份(Geo-replication)源码分析

Gluster Geo-replication工作原理[转载]

10.8 kosmosfs

11 In-Memory 存储

11.1 Redis cluster

11.2 Mysql cluster

11.3 Gemfire/Gemde

11.4 VoltDB

内存数据库

12 私有存储

12.1 Amazon

12.1.1 Amazon Dynamo

Key-value

Amazon’s Dynamo paper

Merkle trees

Gossiping of membership

Gossiped synchronization of partitions

12.2 Google

12.2.1 BigTable

Bigtable: A Distributed Storage System for Structured Data

12.2.2 F1

F1: A Distributed SQL Database That Scales

12.2.3 Spanner

Spanner: Google’s Globally-Distributed Database

Exclusive: Inside Google Spanner, the Largest Single Database on Earth

解析全球级分布式数据库Google Spanner

12.2.4 Megastore

Megastore: Providing Scalable, Highly Available Storage for Interactive Services

12.3 Baidu

12.3.1 Baidu Mola

Key-value

12.3.2 Baidu BDRP

12.3.3 Baidu DDBS

12.4 腾讯

单机MySQL到NoSQL集群 腾讯存储进阶路

腾讯十多个人管理一万多台NoSQL存储服务器的秘密

12.4.1 腾讯CKV

腾讯CKV海量分布式存储系统

12.4.2 QuorumKV

微信PaxosStore内存云揭秘:十亿Paxos/分钟的挑战

12.6 京东

12.6.1 京东JIMDB

解读JIMDB 京东分布式缓存与高速KV存储

解读JIMDB 京东分布式缓存与高速KV存储

12.7 滴滴

12.7.1 滴滴Rockstable

滴滴高性能KV存储系统实践

12.8 美团

12.8.1 Cellar

基于tair研发的新一代KV存储服务

美团cellar讲座笔记

12.9 360

12.9.1 360 Bada

Key-value

360自研分布式存储系统Bada的架构设计和应用

1.9.2 HustStore

HustStore 360高性能分布式存储服务

13 云产品

13.1 Amazon DynamoDB

13.2 AWS Aurora

2.3 Google云

2.4 阿里云

2.5 美团云

Mangix:分布式对象存储

Mangix: 美团云分布式对象存储系统

你可能感兴趣的文章
vue (v-if show 问题)
查看>>
转://Oracle 高可用技术与云基础架构
查看>>
[转载] 七龙珠第一部——第080话 悟空对抗天龙
查看>>
Spring 并发事务的探究
查看>>
HDOJ_ACM_Big Event in HDU
查看>>
HTML提供的6种空格
查看>>
【总结整理】微信产品-张小龙-PM学习总结
查看>>
Neutron local network 学习
查看>>
git提交代码到码云详解
查看>>
189. Rotate Array
查看>>
209. Minimum Size Subarray Sum
查看>>
VS配置FFmpeg开发环境
查看>>
MFC窗口客户区全屏显示以及PICTURE控件全屏显示
查看>>
js日期选择控件
查看>>
[POI2011]Lightning Conductor
查看>>
6.caption标签,为表格添加标题和摘要
查看>>
7.系统常量
查看>>
jQuer基础
查看>>
java开发中遇到的问题及解决方法(持续更新)
查看>>
静态代码块与构造代码块
查看>>