java seriliazers benchmarks(2011-07-13).mht

Gary did some tests in CASSANDRA-1765 and found no significant advantage over Thrift. Given that, and our brief experience supporting a second rpc protocol (Avro in the 0.7 series), I don't think this is going anywhere.

Performance improvement available with this patch will be the following:

• Reducing serialization cost and the data size
• Increase throughput between clients and a Cassandra node

I have also measured the performance of MessagePack, from the viewpoints of reducing serialization cost and throughput. I will discuss details below.

== Reduction of serialization cost and the data size ==

(Summary)
MessagePack has proved to be better in reducing serialzation cost and the data size compared to other serialization libraries in the test below.

(Test environment)
I used "jvm-serializers" which is a well-known benchmark and compared performances with Protocol Buffers, Thrift, and Avro. Machine used for this benchmark has Core2 Duo 2GHz with 1GB RAM.

(Results)
　　　　　　create　　ser　+same　deser　+shal　+deep　total　size　+dfl
protobuf 　　　683　6016　2973　　3338　　3454　3759　9775　239　149
thrift 　　　　　572　6287　5565　　3479　　3616　3770　10057　349　197
msgpack 　　　291　4935　4750　　3468　　3545　3708　8748　236　150
avro　　　　　2698　6409　3623　　7480　　9301　10481　16890　221　133

(Comments)
It may be better to compare serialization cost using objects with Cassandra like a Column object. But such objects and sizes vary by users, and is not suitable for comparing serialization cost of various data. According to the above result, the size of MessagePack's serialized data is slightly larger than Avro. But MessagePack has significantly low serialization cost compared to Avro and Thrift.

== Increasing throughput ==

(Summary)
I compared MessagePack based RPC of Cassandra to that of Thrift. Random read throughput of MessagePack based RPC is 15% higher than that of Thrift and random write throughput is 21% higher.

(Test environment)
In this evaluation, Cassandra node ran as a standalone on a machine with Core2 Duo 2GHz and 1GB RAM. Client programs ran on two machines both with Core2 Duo 2GHz and 1GB RAM. Client program was based on ring cache. It created 100 threads per a JVM on each machine and accesses to a Cassandra node with ring cache.

(Results)

• Thrift based RPC part of Cassandra(read: 5,200 query/sec., write: 11,200 query/sec.)
• MessagePack based RPC part of Cassandra (read: 6,000 query/sec., write: 13,600 query/sec.)

(Comments)
I measured the max throughput of random access (read/write) after 100 items (size of each item is small) were stored in the Cassandra node. The reason is because I wanted to make the state of CPU bottle neck for the Cassandra node. If the Cassandra node is the state of Disk IO bottle neck, I thought that I cannot properly evaluate max throughput of the RPC part.

I did not measure the amount of data transferred in network during the evaluation directly. But from the benchmark result of jvm-serializers, I believe that the amount of transferred data for MessagePack-based Cassandra would be reduced compared to that of Thrift.

시스템을 설계하면서 네트웍으로 데이터를 전송/수신하기 위한 포맷은 위의 자료로 대부분 살펴본 것 같습니다. 

개발하는 시스템의 성격에 따라서 속도와 사이즈 어떤 것이 중요할지는 개발자의 판단이고.. 

이 자료가 매우 도움이 되었으면 좋겠습니다.. ^^

How does Cassandra differ from Hadoop?

The primary difference between Cassandra and Hadoop is that Cassandra targets real-time/operational data, while Hadoop has been designed for batch-based analytic work.

There are many different technical differences between Cassandra and Hadoop, including Cassandra’s underlying data structure (based on Google’s Bigtable), its fault-tolerant, peer-to-peer architecture, multi-data center capabilities, tunable data consistency, all nodes being the same (no concept of a namenode, etc.) and much more.

How does Cassandra differ from HBase?

HBase is an open-source, column-oriented data store modeled after Google Bigtable, and is designed to offer Bigtable-like capabilities on top of data stored in Hadoop. However, while HBase shared the Bigtable design with Cassandra, its foundational architecture is much different.

A Cassandra cluster is much easier to setup and configure than a comparable HBase cluster. HBase’s reliance on the Hadoop namenode equates to there being a single point of failure in HBase, whereas with Cassandra, because all nodes are the same, there is no such issue.

In internal performance tests conducted at DataStax (using the Yahoo Cloud Serving Benchmark – YCSB), Cassandra offered literally 5X better performance in writes and 4X better performance on reads than HBase.

How does Cassandra differ from MongoDB?

MongoDB is a document-oriented database that is built upon a master-slave/sharding architecture. MongoDB is designed to store/manage collections of JSON-styled documents.

By contrast, Cassandra uses a peer-to-peer, write/read-anywhere styled architecture that is based on a combination of Google BigTable and Amazon Dynamo. This allows Cassandra to avoid the various complications and pitfalls of master/slave and sharding architectures. Moreover, Cassandra offers linear performance increases as new nodes are added to a cluster, scales to terabyte-petabyte data volumes, and has no single point of failure.

README.txt는 카산드라에 대한 개략적인 내용과 설치에 대한 내용이 들어 있습니다..
NEWS.txt는 카산드라가 버전업이 되면서, 바뀌는 내용을 개략적으로 기술한 파일입니다..

NOTICE.txt 파일에는 카산드라가 사용하고 있는 Dependency Library에 대한 내용을 포함하고 있습니다..
CHANGES.txt 파일에는 카산드라 버전이 업데이트 되면서, 새롭게 추가되거나 수정된 내용을 이슈트래킹 번호를 포함해서 보여주고 있습니다.. 따라서 자세한 내용은 CHANGES.txt 파일의 이슈트래킹 번호를 따라 들어가게 되면, 버전 업에 따른 좀 더 세부적인 내용을 살펴볼 수 있습니다.. 

Cassandra.Client는 org.apache.cassandra.thrift 패키지의 클래스이고, 결국, Client는 Cassandra의 Inner 클래스가 됩니다..그리고, 위 패키지는 Thrift라는 데이타 serialize/deserialize 라이브러리(Google Protocol Buffers랑 비슷)를 통해서 전송될 데이타를 만들고, 전송된 데이타를 처리하고 있겠죵.. Cassandra의 언어별 클라이언트 라이브러리는 http://wiki.apache.org/cassandra/ClientOptions 페이지에 자세히 기술이 되어 있습니다.. 클라이언트 라이브러리들은 보통 Cassandra 서버에 붙는 Connection에 대한 풀링을 제공하는데, Connection 풀링은 org.apache.cassandra.thrift 패키지의 Cassandra.Client를 풀링해서 구현 할 수 있습니다. 
아래 코드는 Cassandra.Client의 풀링을 통한 Cassandra 서버의 연결에 대한 풀링기능을 제공하고 있습니다..^^

* CassandraClientArrayFactory.java

* CassandraClientArrayFactoryTest.java 

1.Definition
2.History & Reference
3.Features
4.Data Model
5.Project Package
6.Performance
7.적용가능 부분
8.Q&A

Cassandra.pdf

/**
 * The ConsistencyLevel is an enum that controls both read and write behavior based on <ReplicationFactor> in your
 * storage-conf.xml. The different consistency levels have different meanings, depending on if you're doing a write or read
 * operation. Note that if W + R > ReplicationFactor, where W is the number of nodes to block for on write, and R
 * the number to block for on reads, you will have strongly consistent behavior; that is, readers will always see the most
 * recent write. Of these, the most interesting is to do QUORUM reads and writes, which gives you consistency while still
 * allowing availability in the face of node failures up to half of <ReplicationFactor>. Of course if latency is more
 * important than consistency then you can use lower values for either or both.
 *
 * Write:
 *      ZERO    Ensure nothing. A write happens asynchronously in background
 *      ANY     Ensure that the write has been written once somewhere, including possibly being hinted in a non-target node.
 *      ONE     Ensure that the write has been written to at least 1 node's commit log and memory table before responding to the client.
 *      QUORUM  Ensure that the write has been written to <ReplicationFactor> / 2 + 1 nodes before responding to the client.
 *      ALL     Ensure that the write is written to <code>&lt;ReplicationFactor&gt;</code> nodes before responding to the client.
 *
 * Read:
 *      ZERO    Not supported, because it doesn't make sense.
 *      ANY     Not supported. You probably want ONE instead.
 *      ONE     Will return the record returned by the first node to respond. A consistency check is always done in a
 *              background thread to fix any consistency issues when ConsistencyLevel.ONE is used. This means subsequent
 *              calls will have correct data even if the initial read gets an older value. (This is called 'read repair'.)
 *      QUORUM  Will query all storage nodes and return the record with the most recent timestamp once it has at least a
 *              majority of replicas reported. Again, the remaining replicas will be checked in the background.
 *      ALL     Not yet supported, but we plan to eventually.
 */