따라서, 데이터를 네트웍으로 전송하기 위한 포맷을 선택하는 것이 시스템 전체적인 성능이나 유연성을 좌우하기에 매우 중요한 팩트가 되었습니다..
그래서, 데이터 포맷별로 테스트한 자료와 성능에 대한 검색자료를 정리해 봅니다.. ^^
1. Java Serialize Benchmarking
- 이 테스트 자료는 자바환경에서 https://github.com/eishay/jvm-serializers 프로젝트에서 테스트한 자료입니다.
- 테스트 결과중에 속도에 대한 차트만 첨부했습니다. 예전의 테스트 자료에서는 kryo가 protobuf보다 더 빠른 결과가 나왔었는데, 2011-07-13일 버전의 테스트 자료에서는 역전을 했네요.. ^^ 그리고, 위의 테스트 결과 위키에서의 답글(2011-10-27)을 보면, 테스트 결과에서는 kyro가 상당히 의미있는 결과를 보여주고 있네요.. 그리고 테스트 결과를 바탕으로 json 포맷을 지원하는 라이브러리 선택도 의미가 있을 것 같습니다.
- 이 테스트 자료는 상당히 방대한 자료에 대한 테스트를 진행했기 때문에 정말 유익합니다.. 꼭 자세히 살펴볼 필요가 있습니다.. 하지만, 최근(?)에 사용을 늘려가고 있는 MessagePack과 YAML에 대한 자료가 없는건 아쉽네요..
- 테스트한 결과에 대한 스냅샷을 위해서 .mht 파일로 첨부했습니다..
java seriliazers benchmarks(2011-07-13).mht2. MessagePack에 대해서 고려한 카산드라..
- 카산드라 0.7버전에서 MessagePack을 thrift의 대안으로 고려를 했었습니다.. 물론 카산드라 버전에서 MessagePack 라이브러리를 살펴볼 수 없기 때문에 채택이 안되었겠죠.. ^^ 카산드라의 JIRA CASSANDRA-1735에서 카산드라 프로젝트의 의장인 Jonathan Ellis의 코멘트를 보면, 의미가 없었다고 하네요..
Gary did some tests inCASSANDRA-1765and found no significant advantage over Thrift. Given that, and our brief experience supporting a second rpc protocol (Avro in the 0.7 series), I don't think this is going anywhere.
- JIRA에서 살펴보면, 테스트 결과는 꽤 성능향상 이점(random read 15%, random write 21%)이 있어 보이긴 하네요.. 흠.. 위의 Jonathan Ellis의 코멘트가 사실이라면, 역시 테스트는 데이타에 따라 결과에 대한 차이가 상당한 것 같습니다..
Performance improvement available with this patch will be the following:
- Reducing serialization cost and the data size
- Increase throughput between clients and a Cassandra node
I have also measured the performance of MessagePack, from the viewpoints of reducing serialization cost and throughput. I will discuss details below.
== Reduction of serialization cost and the data size ==
(Summary)
MessagePack has proved to be better in reducing serialzation cost and the data size compared to other serialization libraries in the test below.(Test environment)
I used "jvm-serializers" which is a well-known benchmark and compared performances with Protocol Buffers, Thrift, and Avro. Machine used for this benchmark has Core2 Duo 2GHz with 1GB RAM.(Results)
create ser +same deser +shal +deep total size +dfl
protobuf 683 6016 2973 3338 3454 3759 9775 239 149
thrift 572 6287 5565 3479 3616 3770 10057 349 197
msgpack 291 4935 4750 3468 3545 3708 8748 236 150
avro 2698 6409 3623 7480 9301 10481 16890 221 133(Comments)
It may be better to compare serialization cost using objects with Cassandra like a Column object. But such objects and sizes vary by users, and is not suitable for comparing serialization cost of various data. According to the above result, the size of MessagePack's serialized data is slightly larger than Avro. But MessagePack has significantly low serialization cost compared to Avro and Thrift.== Increasing throughput ==
(Summary)
I compared MessagePack based RPC of Cassandra to that of Thrift. Random read throughput of MessagePack based RPC is 15% higher than that of Thrift and random write throughput is 21% higher.(Test environment)
In this evaluation, Cassandra node ran as a standalone on a machine with Core2 Duo 2GHz and 1GB RAM. Client programs ran on two machines both with Core2 Duo 2GHz and 1GB RAM. Client program was based on ring cache. It created 100 threads per a JVM on each machine and accesses to a Cassandra node with ring cache.(Results)
- Thrift based RPC part of Cassandra(read: 5,200 query/sec., write: 11,200 query/sec.)
- MessagePack based RPC part of Cassandra (read: 6,000 query/sec., write: 13,600 query/sec.)
(Comments)
I measured the max throughput of random access (read/write) after 100 items (size of each item is small) were stored in the Cassandra node. The reason is because I wanted to make the state of CPU bottle neck for the Cassandra node. If the Cassandra node is the state of Disk IO bottle neck, I thought that I cannot properly evaluate max throughput of the RPC part.I did not measure the amount of data transferred in network during the evaluation directly. But from the benchmark result of jvm-serializers, I believe that the amount of transferred data for MessagePack-based Cassandra would be reduced compared to that of Thrift.
3. Serializing data speed comparison: Marshal vs. JSON vs. Eval vs. YAML
- 이 테스트 자료는 루비로 테스트를 했다고 하네요..
Last night at the NYC Ruby hackfest, I got into a discussion about serializing data. Brian mentioned the Marshal library to me, which for some reason had completely escaped my attention until last night. He said it was wicked fast so we decided to run a quick benchmark comparison.
The test data is designed to roughly approximate what my stored classifier data will look like. The different methods we decided to benchmark were Marshal, json, eval, and yaml. With each one we took the in-memory object and serialized it and then read it back in. With eval we had to convert the object to ruby code to serialize it then run eval against that. Here are the results for 100 iterations on a 10k element array and a hash with 10k key/value pairs run on my Macbook Pro 2.4 GHz Core 2 Duo:
user system total realarray marshal 0.210000 0.010000 0.220000 ( 0.220701)array json 2.180000 0.050000 2.230000 ( 2.288489)array eval 2.090000 0.060000 2.150000 ( 2.240443)array yaml 26.650000 0.350000 27.000000 ( 27.810609)
hash marshal 2.000000 0.050000 2.050000 ( 2.114950)hash json 3.700000 0.060000 3.760000 ( 3.881716)hash eval 5.370000 0.140000 5.510000 ( 6.117947)hash yaml 68.220000 0.870000 69.090000 ( 72.370784)The order in which I tested them is pretty much the order in which they ranked for speed. Marshal was amazingly fast. JSON and eval came out roughly equal on the array with eval trailing quite a bit for the hash. Yaml was just slow as all hell. A note on the json: I used the 1.1.3 library which uses c to parse. I assume it would be quite a bit slower if I used the pure ruby implementation. Here's a gist of the benchmark code if you're curious and want to run it yourself.
If you're serializing user data, be super careful about using eval. It's probably best to avoid it completely. Finally, just for fun I took yaml out (it was too slow) and ran the benchmark again with 1k iterations:
user system total realarray marshal 2.080000 0.110000 2.190000 ( 2.242235)array json 21.860000 0.500000 22.360000 ( 23.052403)array eval 20.730000 0.570000 21.300000 ( 21.992454)
hash marshal 19.510000 0.500000 20.010000 ( 20.794111)hash json 39.770000 0.670000 40.440000 ( 41.689297)hash eval 51.410000 1.290000 52.700000 ( 54.155711)
시스템을 설계하면서 네트웍으로 데이터를 전송/수신하기 위한 포맷은 위의 자료로 대부분 살펴본 것 같습니다.
개발하는 시스템의 성격에 따라서 속도와 사이즈 어떤 것이 중요할지는 개발자의 판단이고..
이 자료가 매우 도움이 되었으면 좋겠습니다.. ^^
'Data' 카테고리의 다른 글
| 데이터 포맷별 성능 비교자료.. (0) | 2011/11/21 |
|---|---|
| .Net Framework 버전별 비교자료.. (2) | 2011/07/18 |
| C# Dataset Java 연동 및 soap vs json 이슈.. (0) | 2011/07/18 |
| Java Proactor 문서.. (0) | 2011/07/18 |
| microsoft application architecture guide v2 (2) | 2010/07/23 |
Net_Framework_버전_별_성능_비교.pdf
NanumGothicCoding_Setup-2.0.exe
