RocksDB CentOS - 6.1 Installation with JNI

RocksDB is a way to leverage SSD hardware optimally. It's a way to un congest the network. However, the single digit micro second performance  comes from C++ simple calls of GET , SET on KV structure. Any complexity of data operation, requires custom logic implementation.

 

This blog is all about connecting to RocksDB from Java application. It can also be done using Thrift API.

Download the installation packages

Set the repository location and enable C++ repo.

 

cd /etc/yum.repos.d
wget http://people.centos.org/tru/devtools-1.1/devtools-1.1.repo
yum --enablerepo=testing-1.1-devtools-6 install devtoolset-1.1-gcc devtoolset-1.1-gcc-c++
export CC=/opt/centos/devtoolset-1.1/root/usr/bin/gcc 
export CPP=/opt/centos/devtoolset-1.1/root/usr/bin/cpp
export CXX=/opt/centos/devtoolset-1.1/root/usr/bin/c++
Set  the rocksdb home, download the rocksdb packae from github. Unzip the package.

export ROCKSDB_HOME=/media/ephemeral0/rocksdb
export JAVA_HOME=/usr/java/jdk1.7.0_51
ls $JAVA_HOME/lib/tools.jar

cd /tmp
wget https://github.com/facebook/rocksdb/archive/master.zip
unzip master
mv rocksdb-master $ROCKSDB_HOME
cd $ROCKSDB_HOME  ; pwd ; ls

Enable C++ 2.0

wget https://gflags.googlecode.com/files/gflags-2.0-no-svn-files.tar.gz
tar -xzvf gflags-2.0-no-svn-files.tar.gz
cd gflags-2.0
./configure && make && sudo make install
Setup SNALLY compression

cd ..
wget https://snappy.googlecode.com/files/snappy-1.1.1.tar.gz
tar -xzvf snappy-1.1.1.tar.gz
cd snappy-1.1.1
./configure && make && sudo make install
cd ..

Install Other compression libraries ZLIB and BZIP
sudo yum install zlib
sudo yum install zlib-devel
sudo yum install bzip2
sudo yum install bzip2-devel

Build ROCKSDB

export LD_LIBRARY_PATH=/usr/local/lib/
make clean; make
make check
make librocksdb.so
make librocksdb.a

Build RocksDB JNI

wget -O rocksdbjni.zip https://github.com/fusesource/rocksdbjni/archive/master.zip
unzip rocksdbjni.zip
wget http://apache.tradebit.com/pub/maven/maven-3/3.1.1/binaries/apache-maven-3.1.1-bin.tar.gz
gzip -d apache-maven-3.1.1-bin.tar.gz
tar -xf apache-maven-3.1.1-bin.tar
export SNAPPY_HOME=$ROCKSDB_HOME/snappy-1.1.1; ls -alt $SNAPPY_HOME
export ROCKSDBJNI_HOME=$ROCKSDB_HOME/rocksdbjni-master; ls $ROCKSDBJNI_HOME
export LIBRARY_PATH=${SNAPPY_HOME}
export C_INCLUDE_PATH=${LIBRARY_PATH}
export CPLUS_INCLUDE_PATH=${LIBRARY_PATH}

cd ${ROCKSDB_HOME}
make librocksdb.a
mkdir ${ROCKSDB_HOME}/dist/
cp librocksdb.so ${ROCKSDB_HOME}/dist/
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${ROCKSDB_HOME}/dist/
cd ${ROCKSDBJNI_HOME}
$ROCKSDB_HOME/apache-maven-3.1.1/bin/mvn clean install
cd rocksdbjni-linux64
$ROCKSDB_HOME/apache-maven-3.1.1/bin/mvn install
cd ${ROCKSDBJNI_HOME}
Distribute the JAR File.

cp rocksdbjni/target/*.jar ${ROCKSDB_HOME}/dist/