Kyoto Tycoon – Installation with Lua and Memcached Protocol

Prepare

Install Kyoto Cabinet

Installation

#kyototycoon, works with lua 5.1 not 5.2 or later!
mkdir -p /tmp/build
cd /tmp/build
version="0.9.56"
wget http://fallabs.com/kyototycoon/pkg/kyototycoon-$version.tar.gz
tar xvzf kyototycoon-$version.tar.gz
cd kyototycoon-$version

#./configure --with-kc  --enable-lua
#error fix
#ktdbext.h:274:29: error: ‘getpid’ was not declared in this scope
# append line #include 
#http://code.google.com/p/modpagespeed/issues/detail?id=420
if [ "`grep 'include ' ./ktdbext.h`" == "" ] ; then
   sed -i -e 's/#define _KTDBEXT_H/#define _KTDBEXT_H\n#include \n/g' ./ktdbext.h
fi;

#error fix
#myscript.cc:3435:33: error: ‘lua_objlen’ was not declared in this scope
#"Ipe requires Lua 5.1, and does not currently work with Lua 5.2."
#http://sourceforge.net/apps/mantisbt/ipe7/view.php?id=113
#include "myconf.h"

# build and install
sudo ./configure --with-kc=/usr/local/kyotocabinet --enable-lua --with-lua=/usr/lib/x86_64-linux-gnu/
make
sudo make install

Troubleshooting

Problem: missing libraries

root@a:~/kyototycoon-0.9.53# ktserver
#ktserver: error while loading shared libraries: libkyototycoon.so.2: cannot open shared object file: No such file or directory

Solution:
Add the directory “/usr/local/lib” to the file /etc/ld.so.conf

if [ "`grep '/usr/local/lib' /etc/ld.so.conf`" == "" ] ; then
    sudo sh -c ' printf "\n/usr/local/lib" >> /etc/ld.so.conf';
fi;
sudo ldconfig

Error Fixed

ktserver
2012-03-04T12:38:10.305461+01:00: SYSTEM: loading a plug-in server file: path=/usr/local/lib/libexec/ktplugservmemc.so
2012-03-04T12:38:10.305608+01:00: ERROR: could not load a plug-in server file: /usr/local/lib/libexec/ktplugservmemc.so
ktplugservmemc path => /usr/local/libexec/ktplugservmemc.so

#http://code.google.com/p/kyototycoon-neko/wiki/KyotoInstall

start server

sudo mkdir -p /var/log/kyoto/ /var/lib/kyoto/db/
cd /var/log/kyoto/
sudo ktserver -dmn \
-pid /var/lib/kyoto/kyototycoon.pid -log /var/log/kyoto/kyototycoon.log \
-ulim 1024m \
-uasi 600 \
-th 1 \
-plsv /usr/local/libexec/ktplugservmemc.so \
-plex "port=11212#opts=f" \
/var/lib/kyoto/db/kyotodatabase.kch#bnum=100000000#dfunit=8

Notes

...
Memcached Protocol
http://cade.ekblad-frank.com/bullshit/2011/04/09/transitioning-from-memcache-to-kyoto-tycoon/
Question: http://cade.ekblad-frank.com/bullshit/2011/04/09/transitioning-from-memcache-to-kyoto-tycoon/
Response:
Did you read the document? http://fallabs.com/kyototycoon/spex.html#tips_pluggableserver
"opts=f" is the suggested way. Although "options=f" as an alias is
undocumented, "opts=f" is documented.

Regards.
Quote from http://fallabs.com/kyototycoon/spex.html#tips_pluggableserver

The configuration expression by the "-plex" option is composed of named parameters separated by "#". Each parameter is composed of the name and the value separated by "=". The supported parameter names are "host", "port", "tout", "th", and "opts". The default port is 11211, which is the same as the original memcached server. Do not forget "#opts=f" if your client library uses "flags" to determine the data type of each record. By default, "flags" are disabled for compatibility of the record format to HTTP.

http://scale.metaoptimize.com/questions/23/key-value-store-with-lowest-read-latency-tokyo-cabinet-redis-voldemort-cassandra-or-what

Installation
Für die Installation gibt es ein recht einfaches Build-Script "setup_kyoto.sh":
Konfiguration
Alle Konfigurations-Parameter werden auf der Kommandozeile übergeben, die vollständige Liste gibt es unter http://fallabs.com/kyototycoon/command.html#ktserver :

        sudo LD_LIBRARY_PATH=$LD_LIBRARY_PATH -u $RUNAS_USER $BASE/tycoon/${TYCOON_VERSION}/bin/ktserver -dmn \
            -pid $PID_FILE -log $LOG_FILE \
            -ulim 1024m \
            -uasi 600 \
            -th 1 \
            -plsv $BASE/tycoon/${TYCOON_VERSION}/libexec/ktplugservmemc.so \
            -plex "port=$PORT" \
            ${DATA_DIR}/database.kch#bnum=100000000#dfunit=8

    -dmn : Daemon
    -ulim num : specifies the limit size of each update log file.
    -uasi num : specifies the interval of synchronization of update log files. By default, it is disabled.
    -pid file : specifies the file to contain the process ID to send signals by.
    -plsv file : specifies the shared library file of a pluggable server.
    -plex str : specifies the configuration expression of a pluggable server, e.g.

    -plex "port=$PORT#tout=5#th=4"

    (listen to port $PORT, timeout for 'get' is 5 seconds, 4 worker threads)
    -th is set to 1: we just use the main server as continer for running ktplugservmemc.so

Option 1 – Install Python Bindings for Kyoto Tycoon

https://github.com/tmaesaka/python-kyototycoon/wiki

cd /tmp
git clone https://github.com/tmaesaka/python-kyototycoon
cd python-kyototycoon
python setup.py build
sudo python setup.py install

# test python import
python -c "from kyototycoon import KyotoTycoon"

Test Code for Kyoto Tycoon Bindings

from kyototycoon import KyotoTycoon
kt = KyotoTycoon()
kt.open(hostname="12.0.0.1", port=1978, timeout=30)
kt.set('key', 'abc')
value = kt.get('key')
kt.close()

Option 2 – Install Memcache Bindings for Kyoto Tycoon

References

1. http://d.hatena.ne.jp/ishibashits/20110430/1304143562
2. http://cequiestaime.blogspot.com/2011/03/kyoto-tycoon.html
3. http://timyang.net/data/mcdb-tt-redis/
Memcached vs Redis vs Kyoto Tycoon
4. http://dailyadminlife.blogspot.de/2011/07/kyoto-tycoon-memcached-alternative-with.html
Kyoto Tycoon Replication