Category Archives: HowTo

HowTo: Debugging with GDB

Posted on by
2

Since some month I am working with MMORPG emulators again, written in C++. Because of this I needed a powerful debugging tool and so I chose GDB.
I will show you some nice-to-knows for debugging with GDB on a simple code sample.

I recommend at least basic knowledge of C++, Linux and debugging to understand most of the things explained in the following part.

Assume we have this small C++ code block:

int MyDivide(int, int);

int main()
{
    int x, y, result;
    x = 10, y = 2;
    result = MyDivide(x, y);
    x = 4; y = 0;
    result = MyDivide(x, y); // Division by zero
    return result;       
}                        

int MyDivide(int _x, int _y)
{   
    return _x / _y;
}

Now put this in a file, called “divide.cpp”, and compile it to a binary, called “divide”:

$ g++ -g divide.cpp -o divide

After doing this, let us start with the GDB magic and load the “divide” binary:

$ gdb divide 
...
(gdb)

Everything is prepared, so here are some useful things you can do with GDB:

  1. Start application:
    2. (gdb) run
Starting program: /home/dennis/Entwicklung/Misc/divide 

Program received signal SIGFPE, Arithmetic exception.
0x0000000000400513 in MyDivide (_x=4, _y=0) at divide.cpp:15
15          return _x / _y;

    In our case the startup of the application results in a crash, due to division by zero.

  2. Backtraces:

    3. Because of the crash, we now can get more detailed information with the backtrace.

    (gdb) bt
#0  0x0000000000400513 in MyDivide (_x=4, _y=0) at divide.cpp:15
#1  0x00000000004004f9 in main () at divide.cpp:9
(gdb) bt full
#0  0x0000000000400513 in MyDivide (_x=4, _y=0) at divide.cpp:15
No locals.
#1  0x00000000004004f9 in main () at divide.cpp:9
        x = 4
        y = 0
        result = 5
  3. Breakpoints:
    4. (gdb) break divide.cpp:7
Breakpoint 1 at 0x4004ca: file divide.cpp, line 7.
(gdb) break divide.cpp:9
Breakpoint 2 at 0x4004ea: file divide.cpp, line 9.
  4. Step debugging:
    5. (gdb) run
Starting program: /home/dennis/Entwicklung/Misc/divide 

Breakpoint 1, main () at divide.cpp:7
7           result = MyDivide(x, y);
(gdb) next
8           x = 4; y = 0;
(gdb) continue
Continuing.

Breakpoint 2, main () at divide.cpp:9
9           result = MyDivide(x, y); // Division by zero

    With the “next” command, GDB will go to the next line, which is executed by the application. As opposed to this the “continue” command will simply continue the application.

  5. Variable manipulation:
    6. (gdb) print y
$1 = 0
(gdb) set y = 1
(gdb) print y
$2 = 1
  6. Exiting application:
    7. (gdb) quit

I hope this tutorial helps you to work with GDB and to get some personal achievements with debugging your applications.

If you have feedback, regards, corrections or questions please let me know and do not hesitate to comment!

HowTo: Convert Subversion to Git

Posted on by
2

To make a start I can say that I personally prefer using Git as Version Control System. I use Subversion and Mercurial for some reasons too, but I do not really like to work with them.

Because of this I wanted to convert some Subversion repositories to Git. If you also have planned to do this, the following may help you.

Three different ways for the conversion:

  1. Copying the source code:

    2. The easiest and fastest way is simply copying the source code into an empty Git repository. For me this solution was impracticable, because you will loose the history.

    # Create and switch to new folder
    mkdir $GIT_REPO
    cd $GIT_REPO

    # Copy all files from the Subversion directory
    # Use -C option to ignore .svn directories
    rsync -C $PATH_TO_SVN_REPO/* $PATH_TO_GIT_REPO/

    # Do the Git steps
    git init
    git add *
    git commit -asm "Initial commit"

  2. Using git-svn:

    3. The next way to convert the repository is using git-svn. With git-svn you are able to adopt all branches and the whole history. The only unsuitable thing is converting tags, because they are created as branches by default and you need to switch them manually afterwards.

    # Install the software
    (sudo) apt-get install git-svn

    # Make and switch to new folder
    mkdir $GIT_FOLDER
    cd $GIT_FOLDER

    # Convert the repository
    git svn clone $SVN_URL --no-metadata --stdlayout .

    # Update the repository
    git svn fetch

  3. Using svn2git:

    4. Svn2git is a utility which uses git-svn internally to do the real conversion, but in addition it does some more jobs around this to perfect the system. With this you will have a accurate converted repository. It uses Ruby, so you need to install some additional packages.
    You can use svn2git for some special actions and custom purposes. If you need some more information you can take a look at the README.

    # Install the software
    (sudo) apt-get install git-svn ruby rubygems
    (sudo) gem install svn2git

    # Make and switch to new folder
    mkdir $GIT_FOLDER
    cd $GIT_FOLDER

    # Convert the repository
    svn2git $SVN_URL

    # Update the repository
    svn2git --rebase

I hope this tutorial helps you to save some time and troubles. If you have feedback, regards, corrections or questions please let me know and do not hesitate to comment!

I will probably write something about Version Control Systems, like a comparison or Cheat-Sheets, in other blog posts.

HowTo: MySQL Performance Tuning

Posted on by
2

At first I want to clarify that this is only a guideline for tuning your MySQL Performance, due to the fact that every MySQL Server is used for custom purposes and needs. There can be huge differences for application use, like READ-heavy or WRITE-heavy databases.

I recommend advanced knowledge of MySQL to understand most of the things explained in the following part.

This post includes some basic tips to increase your MySQL Server performance. Maybe I will write some other posts about more specific topics, like buffers or caching.

Tips for MySQL performance optimization:

  1. Storage Engine:

    2. In my last post I described and explained the differences, advantages and disadvantages of the most popular MySQL storage engines: MyISAM vs. InnoDB. The choice of the storage engine that fits best to your needs is very important. If you have done the correct selection this can have huge performance enhancement and more features too!

  2. Table optimization:

    3. Another common problem is table fragmentation, which can have great performance impact too. The main problem with fragmentation is bad ordering of indexes on disks, which results in slower query processing due to I/O issues.
    On my github profile you will find a tool for checking and optimizing your tables: MySQL fragmentation tool.

    Here is a short explanation how you can use this:
    # Show table count of fragmented tables in your databases
    ./mysqlfragfinder.sh --user $USER --password $PASS --check
    # Show fragmented tables in your databases
    ./mysqlfragfinder.sh --user $USER --password $PASS --check --detail
    # Optimize all tables
    ./mysqlfragfinder.sh --user $USER --password $PASS
    # Optimize all tables of one specific database
    ./mysqlfragfinder.sh --user $USER --password $PASS --database $DB

    So for better performance you should optimize your tables regularly. But beware of using this with productive databases, because optimizing a table will lock it!

  3. Tools for analysis:

    4. This section is only for advanced users, because you need experience in MySQL server configuration. There are some tools with which you can analyze your MySQL server very well. With the help of them, you are able to find possible configuration failures or optimization targets. You find them on my github account too: MySQL Tuner and MySQL Report.
    MySQL Tuner itself displays tips and data in its output, whereas MySQL Report displays more detailed data. Besides MySQL Report has a really great guide in which their output is explained: Guide for MySQL Report.
    Both tools are really helpful, but they will not do the configuration for you. So be sure you know what you are doing if you follow their tips!

    My recommendation is taking a closer look on following topics:

    • Slow queries

      • You should ensure that the slow query log is enabled in your configuration. If you have slow queries try to get rid of them immediately! There is nothing worse than queries, which keep your MySQL server stuck.

    • Key buffer

      • The key buffer is very important for your indexes and you should always guarantee that there is enough free buffer space left. With the help of this the indexes of the tables are processed in memory and not on disks. So if you use them this will result in a huge performance boost.

    • Query cache

      • The query cache is used by SELECT-statements which for example get executed very often and their result changes very rarely. If they are cached the result comes out of the memory, which is much faster than from disk. So you should set the query cache value high enough to profit from this advantage.

    • InnoDB buffer pool

      • This is only available within the InnoDB storage engine and it should be set to the size of all InnoDB tables plus a margin, so that they can expand. The buffer pool is one huge advantage of InnoDB, because it reads additional data from memory too.

    Of course most of the options and variables depend on your custom purposes and vary due to that. Because of that you need to check the special output of the tools for possible problems by yourself.

I hope this tutorial gave you some hints and tips to optimize your MySQL server. If you have feedback, regards, corrections or questions please let me know and do not hesitate to comment!

If my time permits and there is some demand, I will write some posts on more specific topics of this post too.

HowTo: Install Percona Server

Posted on by
Reply

As I promised in the last post, I am going to show you how to install the MySQL-Server solution of Percona.

There are two ways of installing Percona Server: Via package manager or from source. Maybe some of you think that it is not needed to install Percona Server from source and they are right. It is not really needed, but it has some advantages which you will see later on. For the sake of completeness I will show both ways.

This HowTo is based on my personal purposes and needs, so there are other possibilities to get Percona running for sure.
My favourite operation systems are Debian and Ubuntu and I currently use Percona-Server v5.5.15-21.0, but will probably upgrade in some weeks.

I recommend at least basic knowledge of MySQL and UNIX to understand most of the things explained in the following part.

The two ways of installing Pecona Server:

  1. Installion via packet manager

    2. The only thing you have to do is getting the signed key of Percona and adding the repositories to your source list. With this you are able to install the software automatically via package manager.
    You will need root access to do following actions.

    # Get the key
    $ gpg --keyserver hkp://keys.gnupg.net --recv-keys 1C4CBDCDCD2EFD2A
    $ gpg -a --export CD2EFD2A | sudo apt-key add -

    # Add the repositories
    # Please replace $RELEASE with your current OS release below
    $ echo -e "\n# Percona Server\ndeb http://repo.percona.com/apt $RELEASE main\ndeb-src http://repo.percona.com/apt $RELEASE main" >> /etc/apt/sources.list

    After that you are ready to install Percona Server yet!

    $ apt-get install percona-server-server-5.5

  2. Installation from source

    3. This version is more complex, but you have the possibility to install the application for custom purposes.
    You will need root access to do following actions.

    # Add mysql user
    $ useradd -s /bin/false -b /opt/mysql -d /opt/mysql -m mysql

    # Install MySQL client
    $ apt-get install mysql-client-5.1

    I prefer installing the client previously, because it already creates the MySQL configuration files like the my.cnf in /etc/mysql/ which is not created by this installation method. Indeed Percona provides different versions of my.cnf files for different purposes, but I will topic this in another tutorial.

    Now we can start with the basic installation.

    # Install required packages
    $ apt-get install automake libtool g++ ncurses-dev bison

    # Get and extract Percona source
    $ wget http://www.percona.com/redir/downloads/Percona-Server-5.5/Percona-Server-5.5.15-21.0/source/Percona-Server-5.5.15-rel21.0.tar.gz
    $ tar xvfz Percona-Server-5.5.15-rel21.0.tar.gz
    $ cd Percona-Server-5.5.15-rel21.0

    # Prepare build
    $ sh BUILD/autorun.sh
    $ ./configure --without-plugin-innobase --with-plugin-innodb_plugin --prefix=/opt/mysql

    # Create directory for logging
    $ mkdir /var/log/mysql
    $ chown mysql:mysql /var/log/mysql

    I prefer to install MySQL in /opt/mysql, so you can specify this with the –prefix option. The two other options trigger the usage of XtraDB instead of the build-in InnoDB storage engine. I wrote something about this in my last article of Percona Server. For increasing InnoDB performance it is very important to use this!

    # Build the sources
    $ make -j
    $ make install

    # Install basic database
    $ cd /opt/mysql
    $ ./scripts/mysql_install_db --user=mysql --basedir=/opt/mysql --datadir=/opt/mysql/data --verbose --log-error=/tmp/mysql.error.log --defaults-file=/etc/mysql/my.cnf

    # Set MySQL user privileges
    $ chown -R mysql:mysql /opt/mysql

    If you followed these steps correctly you should have a database which is almost ready to run. We will now create some files to be able to start and stop MySQL server easily and provide correct startup and shutdown behaviour with OS actions.

    # Use Percona support files
    $ echo "/opt/mysql/support-files/mysql.server start" > /etc/init.d/mysql_start
    $ echo "/opt/mysql/support-files/mysql.server stop" > /etc/init.d/mysql_stop
    $ chmod 755 /etc/init.d/mysql_*

    # Link them to rc directories
    $ ln -s ../init.d/mysql_start /etc/rc2.d/S19mysql
    $ ln -s ../init.d/mysql_stop /etc/rc0.d/K21mysql
    $ ln -s ../init.d/mysql_stop /etc/rc6.d/K21mysql

    The last thing you have to do before startup are small adjustments to your MySQL configuration. There have to be some necessary changes to get this working correctly. The advantage of my.cnf is that it has an include path for custom configuration files. So let us use this due to the fact that the content of these files overwrite my.cnf options.

    # Add custom paths
    $ echo -e "[mysqld]\n\nbasedir = /opt/mysql\ndatadir = /opt/mysql/data\n" > /etc/mysql/conf.d/mysql_additon.cnf

    # Required XtraDB config
    $ echo -e "innodb_file_per_table = 1\ninnodb_file_format = barracuda" >> /etc/mysql/conf.d/mysql_additon.cnf

    I just set the correct base- and data-dir of MySQL, because I prefer this setting. It is up to you how handle this. The XtraDB configuration settings I did are really important! The first one (innodb_file_per_table) is set to 1 and used for reducing file access and therefore reducing I/O. This becomes more important if you have slow disks, but is recommended by me anyways. The second one (innodb_file_format) is set to barracuda, which is the only supported table format for XtraDB.

    Now you should be able to start and stop your MySQL server with following commands:

    # Start
    $ /etc/init.d/mysql_start
    # Stop
    $ /etc/init.d/mysql_stop

I hope this tutorial helped you to install and configure your Percona Server. If you have feedback, regards, corrections or questions please let me know and do not hesitate to comment!

Keep in mind that there will follow guides for MySQL performance tuning and a backup solution for InnoDB and MyISAM tables.