Dariusz on Software

Dziś wchodząc na stronę Allegro znalazłem następujący komunikat:

W godzinach 01:00 do 05:00 serwis Allegro będzie niedostępny. Oferty sprzedaży, które powinny zakończyć się w tym czasie, zostaną przedłużone o 24 godziny. Przepraszamy.

Ciekawe co im się spaliło, że w środku dnia wyłączają wszystkie serwisy ;-) ?

Aktualizacja: (zmienie uległ czas niedostępności):

W godzinach 13:38 do 14:45 serwis Allegro będzie niedostępny. Oferty sprzedaży, które powinny zakończyć się w tym czasie, zostaną przedłużone o 24 godziny. Przepraszamy.

Aktualizacja: kolejna zmiana:

W godzinach 13:38 do 15:15 serwis Allegro będzie niedostępny. Oferty sprzedaży, które powinny zakończyć się w tym czasie, zostaną przedłużone o 24 godziny. Przepraszamy.

Aktualizacja: kolejny termin już został dotrzymany. Allegro działa poprawnie.

Do you want to see man pages in colors? I found this recipe:

export LESS_TERMCAP_mb=$'\E[01;31m'
export LESS_TERMCAP_md=$'\E[01;31m'
export LESS_TERMCAP_me=$'\E[0m'
export LESS_TERMCAP_se=$'\E[0m'
export LESS_TERMCAP_so=$'\E[01;44;33m'
export LESS_TERMCAP_ue=$'\E[0m'
export LESS_TERMCAP_us=$'\E[01;32m'

Here's the result:

Enjoy!

This article shows techniques used to trim memory usage on OpenVZ system (with 128 MB RAM burstable). Mostly inspired by this article from lowendbox.com.

Minimal Debian Lenny install (33MB used):

# ps xo vsz,rsz,ucmd
VSZ    RSZ CMD
1980   692 init
26988  1236 rsyslogd
5272  1024 sshd
2036   792 cron
8016  2916 sshd
4324  1632 bash
3604   804 ps

# free -m
total       used       free     shared    buffers     cached
Mem:           128         33         95          0          0          0
-/+ buffers/cache:         33         95
Swap:            0          0          0

After setting "ulimit -s 128" in /etc/init.d/rc (9MB used):

# ps xo vsz,rsz,ucmd
VSZ    RSZ CMD
1980   692 init
2744  1188 rsyslogd
5272  1020 sshd
2036   792 cron
8016  2888 sshd
4324  1628 bash
3604   804 ps

# free -m
total       used       free     shared    buffers     cached
Mem:           128          9        118          0          0          0
-/+ buffers/cache:          9        118
Swap:            0          0          0

Next operation: replace rsyslogd with syslog-ng (still 9 MB used):

# ps xo vsz,rsz,ucmd
   VSZ   RSZ CMD
  1980   688 init
  5272  1040 sshd
  2036   792 cron
  2848   748 syslog-ng
  8016  2892 sshd
  4360  1712 bash
  3604   808 ps

# free -m
             total       used       free     shared    buffers     cached
Mem:           128          9        118          0          0          0
-/+ buffers/cache:          9        118
Swap:            0          0          0

I replaced bash with dash (smaller memory usage). Free memory remains the same (we keep bash for root interactive login). Bigger benefits come from using dropbear instead of sshd:

# ps xo vsz,rsz,ucmd
   VSZ   RSZ CMD
  1980   688 init
  2036   792 cron
  2848   748 syslog-ng
  2040   468 dropbear
  2700  1240 dropbear
  2788  1512 bash
  2084   704 ps

# free -m
             total       used       free     shared    buffers     cached
Mem:           128          4        123          0          0          0
-/+ buffers/cache:          4        123
Swap:            0          0          0

Yes! we're using only 4MB of VSZ now!.

Next, we would like to install a database, MySQL is a most popular choice here:

# ps xao vsz,rsz,ucmd
   VSZ   RSZ CMD
  1980   688 init
  2036   792 cron
  2848   808 syslog-ng
  2040   468 dropbear
  2700  1332 dropbear
  2796  1564 bash
  1704   516 mysqld_safe
 62440 17968 mysqld
  1628   532 logger
  2084   700 ps

# free -m
             total       used       free     shared    buffers     cached
Mem:           128         61         66          0          0          0
-/+ buffers/cache:         61         66
Swap:            0          0          0

Pretty fat. Let's remove innodb support first:

# ps xao vsz,rsz,ucmd
   VSZ   RSZ CMD
  1980   688 init
  2036   792 cron
  2848   808 syslog-ng
  2040   468 dropbear
  2700  1332 dropbear
  2804  1572 bash
  1704   516 mysqld_safe
 44648  7208 mysqld
  1628   532 logger
  2084   700 ps

# free -m
             total       used       free     shared    buffers     cached
Mem:           128         43         84          0          0          0
-/+ buffers/cache:         43         84
Swap:            0          0          0

After applying minimal settings on my.conf file:

# ps xao vsz,rsz,ucmd
   VSZ   RSZ CMD
  1980   688 init
  2036   792 cron
  2848   808 syslog-ng
  2040   468 dropbear
  2700  1332 dropbear
  2804  1572 bash
  1704   516 mysqld_safe
 28256  5320 mysqld
  1628   532 logger
  2084   708 ps

# free -m
             total       used       free     shared    buffers     cached
Mem:           128         27        100          0          0          0
-/+ buffers/cache:         27        100
Swap:            0          0          0

Now lightpd installation:

# ps xao vsz,rsz,ucmd
   VSZ   RSZ CMD
  1980   688 init
  2036   792 cron
  2848   808 syslog-ng
  2040   468 dropbear
  2700  1332 dropbear
  2804  1572 bash
  1704   516 mysqld_safe
 28256  5320 mysqld
  1628   532 logger
  1764   512 portmap
  3136   504 famd
  5408  1004 lighttpd
  2084   704 ps

# free -m
total       used       free     shared    buffers     cachedMem:           128         28         99          0          0          0
-/+ buffers/cache:         28         99
Swap:            0          0          0

Not bad. 28 MB used so far.

UPDATE: I found interesting article on optimizing MySQL and added the following options:

query_cache_size = 50K
query_cache_limit = 50K

Memory usage after change:

# ps xao vsz,rsz,ucmd
VSZ   RSZ CMD
1980   688 init
1764   512 portmap
2796  1248 rsyslogd
5404  1008 lighttpd
3136   500 famd
2036   796 cron
2040   468 dropbear
2696  1340 dropbear
2804  1576 bash
1704   516 mysqld_safe
11436  5112 mysqld
1628   532 logger
2084   700 ps

# free -m
total       used       free     shared    buffers     cached
Mem:           128         12        115          0          0          0
-/+ buffers/cache:         12        115
Swap:            0          0          0

12 MB VSZ with lightpd and MySQL! :-)

Inspired by lowendbox.com article I wanted to experiment with custom OS installations on low-end OpenVZ environment (where memory constraints differ when compared to Xen or dedicated server), so purchased lowest plan from Ram Host: Nano:

• OpenVZ
• 80MB RAM
• 10 GB disk space
• 50 GB data transfer

It cost only $2.99/mo, so it's very easy to start experiments with VPS environment for anyone. I paid using PayPal and have to wait next day to service become visible (manual fraud checking on PayPal notifications maybe?).

OpenVZ vs Xen

OpenVZ is technology that is much easier to oversell: disk storage can be easily shared between customers (one filesystem for all customers), the same with memory. Another problem that may occur when many customers are placed on one node is host swapping: your "dedicated" memory may be placed inside swap file. Imagine your VPS response time then ;-)

For serious applicances I'm using only Xen-based solutions. You have dedicated RAM, dedicated disk (filesystem is truly yours). Sharing is only performed on CPU (equal share typically) and bandwitch. Typical Xen offer starts at $20/mo (I recommend Linode.com for stable Xen solution).

So: why I'm testing RAM Host then? I was intrigued by fact that they publish host usage statistics (not very common among hosting providers, a good sign host has nothing to hide.):

Control panel

After a2b2.com disaster I don't trust HyperVM users anynmore. RAM Host uses custom panel called RAMCP.

Directly after installation you have clean system even without SSH installed. Good (no unsafe defaults). You must:

1. log to host machine (not your VPS!) on "vz" account
2. then authorize youself on "Console Drop" by your credential configured during sign-up process
3. after that you have root shell and you can install SSH and configure root password to be able to log using SSH

I had problem with executing "aptitude install ssh" (cannot fork new process error) and have to lower default stack size first:

# ulimit -s 128
# aptitude install ssh

After that operation installation was succesful. I installed SSH keys and got the root console.

GIT is a very poverful Source Control Management System we are using for managing most our projects. Let's give a voice GIT's creator, Linux Torvalds:

I assume you're a developer and want to control global log level you are getting on your console window in Eclipse. Just log level. You don't want to learn all Log4J machinery to create many log files, customize logging format etc. I'll describe simplest steps to achieve this.

First, create commons-logging.properties file in your src/ directory (or directory on your classpath):

org.apache.commons.logging.Log=org.apache.commons.logging.impl.SimpleLog

Next, create simplelog.properties in the same location:

org.apache.commons.logging.simplelog.defaultlog=warn
org.apache.commons.logging.simplelog.showlogname=false
org.apache.commons.logging.simplelog.showShortLogname=false
org.apache.commons.logging.simplelog.showdatetime=false

That's all! Your app will now log on WARN level and above in short, one-line format. Isn't it simple?

If you want to give different log levels (info for instance) to different packeges:

org.apache.commons.logging.simplelog.log.<package prefix>=info

In this article I'll present Trac plugins that are most valuable IMHO. Trac is very easy to extend:

1. Download plugin code (as zip of checkout it from subversion)
2. Call python setup.py install as root
3. (Optional) Upgrade database schema if needed: trac-admin <trac-env-path> upgrade
4. Add some options to ini file
5. Restart Trac process

After above steps you should see Your plugin is working.

TracWysiwygPlugin

TracWysiwygPlugin is a must when you are going to place big tables on Your Wiki pages. It's very useful also for novice users that do not know Wiki Syntax very well.

MasterTicketsPlugin

MasterTicektsPlugin allows you to link between tickets. By default it uses as "Blocking" / "Blocked by" marker, I renamed to "Parent", "Children" to express hierarchies in tickets.

Mantis and Trac are both bug tracking systems. A bug tracking system is a software application that is designed to help quality assurance and programmers keep track of reported software bugs in their work. It may be regarded as a sort of issue tracking system (Wikipedia). In this article we will use Trac's alias for an issue: Ticket.

Recently I decided to replace Mantis installation used for one of my customers into Trac. Mantis worked pretty well until the date, but had missing features when compared to Trac:

• Trac supports Wiki syntax both in standalone wiki pages and inside tasks and comments
• Trac's reports and queries are superb! You can easily create custom views of Your tickets and even place it inside Your Wiki pages! Wow!
• Has integration with many version control systems - I'm going to use git integration (existing GIT-based projects)

On the other hand Mantis has some benefits over Trac:

• Comments could be edited after adding (sometimes I need to update my comment, to clarify it for instance)
• Authorisation is configurable (you can restricts access even to single comment)
• You can disable e-mail notifications about Your actions (Trac sends everything)

Migration

First of all you have to create Your Trac project:

$ trac-admin /path/to/myproject initenv

Trac supports importing Mantis database by using this script. There are few small defects in import script so execution is not straightforward (you have to change current directory to Trac project directory):

$ cd <trac-project-directory> $ python ../mantis2trac.py --clean --db <mysql-mantis-database> --tracenv ./ -u <mysql-user> -p<mysql-password>

And finally you have to deploy Trac somewhere. Trac supports the following deployment methods:

• standalone server
• mod_python (Apache)
• cgi
• fastcgi
• mod_wsgi

I selected FastCgi (as it's two times faster than CGI and less web-server dependant than mod_python):

$ trac-admin /path/to/env deploy /path/to/www/trac

Happy Tracking!

When opening JSP file in visual mode in Eclipse (Websphere Integrated Developer 6.1.2) I'm getting the following error (under Debian Lenny):

Caused by: java.lang.UnsatisfiedLinkError: /opt/IBM/WID61/configuration/org.eclipse.osgi/bundles/2374/1/.cp/libswt-mozilla-gtk-3236.so (libxpcom.so: cannot open shared object file: No such file or directory)

I checked for shared library dependicies:

$ ldd /opt/IBM/WID61/configuration/org.eclipse.osgi/bundles/2374/1/.cp/libswt-mozilla-gtk-3236.so linux-gate.so.1 =>  (0xb7fa9000) libxpcom.so => not found libnspr4.so => /usr/lib/libnspr4.so (0xb7f4a000) libplds4.so => /usr/lib/libplds4.so (0xb7f46000) libplc4.so => /usr/lib/libplc4.so (0xb7f42000) libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb7e54000) libm.so.6 => /lib/i686/cmov/libm.so.6 (0xb7e2e000) libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7e21000) libc.so.6 => /lib/i686/cmov/libc.so.6 (0xb7cc6000) libpthread.so.0 => /lib/i686/cmov/libpthread.so.0 (0xb7cac000) libdl.so.2 => /lib/i686/cmov/libdl.so.2 (0xb7ca8000) /lib/ld-linux.so.2 (0xb7faa000)

and checked ldconfig:

$ sudo ldconfig -v | grep libxpcom libxpcom.so.0d -> libxpcom.so.0d libxpcomglue.so.0d -> libxpcomglue.so.0d

library is installed in system:

$ dpkg -S libxpcom.so libxul0d: /usr/lib/xulrunner/libxpcom.so libxul0d: /usr/lib/libxpcom.so.0d xulrunner-1.9: /usr/lib/xulrunner-1.9/libxpcom.so icedove: /usr/lib/icedove/libxpcom.so

I linked the missing libraries:

cd /usr/lib sudo ln -s libxpcom.so.0d libxpcom.so sudo ln -s libxpcomglue.so.0d libxpcomglue.so

That's all!