Dariusz on Software Quality & Performance

12/04/2013

Ignore sockets during tar backup

Filed under: en — Tags: , — dariusz.cieslak @

If you want to skip error sockets-related code during backup of whole filesystem you will be surprised there's no –ignore-sockets switch. But there's elegant method to to that:

cd /
sudo find . -type s > /tmp/sockets.lst
if sudo tar zcvpf $TMP_FILE --one-file-system --exclude-from=/tmp/sockets.lst -C / .
then
    echo backup OK
fi

25/02/2012

Linux: How To Locate Duplicated Files Quickly

Filed under: en — Tags: , — dariusz.cieslak @

Locate duplicates quickly: I mean only size+filename check, not expensive MD5 sum computation:

find . -printf "%f:%s:%p\n" -type f | \
    awk -F: '
        {
            key=$1 " " $2;
            occur[key]++;
            loc[key]=loc[key] $3 " "
        }
        END {
            for(key in occur) {
                if(occur[key]>1) {
                    print key ": " loc[key]
                }
            }
        }
    ' | sort

A bit of explanation of above magic:

  • printf: tells find command to output file metadata instead of only file path (the default), this metadata (size, filename) will be used later
  • -F: :We want to handle properly paths with spaces, that's why special separator is used
  • key=$1 " " $2: we use file name (without dir) and file size to create ID for this file
  • occur: table (key -> number of file occurences)
  • loc: maps file ID to list of locations found
  • occur[key]>1: we want to show only files that have duplicates
  • sort: results are sorted alphabetically for easier navigation

18/01/2012

HOWTO: cleanup old files under Linux

Filed under: en — Tags: , — dariusz.cieslak @

Sometimes you want to separate old files (for example older than 30 days) into separate subdirectory to make navigation easier. Using Linux tools it's pretty easy task:

mkdir -p OLD; find . -maxdepth 1 -mtime +30 -exec mv \{\} OLD \;

Explanation:

  • mkdir -p OLD: create OLD subdirectory if not exists
  • -maxdepth 1: search only one level under current working directory
  • -mtime +30: find only files that are older than 30 days
  • -exec …. \;: for every file/directory found execute the following command
  • \{\}: will be replaced by filename

Above command will move all old files/subdirectories into OLD subdirectory.

04/01/2012

GIT hooks: commit-msg to enforce commit rules

Filed under: en — Tags: , , — dariusz.cieslak @

Recently I forgot to add #reviewthis directive for modifications of codebase that belongs to team A. And a subtle bug was introduced that way. Ops! I agreed earlier that all changes done to moduleB should be passed to a reviewer that will do peer review for that particular change. What a shame :-( (We are using excellent GitHub's  review mechanism, BTW).

How to avoid that situation in a future? Should I rely on my memory? Is it possible for a human to track so many small rules manually? My intuition tells me that enforcement of those small ruleset should be automated.

GIT allows you to specify so called "commit hooks" that can validate many stages of GIT workflow. I'll use simplest local verification of commit message, first the rule in plain text:

If you are changing moduleB you should notify developerC about this change

(more…)

21/09/2011

Automatic free disk space monitoring on Linux server

Filed under: en — Tags: , , — dariusz.cieslak @

The simplest way to monitor free disk space on Linux serwer, just place it in crontab:

0 8 * * *     df -h | awk '/^\// && $5 > 95 { print "missing space", $0 }'

and ensure e-mail to this user is forwarded to you then you will see e-mail when occupied space is higher than 95% on any disk.

Pretty simple.

Older Posts »

Powered by WordPress