I assume you want to encrypt some files using your public GPG key. I’ll focus on simplicity rather than completeness (minimal steps required to implement encryption).
First you have to generate key pair:
$ mkdir -p ~/.gnupg
$ gpg --gen-key
Then see your new key ID and export it to public key storage:
$ gpg --list-keys # get KEY_ID from output
$ gpg --keyserver "hkp://subkeys.pgp.net" --send-key <KEY_ID>
On remote machine import the key and make it trusted (to avoid warnings during encryption):
$ gpg --keyserver "hkp://subkeys.pgp.net" --recv-keys <KEY_ID>
$ gpg --edit-key <KEY_ID>
> trust
Then you can used this key to encrypt files and delete original (if needed):
$ gpg -r <KEY_ID> --output <FILE>.gpg --encrypt <FILE>
$ rm <FILE>
And the decryption (on host where private key is stored):
$ gpg -r <KEY_ID> --output <FILE> --decrypt <FILE>.gpg
Oops! Someone forgot to renew a SSL certificate

Recently I’ve registered an account on twitterfeed.com site that forwards blog RSS-es to Twitter and Facebook accounts. Headers of incoming mail attracted my attention:
Return-Path: root@mentiaa1.memset.net
(...)
Received: from mentiaa1.memset.net (mentiaa1.memset.net [89.200.137.108])
by mx.google.com with ESMTP id 11si7984998ywh.80.2010.02.12.06.24.18;
(...)
Received: (from root@localhost)
by mentiaa1.memset.net (8.13.8/8.13.8/Submit) id o1CERcGq004355;
(...)
From: noreply@twitterfeed.com
(...)
Interesting parts are bolded out. As you can see registering e-mail was sent from root account. Probably the same user id is used for WWW application. That means if you break the WWW application you can gain control over whole server.
The preferred way to implement WWW services is to use account that has low privileges (www-data in Debian) because breaking the service will not threat whole server.
