Magic Reboot: When SSH gets stuck

If you ever need to reboot a host, but you cannot SSH in (you have the password, but the session is frozen), you can use the magic reboot method. Here it is:

ssh root@192.168.0.2 "echo b > /proc/sysrq-trigger"

This will tell host 192.168.0.2 to write b to the /proc/sysrq-trigger file. This will cause an immediate reboot of the host.

Using SCP to send a file to another host

If you can SSH into a host, you can use SCP to send a file securely to it. Here's an example:

scp foobar.txt root@192.168.0.2:/var/log

This will send the file foobar.txt to host 192.168.0.2 as user root and save it to the /var/log directory. SCP will prompt you for a password for the root user, of course, but it works like a charm.

Dumping MySQL Privileges and Users

Doing a mysqldump of a database is easy. You can take the resulting SQL file and re-create the database on another server. What you are missing is the users and their privileges that your app depends on for accessing the database. Here is how you can dump your users and privileges to a SQL file that can be easily imported on another server.

** Make sure to replace XXXXX with your actual root password. There is no space after -p

mysql -u root -pXXXXX -B -N $@ -e "SELECT DISTINCT CONCAT('SHOW GRANTS FOR ''', user, '''@''', host, ''';') AS query FROM mysql.user" | mysql -u root -pXXXXX $@ | sed 's/\(GRANT .*\)/\1;/;s/^\(Grants for .*\)/## \1 ##/;/##/{x;p;x;}' > users.sql

The resulting users.sql file can be edited down to just the users you need and then used with the source command to import them into another MySQL server.

Reduce InnoDB Hard Drive Usage: ibdata1 ib_log

InnoDB databases in MySQL are stored in one huge file by default. This file does not release used space when you delete tables. If you are developing a database and make frequent changes to the structure, you will quickly notice that the size of this file is much larger than all of your databases combined. The file is called ibdata1.

So what can you do about it? MySQL allows you to store InnoDB databases on a file-per-table basis. Here's how you do it:

1. dump all databases, sprocs, triggers, etc (you need to restore them after)
2. DROP all databases (except phpmyadmin or other system DBs)
3. /etc/init.d/mysql stop (stop MySQL server)
4. nano /etc/mysql/my.cnf (edit the MySQL configuration file, add this line)
   
   [mysqld]
   innodb_file_per_table=1
   
5. rm /var/lib/mysql/ibdata1 (delete ibdata1 and ib_log files)
6. /etc/init.d/mysql/start (Start MySQL server)
7. restore dbs from dump (Now all databases will be split into many files)

The difference is that when you drop a table, the corresponding file is deleted and your OS can reclaim that hard drive space.

How To Find The Largest Files On Your LAMP

My development server has been running low on space lately which is strange as I haven't been adding anything to it. I track my space usage with the df command.

df -h

Filesystem            Size  Used Avail Use% Mounted on
/dev/xvda2            8.0G  4.8G  3.3G  60% /
tmpfs                 306M     0  306M   0% /lib/init/rw
udev                  286M   40K  286M   1% /dev
tmpfs                 306M     0  306M   0% /dev/shm
/dev/xvda1            130M   15M  109M  12% /mnt/boot
none                  306M     0  306M   0% /dev/shm

I've noticed the space on xvda2 slowly decreasing over the last few months. What could be causing it? None of my projects are that big, so how do I find what is responsible? Here's the command you'll need:

du -sm * | sort -nr | head -15

Just make sure you cd to the desired directory and this command will show you the 15 largest files and directories below your current directory. I started in my root ( cd / ) and followed the largest directory it found each time. In my case it was the ibdata1 file in MySQL that grew to over 2.6 Gb. I had been making lots of structure changes to my databases and it turns out that InnoDB does not reallocate space after you delete a table or modify something. After a few months my 300 Mb worth of databases were taking up almost 3 Gb.

Jobs in Linux: Sending stuff to the Background

I use SSH to access my servers and often run commands that take a long time to finish. I could open another SSH window, but it's actually possible to multitask in the same command shell. The command you need to know are:

• & (start job in background)
• jobs (list all jobs)
• fg (move a job to the foreground)
• bg  (move a job to the background)
• CTRL + Z (suspend the current job)
• CTRL + C (kill the current job)

The & (ampersand) is used to run a command and send it to the background all in one go. (Example: sleep 100&) If you need to run a command and you know it will take a while, you can run it this way and you'll be able to run other jobs while it runs in the background. You can always see a list of your jobs and their job numbers by using the jobs command.

If you run a command and it's taking too long, you can press CTRL + Z (send it to the foreground AND pause it), then type bg [job number] to send it to the background and resume it. For example:

sleep 100 

CTRL + Z 

^Z 

[1]+  Stopped sleep 100
bg 1
[1]+  Running sleep 100

This will allow you to run other commands while sleep 100 continues running in the background.

If you need to suspend a job running in the background, you can use the fg [job number] command to bring it to the foreground (your active job) and press CTRL + Z to pause it. In the event that you need to kill a job, bring it to the foreground and use CTRL + C to kill it.

Checking File Size and Disk Space in Linux

Linux comes with two commands that are useful for checking disk space usage.

df is used to check the space of mounted partitions
du is used to check the file sizes of files and directories

The most common use for df is to check partition free space. To make the output more human readable, use the -h option.

df -h

If you need to check on the size of a directory or its contents, use the du command. The most common options are -h (human readability), -a (display files and folders, not just folders) and -c (display a total size at the bottom).

cd /some/dir
du -ahc

For more advanced uses, run each command with the --help option for a list of additional options.

Download a File From Linux Command Line Terminal

Getting a file from the internet using the command line is super easy. Simply cd to the directory you want:
   cd /var/www/images

And use wget to download the file you need:
   wget http://www.someplace.com/image3.jpg

That's it! Don't let it fool you, wget is a very advanced tool. For more usage examples go here:
   http://en.wikipedia.org/wiki/Wget#Using_Wget

Install an FTP server on Amazon EC2 LAMP

I've recently created my first server on Amazon EC2 (more on that soon) and I needed to move some stuff over. Here's how to setup a secure FTP server and configure it for your LAMP.

First, install the ftp software:
   apt-get install vsftpd

Next, edit /etc/vsftpd.conf and uncomment or change the following:
   listen=YES
   anonymous_enable=NO
   local_enable=YES
   write_enable=YES
   connect_from_port_20=YES
   guest_enable=NO

Add following lines at the end of file:
   pasv_enable=YES
   pasv_min_port=1044
   pasv_max_port=1048

Next, you'll want to create a local user and a home directory where files will be saved. This keeps your www directory safe from intrusion. Upon connecting over FTP you'll be able to see and read from anywhere on the system, but only upload to one directory.
   adduser ftp_user
   passwd ftp_user
   mkdir /home/ftp_user
   chown ftp_user:users /home/ftp_user

You might only need the first command as your distro will take care of all the rest by prompting you.

Now you need to restart the ftp server to load the new configuration:
   /etc/init.d/vsftpd restart

If you are wondering why I chose to restrict the passive ports to 1044 - 1048, this is because Amazon blocks all ports unless specifically allowed by the security group. You will need to open those ports in your Amazon Console. Click Security Groups, chose your group, click the Inbound tab and create a new Custom TCP rule. Specify ports 1044-1048, click Add Rule and Apply Rule Changes.

Presto! You can now connect to your LAMP over FTP using ftp_user. Your files will be uploaded to /home/ftpuser

Multiple domains and subdomains in Apache2

If you want to host multiple websites and have one server running Apache2, it's pretty simple to setup virtual hosts so that each domain and subdomain will show different content. You'll need to point your domains and subdomains to the IP address of your server first, as DNS takes some time to update. Once that is done it's just a matter of configuring Apache.

You'll want to find your sites folder, it should be /etc/apache2/sites-available  Here you can either edit the default file or create your own. The advantage to creating your own is being able to enable/disable parts of your configuration rather than the whole thing. Either way, below is what you need for each domain/subdomain you wish to configure:

<VirtualHost *>
        DocumentRoot "/var/www/subdomain/"
        ServerName subdomain.cezary.com

        <Directory /var/www/subdomain/>
                Options Indexes FollowSymLinks MultiViews +Includes
                AllowOverride None
                Order allow,deny
                allow from all
        </Directory>
</VirtualHost>

What does all that do? I'm glad you asked.

DocumentRoot  is the local path to your content
ServerName  has to match the domain or subdomain of your site
<Directory ...>  contains options and settings for your site. The ones listed here are pretty standard. Modify these to your liking.

You'll want to make a new <VirtualHost *> entry for each domain and subdomain you host. If you put this configuration in the default file then you are almost done. If you made your own file then there is one more step. You need to enable your custom configuration file. You can do this with the following command (as root):

a2ensite xxFILExx

Where xxFILExx is the name of your own config file. And that's it! Now you just need to tell Apache to reload the new settings.

/etc/init.d/apache2 reload

Enjoy!

Generating QR codes

To generate a QR code on your page simply insert the following:

<img src="http://chart.apis.google.com/chart?cht=qr&chs=100x100&chl=xxxURLxxx" width="100" height="100" />

Replace xxxURLxxx with the link you want. You can also change the size, but 100 by 100 is usually large enough to accommodate most URLs.

If you want to dynamically generate the QR code, you can replace xxxURLxxx with PHP like so:

<img src="http://chart.apis.google.com/chart?cht=qr&chs=100x100&chl=<?php echo $varURL; ?>" width="100" height="100" />

How to install cURL

If you are making your first shopping cart, crawler or simply need cURL to make existing code work, here's how to get it.

First things first, check to see if cURL is already running. Create the following PHP page:

<?php
   phpinfo();
?>

Visit that page and search for "curl". If nothing is found, you don't have cURL. A proper installation should have a reference under "Additional .ini files parsed" as well as a section called "curl" with all the details.

There are several ways to install/enable cURL, depending on your setup. Here are the two most common:

LAMP (as root or using sudo)

apt-get install curl libcurl3 libcurl3-dev php5-curl php5-mcrypt

XAMPP (edit xampp\apache\bin\php.ini and uncomment or add this line)

extension=php_curl.dll

In either case you'll need to restart Apache to load cURL after the installation.

To verify the installation, go to the PHP page you created and search for "curl".

Manage Your Code Deployment: Subversion To Web Server In One Click

Here's the simplest way to push a folder out of your repository onto your web server for testing.

<?php
$result = array();
$command  = 'svn export ';
$command .= 'https://your_server/svn/project /path/to/www ';
$command .= '--username USER --password PASS --force ';
$command .= '--non-interactive --trust-server-cert';
exec($command, $result);
print_r($result);
?>

- Host name HAS to match certificate (where I have "your_server")
- check .htaccess files, mod rewrite, ensure permissions are 755 and www-data is owner
- Version # can be forced with -r switch (svn -r 5 export ...)