SVN backup script

Here something to drop to an SVN server:

#!/bin/bash
SUFIX=`date +%Y%m%d`
DIR_BASE=/tmp/`hostname`_svn_
DIR_WRK=$DIR_BASE$SUFIX
DIR_REPOS=/path/to/a/directory/with/svn/repositories
REPOS="SVNRepo1 SVNRepo2"
STORAGE=backup@int.storage.server:/path/to/backup/location

# -- init working directory
echo "`date` - Preparing backup directory"
rm -rf $DIR_BASE*
mkdir -p $DIR_WRK
echo "`date` - Changing into backup directory '$DIR_WRK'"
cd $DIR_WRK

for repo in $REPOS
do
 path=$DIR_REPOS$repo
 if [ -d $path ]; then
  mkdir $repo
  echo "`date` - Backing up '$repo' to '$repo/repository.dump'"
  svnadmin dump $path > $repo/repository.dump
  cp $path/access-policy $repo/.
  cp $path/svn-auth-file $repo/.
  echo "`date` - Compressing '$repo' to '$repo.tgz'"
                tar -czf $repo.tgz $repo
                echo "`date` - Deleting '$repo'"
                rm -rf $repo
 else
  echo "`date` - Repository '$path' does not exist"
  continue
 fi
done

# -- transfer to a safe place
echo "`date` - Transfering backup directory to a remote location"
rsync -re ssh $DIR_WRK $STORAGE

echo "`date` - Cleaning up backup directory"
rm -rf $DIR_WRK
echo "`date` - Done"

MySQL backup script

Nothing extraordinary, just a backup script which runs on our production db servers each night:

#!/bin/bash
SUFIX=`date +%Y%m%d`
DIR_BASE=/tmp/`hostname`_db_
DIR_WRK=$DIR_BASE$SUFIX
DB_USER=backup
DB_PASSWD=abcde
STORAGE=backup@int.storage.server:/path/to/backup/location

# -- init working directory
echo "`date` - Preparing backup directory"
rm -rf $DIR_BASE*
mkdir -p $DIR_WRK
echo "`date` - Changing into backup directory '$DIR_WRK'"
cd $DIR_WRK

# -- backup all databases
for db in `mysql -u $DB_USER -p$DB_PASSWD -e 'SHOW DATABASES;'`
do
case $db in
 Database|information_schema) continue;;
 *)
  echo "`date` - Backing up '$db' to '$db.sql'"
  mysqldump -u $DB_USER -p$DB_PASSWD --opt $db > $db.sql
  echo "`date` - Compressing '$db.sql' to '$db.tgz'"
  tar -czf $db.tgz $db.sql
  echo "`date` - Deleting '$db.sql'"
  rm $db.sql
  ;;
esac
done

# -- transfer to a safe place
echo "`date` - Transfering backup directory to a remote location"
rsync -re ssh $DIR_WRK $STORAGE

echo "`date` - Cleaning up backup directory"
rm -rf $DIR_WRK
echo "`date` - Done"

Automatic deployment part 5

Here is the current version of the build script:

<?xml version="1.0" encoding="UTF-8"?>
<project name="my_project" default="deploy">
<property name="config.file" value="builder/build.properties.${dest}" />
<property name="dist.dir" value="${ant.project.name}-r${svn_revision}-b${build_number}" />
<property name="build.dir" value="${workspace}/${dist.dir}" />

<record name="${build.dir}.log" />

<target name="init">
<available file="${config.file}" property="config.exists"/>
<fail message="Config file '${config.file}' not found." unless="config.exists" />
<echo>Config file '${config.file}' exists.</echo>
<property file="${config.file}" />

<available file="${workspace}" property="workspace.exists"/>
<fail message="Workspace '${workspace}' not found." unless="workspace.exists" />
<echo>Workspace '${workspace}' exists.</echo>
<delete dir="${build.dir}" />
<mkdir dir="${build.dir}" />
</target>

<target name="prepare" depends="init">
<copy todir="${build.dir}">
<fileset dir=".">
<exclude name="**/.*" />
<exclude name="build*" />
<exclude name="bkp/" />
<exclude name="builder/" />
<exclude name="cache/" />
<exclude name="scripts/" />
<exclude name="sources/" />
</fileset>
</copy>
<copy file="builder/.htaccess.${dest}" tofile="${build.dir}/.htaccess" verbose="true" />
</target>

<target name="transfer" depends="prepare">
<scp todir="${dest_user}@${dest_host}:${dest_path}/" password="${dest_passwd}" trust="true">
<fileset dir="${workspace}">
<include name="${dist.dir}/"/>
</fileset>
</scp>
</target>

<target name="remote-deploy" depends="transfer">
<sshexec
host="${dest_host}"
username="${dest_user}"
password="${dest_passwd}"
command="
cd ${dest_path};
rm current;
ln -s ${dist.dir} current;"
/>
</target>

<target name="deploy" depends="remote-deploy">
</target>
<project>

The script needs to be invoked like this:
ant -Dworkspace=/tmp -Ddest=dev -Dsvn_revision=123 -Dbuild_number=123

Automatic deployment part 4

The standard installation of Hudson or Ubuntu does not have all the optional tasks and dependencies. In manage Hudson - configure - I had to create another installation of ant. Then I changed configuration of the project to use that specific installation of the Ant.

On next build Hudson downloaded and installed Ant. ...Which again didn't contain optional tasks. I used command line to get to the Hudson installation directory and in there to the new Ant installation. over there as a hudson user (this is important) I ram ./bin/ant -f fetch.xml -Ddest=system

Then I reloaded configuration of Hudson through manage configuration. Since then I was able to use optional tasks as scp and sshexec.

Automatic deployment part 3

I finished writing the Ant build script for first of our PHP projects. I made plenty of tests. At the end script consists of four parts. I send several parameters to Ant. The most important is destination - devel, test, prod. For each destination I have separate configuration file. The Ant build script then does following:

1. Init - prepares build directory structure.
2. Prepare - copies all files to build directory. In this step an .htaccess for proper destination is copied at proper place in the build directory structure.
3. Transfer - transfers whole build directory to the destination.
4. Remote-deploy - executes series of commands over sshexec. Most of them creates links to newly uploaded build.

So far it works. I'd like to figure out dependencies on other projects as this project itself needs our framework....

The biggest problem for me was to figure out why copy target did not exclude certain directories. But after few hours I figured that out also.