The tips here are meant as quick-and-dirty solutions, more for the learning experience. It is probably enough, if you are the sole developer, and it's a small project without any deployment complexities (like deploying to multiple servers for load balancing, or updating the database, emptying caches, and so on). If you are seriously considering automating your deployment process, there are already some great tools for that, like Phing, or Maven. They will do the job in a much more secure and error resilient way. I have no experience with them yet, but it's on my "try it, and blog it" list.
I'm assuming you are using some kind of version control. If you don't, well, I'm not even sure what arcane methods you are using now. Still, read on, maybe you can learn something new. I will be providing the examples for hg, since that's what I mainly use, but I'd really appreciate it if you could leave the commands for the other VCSs in the comments.
Let's look at a usual scenario. You have just finished fixing some bugs on an existing site, tested them, and you are ready to transfer the files to production. Now, instead of firing up your ftp/sftp/other client, let's look at how we can automate this process. There are two things, and only two things that we want to do:
- Upload the files that have changed
- Delete the files that we have deleted locally
No automatic rollbacks if you screw up, no permission changing on the files, no cache cleaning, no database updates, no nothing. I want to keep it simple, but of course you can always add everything you need.
Uploading files
First task, determine what files have changed, since the last deploy. For this, we need to know what is the current revision that is deployed. If you don't use a system for tracking that, then now is the time to choose one. You can either tag the revisions that are deployed, with tags like '1.0', '1.1', '1.2', and assume that it is always the "highest" version, that is deployed.
Another option is to keep a seperate branch, called "stable", and always merge into that branch what you want to deploy, so it is always the latest revision in stable, that is currently deployed. I prefer this way.
Don't go being simple on it and "just keeping it in mind". You will eventually lose track, so just pick one. Both methods require self discipline, if you deploy a revision, and forget to tag it/merge it to stable, you can run into problems the next time. Also, have backups at hand, because you will probably mess up the first few times.
Determine what files have changed
We will parse the output of our VCS's status command, and build a script on that. Let's assume we are using the second (seperate stable branch) approach. Merge the changes into the stable branch:
hg update stable hg merge default hg commit -m "Merging default"
Now, we can look at the logs, to see the id of the last deployed revision, and the id of the one that we want to deploy. Let's assume that the id of the last deploy was 'abc123', and the revision that we want to deploy is 'xyz123'. We need to see what files have been added, changed, and removed:
hg status -mard --rev abc123:xyz123 # Shows: # M index.php # A admin.php # R info.php
M stands for modified, A stands for added, and R stands for removed. This is the output that we want to parse.
Upload the files
This step varies, depending on the kind of access you have to the server. I'm going to cover SSH/SFTP, and FTP. If you have both, use SSH. If you only have FTP, ask for SSH. Bang on the desk of your boss/sysadmin/whoever is in charge for that, until you get it.
FTP
You will need a CLI FTP client, that can accept commands from STDIN[ref]Standard input.[/ref]. We are going to use the file list generated from the command above, transform it to FTP commands in a loop, then feed it to the FTP client through STDIN. Feels like duct taping things together? Good, this is The Unix Way, at it's finest. My example will use the yafc FTP client, but it's easy to hack it together for a different one.
#!/bin/bash REVFROM="$1" REVTO="$2" USER= PASS= HOST= WEBROOT= if [ -z "$REVFROM" -o -z "$REVTO" ]; then echo "Usage: $0 revfrom revto" exit 1 fi echo "open $USER:$PASS@$HOST cd $WEBROOT/ lcd $PWD" while read -rd $'\0' FILE; do echo put -fo $FILE $FILE; done < <(hg st -man0 --rev $REVFROM:$REVTO) while read -rd $'\0' FILE; do echo rm -f $FILE; done < <(hg st -rdn0 --rev $REVFROM:$REVTO)
Save this as ftpdeploy.sh, and fill in your user/pass/host/webroot. WEBROOT is the directory that is your document root (where your index.html is stored), you can use '.' if it's the root directory of your FTP account. You will always have to invoke this from the root of your project's directory (more precisely, the directory that is the root of your VCS repository). Don't forget this, or bad things will happen.
The first three lines open a connection to the remote server, cd into the webroot, and change the local dir to the project's dir. The read loop will process each filename, and tack either "put -fo", or "rm -f" before each filename. The switches in hg -man0 mean, that we only want the changed, and added files, without status indicators (so, only the filenames), and they should be seperated by NULL characters. The last step is needed, so we don't slip up on idiotic characters[ref]Filenames in Linux can contain each, and any character, except NULLs. This a point where I think Windows actually made the wiser choice, by constricting the character set.[/ref] in file names. The hg -rd0 loop does the same for deleted files. The read -rd '\0' command splits each line on the aforementioned NULL character.
In the end, the output that the script will generate look like this:
open user:pass@host cd webroot lcd project put -fo index.php index.php put -fo admin.php admin.php rm -f forum.php exit
First, chmod u+x it. Then, invoke it with ./ftpdeploy abc123 xyz123, and see if it generates a similar output. If it looks good, feed it to yafc: ./ftpdeploy abc123 xyz123 | yafc.
Keep in mind, that this has a small drawback: If you are using a VCS that does not track empty directories, like hg, those will never be deleted by this script. I'm almost 100% sure there is no way to ask an FTP server if a directory is empty, so we can't automate that. Okay, we could, by issuing a dir command for each deleted file, and parsing the output with another shell script, but this exercise is left to the reader.
SSH
Reasons why this is so much better:
- transfering files over FTP is orders of magnitude slower because of the "chat" on the control connection
- with SSH, you can transport all the files as a single file
- you get compression with bzip2 (or your preferred method)
- inherently secure
You must have SFTP enabled on the remote side, and tar installed. We will compress the files with tar+bzip2, and transfer it through SSH, and extract it on the remote. After that, we will collect the removed files, and issue rms for them.
#!/bin/bash REVFROM="$1" REVTO="$2" HOST=user@127.0.0.1 PROJECTDIR=/tmp/project if [ -z "$REVFROM" -o -z "$REVTO" ]; then echo "Usage: $0 revfrom revto" exit 1 fi while read -rd $'\0' FILE; do echo rm -f $FILE; done < <(hg st -rd0 --rev $REVFROM:$REVTO) | ssh $HOST tar -cjf - $(hg st -man --rev $REVFROM:$REVTO) | ssh $HOST "tar -xjf - -C $PROJECTDIR"
Change the USER= line, and save it as sshdeploy.sh, and call it with ./sshdeploy.sh abc123 xyz123
It is a good idea to set up passwordless auth with keys, so you don't have to enter those at each deploy (google it). As you see, this makes two connections to the remote server, one for deploying the new files, and another one to remove the deleted ones. Inefficient, but I'm not sure how this could be handled in one operation (comments are welcome).
Like the ftpdeploy.sh script, this also assumes that you are currently in the project's root directory. Replace 'projectdir' with the folder name on the remote side. And, this also leaves empty directories behind. You could clean this up with find . -type d -empty -delete, but some folders might be actually needed for the application function, even if they are empty, like upload folders, so don't just blindly do that without filtering it first.
Now, clever readers probably noticed that I'm skipping the usual dance with NULL characters when tarring. Indeed, this only works if your files have sane names. It could probably be worked around with the --null, and -T options for tar, but I couldn't get that working. If you happen to solve this, please let me know.
Possible improvements
With SSH, you could even solve the empty directory problem, by looking up the dirname of each deleted file, and checking to see if it's empty. You could also connect through a control socket, so you could spare connecting twice. You can also mount the remote side with SSHFS, and issue simple cp/rm commands. The possibilities are pretty much endless, thanks to The Unix Way of duct taping small programs together, but ask yourself the question if it is really worth it. Your simple scripts will pretty soon become bloated, and as I said in the beginning, it is better to look into a correct deploy system when that happens.