How to Automate FTP

Programming with the File Transport Protocol Macros

© Mark Alexander Bain

Nov 11, 2008
Automating FTP, Mark Alexander Bain
FTP (File Transport Protocol) has been around for a long time, but it's still as useful as it ever was - especially when it comes to automating processes with FTP macros.

Most people's experience of FTP (File Transport Protocol) is through a form based GUI (Graphical User Interface) - the kind of interface that allows a user to drag and drop files from their pc to their web site; however, for the programmer, FTP has much more to offer:

  • FTP can be run from the command line and can, therefore, be used in scripts
  • FTP actually has its own programming language.

Using FTP from the Command Line

The majority of FTP users will be happy to use the graphical interface, but that will not suffice for any programmers or web site owners who want to automate the upload (or download) process - and that's where the command line FTP can be useful.

At it's simplest level the command line FTP works in the same way as the GUI FTP, and the user will:

  • connect to the FTP server using their own user name
  • enter a password
  • upload (or download) selected files

A typical FTP session will look like:

$ ftp ftp.linuxtalk.co.uk
Connected to ftp.linuxtalk.co.uk.
220 FTP server ready
Name (ftp.linuxtalk.co.uk:bainm): linuxtalk
331 Password required for linuxtalk
Password:
230 User linuxtalk logged in
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> put testfile.txt
local: testfile.txt remote: testfile.txt
200 PORT command successful
150 Opening BINARY mode data connection for testfile.txt
226 Transfer complete
ftp>

However, there is a limitation with using FTP in this way - the user's password must be entered manually, preventing the process from being automated; and that's where the .netrc file is useful.

Using the .netrc File

The .netrc file does not exist by default - it needs to be created manually and placed in the user's home directory; it needs to contain the connection details for any FTP accounts to be used:

machine ftp.linuxtalk.co.uk
login linuxtalk
password abcDEF123

The .netrc must always end with a blank line (and a blank line must also be inserted between any FTP account definitions).

Obviously, before continuing, there is also a security risk to be considered here - the user's password is being saved in a text file; however, FTP reduces this risk by only running if the .netrc file has its permissions set to user read only (and unreadable by anyone else); therefore the most important job is to do:

$ chmod 600 .netrc

With the .netrc file in place FTP access becomes very simple:

$ ftp ftp.linuxtalk.co.uk
Connected to ftp.linuxtalk.co.uk.
220 FTP server ready
331 Password required for linuxtalk.co.uk
230 User linuxtalk.co.uk logged in
Remote system type is UNIX.
Using binary mode to transfer files.
ftp>

And now any file processing can be automated:

echo put testfile1.txt | ftp ftp.linuxtalk.co.uk
echo get testfile2.txt | ftp ftp.linuxtalk.co.uk

FTP Macros

The automation process is made much easier by the fact that macros can be used with FTP - they can use any standard FTP command and are defined in the .netrc file by using the macdef keyword:

macdef updateWeb
prompt off
LCD $1
mkdir $2
cd $2
put *
macdef myWebUpdate
$ updateWeb /home/bainm/webfiles webdirectory

In the example above all of the files in a directory are uploaded on to a server (creating a new directory if necessary) - and it's worth noting that one macro can call another (by using the $ symbol), and that the macros can also be called from the command line:

echo "$ myWebUpdate" | ftp ftp.linuxtalk.co.uk

Conclusion

FTP has always been a powerful tool, enabling even the least technically minded person upload and download files with ease; but the .netrc file and macros turn FTP into a useful programming tool - especially when combined with other tools such as cron. FTP may be quite old technology but it's still as vital as it ever was.


The copyright of the article How to Automate FTP in Command Line Programming is owned by Mark Alexander Bain. Permission to republish How to Automate FTP in print or online must be granted by the author in writing.


Automating FTP, Mark Alexander Bain
A Simple FTP Batch File, Mark Alexander Bain
Automatic FTP Upload, Mark Alexander Bain
   


Post this Article to facebook Add this Article to del.icio.us! Digg this Article furl this Article Add this Article to Reddit Add this Article to Technorati Add this Article to Newsvine Add this Article to Windows Live Add this Article to Yahoo Add this Article to StumbleUpon Add this Article to BlinkLists Add this Article to Spurl Add this Article to Google Add this Article to Ask Add this Article to Squidoo

Comments
Sep 27, 2009 1:25 AM
Guest :
Don't bother posting this. I just want to say that I'm bamboozled at how perfect this article is. You people are the modern day equivalent of the Gods on Mt. Olympus.
1 Comment: