It was in the dark ages of the Reagan era when I logged my first encounter with the File Transfer Protocol. I was involved in some nefarious video game-trading ring of the innocently naive variety, and a friend of mine had that hot new copy of Vladivostok Putting Challenge that I wanted oh so badly. The only problem was that he lived all the way on the other side of Orange County. Being too young to drive, neither of us had access to a car, and snail mail was just too darned slow. I wanted to lay down sloping fifteen-footers and rub it in the face of the Reds today, not in a week. “No problem,” my friend assured me. “Just log in to my server and grab it with FTP.”
“FTP? What is that?” I asked. My friend let out an Oscar-worthy sigh and gave me the quick run down of the Internet protocol that would forever transform my life.
Last I heard, my friend is collecting Galaxie 500s in Michigan, so calling him up with your FTP questions isn’t really an option anymore. Lucky for you, I’m here to give you a holier-than-thou sigh of my own and send you down the dharma path. For those of you who are entirely new to this whole game, we’ll start with a basic primer on the most comely of acronyms.
What is FTP?
FTP is short for File Transfer Protocol. A protocol is a set of rules that networked computers use to talk to one another. And FTP is the language that computers on a TCP/IP network (such as the internet) use to transfer files to and from each other.
You’ve probably encountered FTP out there on the net already. Ever downloaded a fresh nightly build of Firefox or grabbed MP3s from some kid’s personal server in Sweden? Then you have probably used FTP without even knowing it. Today’s web browsers allow you to download files via FTP from within the browser window. It’s very convenient, and it’s great for those times you need to download a file or two, but the browser-download method does not offer much in the realm of flexibility. You can’t upload, force a particular transfer mode, or ask the server any questions. And don’t even get me started on the security issue. But if you are doing any sort of web development, you need all this functionality.
The best way to pursue file transfers is with a bona fide FTP client. You use an FTP client to log into an FTP server, navigate the server’s folder structure, and exchange files. That’s pretty much all FTP clients do — unlike Web browsers, FTP clients are tailor-made for such duties. The latest full-featured web editors like Dreamweaver and RapidWeaver have FTP fuctions built-in. As far as stand-alone FTP clients go, there are literally hundreds out there. Some are free, some are more expensive than a night on the town.
We’ll take an in-depth look at the joy of using an FTP client in the pages that follow, but first, let’s look at exactly how an FTP session works.
Logging In
Connecting to an FTP server is very similar to connecting to just about any other server on the Web. When you log in to your Hotmail account or a secure shopping cart system (such as the one at Amazon.com), you have to provide a server address, a user name, and a password before you can exchange information with the server.
Let’s take a look at an example login set for an FTP server.
site:ftp.fakesite.org login:mcalore pass:h4x0r4lyfe port:21
The first line in the example above, the site address, is simply the address of the FTP server that I’m connecting to. The “ftp” in the prefix of the URL identifies it as an FTP server. Pretty straightforward so far. The second and third lines are my user name and password, respectively. In this particular case, I have already had the server administrator set up a login account for me on the server. Even if I didn’t have a login account, I still might be able to gain access to the server by logging in anonymously. We’ll go into that more a little later.
The last line of the example login is the port number that I’m connecting to on the server. A port is best envisioned as a little door on the server. Most servers on the web have a multitude of such doors, each with its own purpose. Each port number is dedicated to a corresponding application on the server, and the traffic intended for an application (such as an FTP server or web server) passes through the application’s dedicated door. Each port handles a specific type of transaction between the server and client. For example, the FTP protocol is handled by ports 20 and 21 by default. HTTP (the protocol that’s used to serve web pages to a browser) requests are handled by port 80.
Often, the port number will be attached to the end of the server’s Web address, separated by a colon, like this:
ftp.fakesite.org:21
It isn’t important that you know every single TCP/IP port for each internet application out there, but you have to be able to recognize a port number when you see one. This becomes especially critical when you are connecting to servers that route FTP connections to other, more obscure ports.
Note: This quandry brings up the sometimes-confusing issue of passive FTP versus active FTP, two different modes of data transfer between a client and a server. Which one is being used depends on how the client and server firewalls are set up. Since this issue is beyond the scope of this article, I will point you to a detailed discussion of active versus passive FTP over at Slacksite.com. So don’t be alarmed if you notice that your FTP connection is jumping to port 1013 all of sudden — that’s just your data being routed via passive FTP.
Anonymous FTP
Many public servers on the Internet allow users to log in and download files via FTP by connecting anonymously. This is a very common practice in the world of open-source and freely distributed software.
When you connect anonymously to an FTP server, you’re not really anonymous in the truest sense. Most of the time, you are required to enter “anonymous” as your user name and your email address as your password. This is a gesture of courtesy toward the server administrators so that they can get some idea of who’s logging in and downloading their files. Even if you are connecting anonymously and you aren’t required to give any personal information during the connection process, any sysadmin worth her salt is logging your session. This means that if you do anything nasty or illegal, the server administrator has your number (quite literally, your host domain and IP address). You can be put on probation, blocked, or prosecuted. So, here’s a tip to remember:anonymous is never really anonymous as far as anonymous FTP is concerned.
Check Out the Logs on That Server!
Sometimes, the easiest way to wrap your head around a new concept is to dig in and examine the bare bones. In this case, we can learn a lot by simply reading the log from an FTP session.
All FTP servers and clients will log FTP sessions. A log is basically a text record of all of the activity that goes on during a session. Your client should save your FTP logs locally, so you can open them up in a text editor and have a peek at all the nitty-gritty FTP commands and transfer hoo-hah.
Let’s take a look at a sample FTP log. I’m logging in to my server with the user name “mcalore”. This log simply shows my client connecting to the server, logging in, and asking for the list of files inside the default directory on the server. That’s all that’s going on here!
Status:Connecting to ftp.fakesite.org ... Status:Connected with ftp.fakesite.org. Response:220 ProFTPD 1.2.4 Server (ProFTPD) [109.41.xx.xxx] Command:USER mcalore Response:331 Password required for mcalore. Command:PASS ********** Response:230 User mcalore logged in. Status:Connected Status:Retrieving directory listing... Command:PWD Response:257 "/users/mcalore" is current directory. Command:LIST Response:150 Opening ASCII mode data connection for file list. Response:226 Transfer complete. Status:Directory listing successful
Starting at the beginning of the log, you can see that my FTP client wants to connect to the server at fakesite.org. My client knows that the server is active and waiting, so it sends the first command, which is “USER mcalore”. The server says, “OK, but I need a password”. My FTP client sends the next command, which is “PASS ********”. This is all pretty simple — my client sends a Command, and the server sends back a Response. The communication goes on like this during the entire session. It looks like my client and my server are becoming fast friends, eh?
There are several commands that a client can send to an FTP server to find out information, switch directories, or request files. In the sample log above, the PWD command — which should be familiar to Unix users — is used to find out the current directory you’re working in on the server. You can view a full list of FTP commands on Columbia University’s Academic Information Systems site. Even though your client will take care of all the server commands for you, a good Monkey should be able to look at the logs and understand what’s going on.
At this point, you’re just about ready to grab your own FTP client and start experimenting. However, by showing you the sample FTP log, I opened a can of worms that we haven’t discussed yet:transfer modes.
ASCII versus Binary
Let’s go back to the last four lines of the sample log that we examined in the previous section.
Command:LIST Response:150 Opening ASCII mode data connection for file list. Response:226 Transfer complete. Status:Directory listing successful
See that server response that reads “Opening ASCII mode data connection for file list”? That’s very important. File transfers over FTP take two different forms, ASCII and binary.
ASCII, otherwise known as American Standard Code for Information Interchange, is a set of 128 symbols that any computer in the world can display. You may be familiar with ASCII text, ASCII art, and ASCII files. Any file that is made up of text or text-based components, such as an HTML file, text file, or PostScript file, is an ASCII file. Binary files, on the other hand, are structured differently, and therefore require a different style of transfer. Images, applications, and algorithmically generated packages such as .zip and .tar files are all binary file types.
Just about every FTP client out there will automatically detect which transfer mode is required and treat your uploads and downloads accordingly. Most clients run in binary transfer mode by default, only using ASCII mode when it’s absolutely needed. This is because both ASCII and binary files can be sent in binary mode with no problems, but sending a binary file in ASCII mode will corrupt the binary file’s structure. ASCII mode makes for faster transfers, but if you accidentally upload a Microsoft Word file or JPG image in ASCII mode, your file will hit the server dead on arrival. If there’s any question, transfer in binary mode.
Of course, we wouldn’t have such a handy rule without one exception:CGI scripts. You must use ASCII mode when sending CGI scripts. They just won’t work otherwise! Luckily, any files ending in .pl or .cgi are usually recognized as CGI scripts by your FTP client.
Quickly, here are some guidelines:
These file types are safe for ASCII transfer
- Text files
- HTML files
- CGI scripts
These file types must be transferred in binary mode
- Images
- Applications
- .zip, .sit or .tar packages
- Proprietary file formats such as .doc, .xls, .fla, .swf
- Anything that’s not made entirely of text characters!
Now that you possess basic knowledge of the File Transfer Protocol and its many gifts, how about digging in and actually trying out some clients? Thought I’d never ask, huh?
Giants Among Clients
Most FTP clients in the less-than-US$40 range have all of the basic features that you’ll need to become a power FTP user. Even so, it may take a short period of trial and error before you find an FTP client that you totally jive with. When choosing an FTP client, you should aim for an application which:
- Has an easy to use interface that clearly displays both the local (your computer) and remote (the FTP server) locations
- Has a tool to manage multiple login accounts on multiple servers
- Logs your FTP sessions
- Provides a way to select between active and passive FTP when connecting
- Auto-detects or warns you when ASCII mode is required
- Allows you to set folder and file permissions on the server (essential when installing a server-side application such as Movable Type)
- Allows anonymous FTP
Windows Clients
An easy-to-use and entirely free application called FileZilla is a great place to start. It was created by an open source community headed by Tim Kosse, and even though the user interface is a little intimidating, FileZilla has all of the features you’ll need to get cracking.
CuteFTP, made by GlobalSCAPE, Inc., costs $40 per user license.
Another crowd favorite in the world of Windows FTP clients is WS_FTP Pro by Ipswitch. Also $40 per user license, WS_FTP offers a clean interface and connection management tools.
Mac OS X Clients
If you’re using a Macintosh, most people point you toward Fetch Softworks’ ever-popular Fetch client. It’s simple, it’s easy, and it costs $25 per user license.
A personal favorite client for the Mac is Panic’s Transmit. It has the cleanest user experience I’ve ever encountered in an FTP client. Transmit costs $24.95 per user license.
An excellent free FTP client for Mac OS X is Fugu. The user interface is a tad spare, but Fugu supports ultra-secure OpenSSH SFTP encryption.
The free FileZilla client is also available for the Mac.
Linux Clients
Linux users who want to try FileZilla can grab it through their system’s package manager or compile it themselves using source code.
FTPing with the Browser
As we have seen, the best way to run FTP is through a real FTP client. However sometimes it’s just plain easier to grab files off an FTP server using a web browser. As you’ll notice, when a browser is pointed to an FTP server instead of a web server, the http:// in the address field changes to ftp:// since you’re working with a different protocol, after all. Your browser’s address field will look something like this:
ftp://username:password@ftp.fakesite.org/
When using a browser to connect to an FTP server, you may be asked to log in just like you would if you were connecting with an FTP client. Once inside the server, your user name and the password you give will be reflected in the server path, as shown above. If you logged in anonymously, your username would show up as the word “anonymous” and your email address would serve as the password. Most browsers are set to log into FTP servers anonymously by default.
A web browser can navigate an FTP server much like any other file system. You can click on folder’s name to browse that particular folder and click on files to download them. But even though browser FTPing is nice an convenient, as a general rule use an FTP client whenever possible. Browsers offer minimal functionality and, not surprisingly, minimal security in FTP mode.
Tips & Advice
Be courteous. The people that run public FTP servers are just that:people. They don’t like being hassled by impolite users, or even innocents who haven’t learned how to properly use their FTP clients. Many FTP admins place readme files in the root directory of their server (“root directory” being the main directory you’re dropped into when you log in). The readmes aren’t there to take up space or look pretty, they are there for you to read. Also, administrators sometimes put their user guidelines in the flood of server information that you receive when you first log in. Open up your FTP logs and browse them for rules, requests concerning your behavior, and the sysadmin’s contact info in case anything goes wrong.
Be patient. Some servers are extremely popular, so it may take you a few tries to gain access. If your client allows you to re-try logging in after a server timeout, all the better. Set your re-tries to one minute or longer, and don’t hammer the FTP server with log in requests. That’s an excellent way to have your IP address banned.
Find the Pub Once you’ve connected to a strange server for the first time, you may have trouble accessing certain folders. That’s because permissions have been set that only allow certain users to get into folders where sensitive files are stored. Unless the readme files point you elsewhere, your best bet is to aim for /PUB, the standard public directory on Unix systems. If you are downloading anything for public use off of a public FTP server, that’s where you’ll find it.
Public Sites You Can Visit
Speaking of public FTP servers, I have a few favorites that I’d like to pass along. There’s some wonderful software available on the Free Software Foundation servers. You can download academic Linux builds from the UK Mirror Service, or find out when your Uncle Jeb emigrated to West Virginia by scrounging up genealogical data. If you have a spare SGI machine lying around that needs some software, the University of California at Santa Cruz has its own directory of FTP sites carrying public domain software. If your girlfriend is an astronomy buff like mine, go grab some space research materials and impress her with your Carl Sagan impersonation.
The world of FTP has billions and billions of practical applications, and now you’re ready to start exploring. Why the blank stares? C’mon, this is the kind of stuff I do for fun. Seriously!
