http://www.swiftmailer.org/wikidocs/v3/uploading

The directory you will want to upload is "lib". Be sure to keep the "lib" directory in-tact when you upload it. It is however safe to rename the directory to something more suitable such as "swift".

Swift does not need to be inside the web root, but it does need to be in a location which you are able to refer to in your PHP scripts which will use it (i.e. PHP needs to be able to see it, but your end-users don't).