Script to extract files information from ftp server

20 pts.
Tags:
CSV
FTP
FTP Server
Microsoft Excel
Scripts
Hi Wonder if you could help me with the following please. I would like to construct a script (in any language whichever it's easiest) which when run, it will log on to a ftp server using a generic account and password, then list the files in a specific folder that begins with the string "abc". The file names along with their creation date are then to be extracted on a csv file (or similar). Can this be easily done? Many thanks in advance!! Ken

Answer Wiki

Thanks. We'll let you know when a new response is added.

You will need to download the files. FTP does not support launching an extract command on the remote server.

<a href=”http://www.columbia.edu/kermit/ftpscripts.html”>This site</a> shows how to automate getting files from an FTP Server.

*************************

FTP Clients can show the user a directory listing from the remote server, so it can be done.
We do something similar (but without getting the creation date), but we don’t do it with a script, but with a small VB application.

****************************

Expect would work well for this. Check out the following for an introduction.

http://www.gnulamp.com/expect.html

http://expect.nist.gov/

Discuss This Question: 3  Replies

 
There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when members answer or reply to this question.

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
  • Gilly400
    Hi, I don't know if your FTP server has this functionality, but on the AS400 you can use a "dir (DISK" command for example :- dir abc* (DISK which will create a file with a list of the directory contents, which you can then retrieve using an FTP get command. Regards, Martin Gilbert.
    23,730 pointsBadges:
    report
  • HolmanSAP
    One method we use is to have the script execute FTP twice. The first time it makes a connection to the ftp server, it issues the "dir" command and puts the output in a file on your local server. The script then closes the ftp session and checks the contents of the local file for anything that it should be picked up from the remote server. The script then calls ftp a second time to do the actual transfer. For each file transferred, you can then check the size of the local file to the information about the file that was obtained on the first ftp call. If they are not the same, then the file was not transferred successfully. For example: #set a variable with the wildcard of files you are looking for FILEWC="S*" These two variables hold the value for the file name where I hold information from the ftp sessions RFINFO="$LDIR/remote_directory_info.dif" RFINFO2="$LDIR/remote_directory_info2.dif" FTPLOG="$LDIR/ftplog.log" FTPHISTORY="$LDIR/ftp_history.log" # other variables are also used to store the name/address of the ftp server and the connection #information. This makes it easy to modify the script to handle multiple ftp requirements: ftp -nciv > $FTPLOG 2>&1 <<EOT open $FTPSERVER user $RUSER $RPWD cd $RDIR dir $FILEWC $RFINFO EOT FTPSTATUS=$? if [ $FTPSTATUS -ne 0 ] ; then return $FTPSTATUS fi NumToTransfer=`cat $RFINFO | /usr/bin/wc -l ` if [ $NumToTransfer -eq 0 ]; then echo "$CURDATE - No files to transfer" >>$FTPHISTORY else echo "$CURDATE - Files to transfer: $NumToTransfer" >>$FTPHISTORY FLIST=`cat $RFINFO | awk -f $LDIR/awkname.dat` fi #in this code snippet, it builds a list of files to retrieve into a variable named FLIST There is then another piece of our script that actually makes the second connection, for each file to be transferred, and retrieves the files. The variabel ftpmode is used to pass the command to set the transfer mode of either BIN or ASC. for eachfile in $FLIST; do ftp -nicv >$FTPLOG 2>&1 <<EOT open $FTPSERVER user $RUSER $RPWD cd $RDIR $FTPMode dir $eachfile $RFINFO2 get $eachfile quit EOT FTPSTATUS=$? if [ $FTPSTATUS -ne 0 ] ; then return $FTPSTATUS fi # Check the file size that was actually transfered FSIZE=`/usr/bin/ls -l $eachfile | /usr/bin/awk '{print $5}'` RFSIZE=`cat $RFINFO |grep $eachfile | awk -f $LDIR/awksize.dat` RFSIZE2=`cat $RFINFO2 |grep $eachfile | awk -f $LDIR/awksize.dat` if [ $FSIZE -ne $RFSIZE2 ] ; then echo "$CURDATE - FILE $eachfile SIZE NOT EQUAL - $FSIZE - $RFSIZE2" >>$FTPHISTORY echo "File size not equal!! Local: $FSIZE, Remote: $RFSIZE2" >> $FTPLOG FTPINCOMPLETE=1 return fi echo "$CURDATE - File $eachfile received - Size: $FSIZE" >>$FTPHISTORY let NumTransferred=$NumTransferred+1 if [ $NumTransferred -ge $FTPLIMIT ] && [ $FTPLIMIT -ne 0 ] ; then break fi I may have missed documenting some of the variables here, but I hope this helps. Regards, Larry
    20 pointsBadges:
    report
  • Mailtobash
    Hi HolmanSAP What is awkname.dat and awksize.dat ? Can you please help me with this? The FLIST is not being created correctly in my script. Please help.
    10 pointsBadges:
    report

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to:

To follow this tag...

There was an error processing your information. Please try again later.

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Thanks! We'll email you when relevant content is added and updated.

Following