By: mailtobash

mailtobash — Thu, 19 Mar 2009 08:51:58 +0000

Hi HolmanSAP
What is awkname.dat and awksize.dat ?
Can you please help me with this?
The FLIST is not being created correctly in my script.

Please help.

By: holmansap

holmansap — Wed, 24 Sep 2008 15:07:42 +0000

One method we use is to have the script execute FTP twice. The first time it makes a connection to the ftp server, it issues the “dir” command and puts the output in a file on your local server. The script then closes the ftp session and checks the contents of the local file for anything that it should be picked up from the remote server. The script then calls ftp a second time to do the actual transfer. For each file transferred, you can then check the size of the local file to the information about the file that was obtained on the first ftp call. If they are not the same, then the file was not transferred successfully.
For example:
#set a variable with the wildcard of files you are looking for
FILEWC=”S*”
These two variables hold the value for the file name where I hold information from the ftp sessions
RFINFO=”$LDIR/remote_directory_info.dif”
RFINFO2=”$LDIR/remote_directory_info2.dif”
FTPLOG=”$LDIR/ftplog.log”
FTPHISTORY=”$LDIR/ftp_history.log”
# other variables are also used to store the name/address of the ftp server and the connection #information. This makes it easy to modify the script to handle multiple ftp requirements:

ftp -nciv > $FTPLOG 2>&1 < open $FTPSERVER
user $RUSER $RPWD
cd $RDIR
dir $FILEWC $RFINFO
EOT
FTPSTATUS=$?
if [ $FTPSTATUS -ne 0 ] ; then
return $FTPSTATUS
fi
NumToTransfer=`cat $RFINFO | /usr/bin/wc -l `
if [ $NumToTransfer -eq 0 ]; then
echo “$CURDATE – No files to transfer” >>$FTPHISTORY
else
echo “$CURDATE – Files to transfer: $NumToTransfer” >>$FTPHISTORY
FLIST=`cat $RFINFO | awk -f $LDIR/awkname.dat`
fi

#in this code snippet, it builds a list of files to retrieve into a variable named FLIST

There is then another piece of our script that actually makes the second connection, for each file to be transferred, and retrieves the files. The variabel ftpmode is used to pass the command to set the transfer mode of either BIN or ASC.

for eachfile in $FLIST;
do
ftp -nicv >$FTPLOG 2>&1 < open $FTPSERVER
user $RUSER $RPWD
cd $RDIR
$FTPMode
dir $eachfile $RFINFO2
get $eachfile
quit
EOT
FTPSTATUS=$?
if [ $FTPSTATUS -ne 0 ] ; then
return $FTPSTATUS
fi
# Check the file size that was actually transfered
FSIZE=`/usr/bin/ls -l $eachfile | /usr/bin/awk ‘{print $5}’`
RFSIZE=`cat $RFINFO |grep $eachfile | awk -f $LDIR/awksize.dat`
RFSIZE2=`cat $RFINFO2 |grep $eachfile | awk -f $LDIR/awksize.dat`
if [ $FSIZE -ne $RFSIZE2 ] ; then
echo “$CURDATE – FILE $eachfile SIZE NOT EQUAL – $FSIZE – $RFSIZE2″ >>$FTPHISTORY
echo “File size not equal!! Local: $FSIZE, Remote: $RFSIZE2″ >> $FTPLOG
FTPINCOMPLETE=1
return
fi
echo “$CURDATE – File $eachfile received – Size: $FSIZE” >>$FTPHISTORY
let NumTransferred=$NumTransferred+1
if [ $NumTransferred -ge $FTPLIMIT ] && [ $FTPLIMIT -ne 0 ] ; then
break
fi

I may have missed documenting some of the variables here, but I hope this helps.
Regards,

Larry

By: gilly400

gilly400 — Tue, 23 Sep 2008 14:38:37 +0000

Hi,

I don’t know if your FTP server has this functionality, but on the AS400 you can use a “dir (DISK” command for example :-

dir abc* (DISK

which will create a file with a list of the directory contents, which you can then retrieve using an FTP get command.

Regards,

Martin Gilbert.

Comments on: Script to extract files information from ftp server

By: mailtobash

By: holmansap

By: gilly400