How to extract key phrases from a log file using a batch file.

60 pts.
Tags:
Performance testing
How can I  extract key phrases from a log file using a batch file.  I am new to this. I need to know how to extract key phrases from a log file using a batch file. I need to create the batch file first to extract the phrases from the log file.  I don't need to pull the whole log file.  Just key words and dump them into a database or excel work sheet or notepad.

Software/Hardware used:
Windows XP, HP 530 desktop

Answer Wiki

Thanks. We'll let you know when a new response is added.

You wouldn’t be able to do that in a batch file, but a basic program can do that very quickly.

Discuss This Question: 14  Replies

 
There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when members answer or reply to this question.

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
  • BobBeechey
    No, if I understand the question correctly, the following command will store exactly what is required in a file "logstrip.txt"
    find /n /i "text required" yourlog.log > logstrip.txt
    750 pointsBadges:
    report
  • carlosdl
    I think BobBeechey's suggestion will do it. That command will find all occurrences of the searched text, and will output the complete line to the resultant file. Additionally, the line number from the original file will be included in the result (because of the "/n" parameter). If you want it to be case-sensitive, remove the "/i" parameter.
    70,220 pointsBadges:
    report
  • Stretch21
    Great.
    60 pointsBadges:
    report
  • Stretch21
    But like I said, Iam new to this. The logstrip.txt”-do I create it and save it as a bat file ? The I run it on the PC?
    60 pointsBadges:
    report
  • carlosdl
    In the example provided logstrip.txt is the resultant file containing the lines where "text required" was found. You would have to create a new file, put the find command in it (with all its parameters as shown in the example above), and save it as a BAT file. You could parameterize your bat script. For example, you could use the find command this way:
    find /n /i %1 %2 > %3
    Then, when you execute this bat, you would have to send the appropriate parameters. For example, assuming that you name your bat file "test.bat" and you want to obtain the same results as the original example, then you would execute your bat file this way:
    test "text required" yourlog.log logstrip.txt
    But it really doesn't make much sense to me creating a bat script to execute just one command.
    70,220 pointsBadges:
    report
  • Stretch21
    Ok I'm still not sure.....Let say I have a log........let's call it status.log. and I need to pull a word like "service" from this log, how would I do this?
    60 pointsBadges:
    report
  • carlosdl
    find /n /i "service" status.log > results.txt
    70,220 pointsBadges:
    report
  • Chippy088
    Can't add much more to this, you guys have it is pretty well covered. However Stretch21, you did say 'phrases' so if you have several queries to do on the same text file, but with different words to find, then a batch file will do it better. However you will have to call the batch file with the word/phrase wanted each time. If you want to put all the results into the same file, then I'd suggest creating a batch file with either of the lines that Carlosdl or BobBeechey gave you, changed slightly. Where the line has a single > in it, change it to >> This tells DOS to append each search to the file. A single > means DOS will overwrite the previous contents of the file with the current search result. For example (using Carlosdl's line - find /n /i %1 %2 > %3) You could replace %3 with a permanent results file, including a path if wanted, and you would always know to which file the search result was writen. and if you used the same parametersvariables for the search each time, the calling method would be simpler as you would have less typing to do. Only 3 parameters needed. batchfile, text to find, which file to search. I've used a batch file this way for searching a series of installation, error or logs for text, for searching all log files I use *.log as the file to search parameter. Saves typing if you are a sysadmin looking for the same text in a series of files. Hope I haven't confused the issue. There are great tutorials on the web if you want to delve into batch files further.
    4,625 pointsBadges:
    report
  • Stretch21
    Ok great.but now don't I have to put in the location of where the log file is ? Because when I lrun this batch file, it comes up as a blank text.
    60 pointsBadges:
    report
  • carlosdl
    "don’t I have to put in the location of where the log file is ?" I guess you can try, and find the answer. From the command prompt, type this to get more help.
    find /?
    70,220 pointsBadges:
    report
  • Stretch21
    I'm not sure you understand. Now when I enter find /?, it flashes but nothing happens. When I run the bach file, it creates a text file but nothing is in it. I'm so confused.
    60 pointsBadges:
    report
  • carlosdl
    Find is a command line tool. If you want to see its results, you have to run it from a command prompt. This is very basic. I would recommend to look for resources to learn the basics, as you will most likely have to deal with this kind of things in the future. Google is your friend ;) Good luck.
    70,220 pointsBadges:
    report
  • RegMel
    Another question. If you are writing a batch file, and you are creating a program to allow users to create their own profiles then saving their info to a .dat file, how can I get this to see if this is the right phrase or make it goto another place in the file if so.    here is my program so far.

    set/p password=What is your password?: 
    find /n /i "%password%" password.dat > goto chat

    how could i make it to where the goto part works. (btw the .dat file is in the same folder)
    10 pointsBadges:
    report
  • jakrabit

    actually:

    create a parse.cmd file and use the following code:

    @echo off
    REM **** Move log files that have not been modified in the past 24 hours to the processing folder
    robocopy C:\inetpub\logs\LogFiles\W3SVC1909006052\ C:\inetpub\logs\LogFiles\processing /MOV /MINAGE:1
    
    REM **** place all lines not excluded (exclusions found in the exclude.txt) in the parsed.log file
    rem cd C:\inetpub\logs\LogFiles\processing
    for %%i in (C:\inetpub\logs\LogFiles\processing\*.log) do findstr /v /i /g:"C:\inetpub\logs\exclude.txt" "%%i" >>C:\inetpub\logs\LogFiles\processing\parsed.log
    
    REM **** Move parsed.log and delete the processed logs
    robocopy C:\inetpub\logs\LogFiles\processing C:\inetpub\logs\LogFiles\ParsedLogs parsed.log /MOV
    robocopy C:\inetpub\logs\LogFiles\processing C:\inetpub\logs\LogFiles\processed *.log /MOV
    
    REM **** determine current date, and subtract 1 (accounting for 30/31/29 day months, Leap Year)
    set yyyy=
    
    set $tok=1-3
    for /f "tokens=1 delims=.:/-, " %%u in ('date /t') do set $d1=%%u
    if "%$d1:~0,1%" GTR "9" set $tok=2-4
    for /f "tokens=%$tok% delims=.:/-, " %%u in ('date /t') do (
    for /f "skip=1 tokens=2-4 delims=/-,()." %%x in ('echo.^|date') do (
    set %%x=%%u
    set %%y=%%v
    set %%z=%%w
    set $d1=
    set $tok=))
    
    if "%yyyy%"=="" set yyyy=%yy%
    if /I %yyyy% LSS 100 set /A yyyy=2000 + 1%yyyy% - 100
    
    set CurDate=%mm%/%dd%/%yyyy%
    set dayCnt=%1
    
    if "%dayCnt%"=="" set dayCnt=1
    
    REM Substract your days here
    set /A dd=1%dd% - 100 - %dayCnt%
    set /A mm=1%mm% - 100
    
    :CHKDAY
    if /I %dd% GTR 0 goto DONE
    set /A mm=%mm% - 1
    if /I %mm% GTR 0 goto ADJUSTDAY
    set /A mm=12
    set /A yyyy=%yyyy% - 1
    
    :ADJUSTDAY
    if %mm%==1 goto SET31
    if %mm%==2 goto LEAPCHK
    if %mm%==3 goto SET31
    if %mm%==4 goto SET30
    if %mm%==5 goto SET31
    if %mm%==6 goto SET30
    if %mm%==7 goto SET31
    if %mm%==8 goto SET31
    if %mm%==9 goto SET30
    if %mm%==10 goto SET31
    if %mm%==11 goto SET30
    REM ** Month 12 falls through
    
    :SET31
    set /A dd=31 + %dd%
    goto CHKDAY
    
    :SET30
    set /A dd=30 + %dd%
    goto CHKDAY
    
    :LEAPCHK
    set /A tt=%yyyy% %% 4
    if not %tt%==0 goto SET28
    set /A tt=%yyyy% %% 100
    if not %tt%==0 goto SET29
    set /A tt=%yyyy% %% 400
    if %tt%==0 goto SET29
    
    :SET28
    set /A dd=28 + %dd%
    goto CHKDAY
    
    :SET29
    set /A dd=29 + %dd%
    goto CHKDAY
    
    :DONE
    if /I %mm% LSS 10 set mm=0%mm%
    if /I %dd% LSS 10 set dd=0%dd%
    
    REM **** Rename parsed.log to Previous Date (yyyymmdd.log)
    rename C:\inetpub\logs\LogFiles\ParsedLogs\parsed.log u_ex%yyyy%%mm%%dd%.log
    
    pay attention to file paths, etc. this i use for log parsing. i have a .txt file called "Exclude.txt" that i can easily modify to add lines to exclude
    10 pointsBadges:
    report

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to:

To follow this tag...

There was an error processing your information. Please try again later.

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Thanks! We'll email you when relevant content is added and updated.

Following