How to filter out the line which counting up from last line from a text file thru batch command?

Question

I've wrote a batch script that to capture a specific location string from a text file, for example:

The text file contains:

Copyright Version 5.39
Network activity progressing...


Thread  Time(s) Throughput(KB/s) Avg B / Compl
======  ======= ================ =============
     0  600.088        73245.829     45502.632
     1  594.574        84312.667     44974.835
     2  594.569        62862.184     44547.486
     3  599.665        66407.148     45056.270
     4  600.633        61846.742     44741.495
     5  594.569        45967.918     46745.891
     6  594.937        72678.901     45115.861
     7  593.981        86081.374     45148.288
     8  593.975        35448.661     44118.093
     9  602.451        64144.439     44708.118
    10  599.760        26404.342     53411.916
    11  594.569        64959.044     44525.327
    12  594.564        63125.969     44512.966
    13  602.999        71045.335     45266.114
    14  599.719        19782.849     54192.569
    15  594.574        61670.399     44341.198
    16  599.681        71804.247     44954.492
    17  593.980        21731.533     43903.776
    18  593.979        22436.796     43919.327
    19  599.748        21296.983     53880.446


#####  Totals:  #####


   Bytes(MEG)    realtime(s) Avg Frame Size Throughput(MB/s)
================ =========== ============== ================
   639903.106318     600.002       4289.297         1066.502


Throughput(Buffers/s) Cycles/Byte       Buffers
===================== =========== =============
            17064.032      41.043  10238449.701


DPCs(count/s) Pkts(num/DPC)   Intr(count/s) Pkts(num/intr)
============= ============= =============== ==============
  3081479.450         0.085     2302796.649          0.113


Packets Sent Packets Received Retransmits Errors Avg. CPU %
============ ================ =========== ====== ==========
    66199455        156432871           4      0     99.998

There are total 58 lines in the text file, and I will need to get Throughput(MB/s) value, which allocated in line 43, and the 4th string, from above example it is 1066.502.

So, I created a batch script to filter out the number:

set "lom1="
for /f "skip=48 delims=" %%i in LOM_1.log) do if not defined lom1 set "lom1=%%i"
echo !lom1! >> temp1.txt
for /f "tokens=4" %%j in temp1.txt do echo %%j >> result.log

But, now the source text got errors and it increases random lines, for example:

Copyright Version 5.39
Network activity progressing...
ERROR: WaitForWorkerThreads failed: WaitForMultipleObjects returned an unexpected value

ERROR: DoWork failed: WaitForWorkerThreads(threads_finished) timed out
ERROR: StartSenderReceiver in thread: 9 failed: closesocket, GetLastError: 10093 - Either the application has not called WSAStartup, or WSAStartup failed.

ERROR: StartSenderReceiver in thread: 13 failed: closesocket, GetLastError: 10093 - Either the application has not called WSAStartup, or WSAStartup failed.

ERROR: StartSenderReceiver in thread: 16 failed: closesocket, GetLastError: 10093 - Either the application has not called WSAStartup, or WSAStartup failed.



Thread  Time(s) Throughput(KB/s) Avg B / Compl
======  ======= ================ =============
     0  600.088        73245.829     45502.632
     1  594.574        84312.667     44974.835
     2  594.569        62862.184     44547.486
     3  599.665        66407.148     45056.270
     4  600.633        61846.742     44741.495
     5  594.569        45967.918     46745.891
     6  594.937        72678.901     45115.861
     7  593.981        86081.374     45148.288
     8  593.975        35448.661     44118.093
     9  602.451        64144.439     44708.118
    10  599.760        26404.342     53411.916
    11  594.569        64959.044     44525.327
    12  594.564        63125.969     44512.966
    13  602.999        71045.335     45266.114
    14  599.719        19782.849     54192.569
    15  594.574        61670.399     44341.198
    16  599.681        71804.247     44954.492
    17  593.980        21731.533     43903.776
    18  593.979        22436.796     43919.327
    19  599.748        21296.983     53880.446


#####  Totals:  #####


   Bytes(MEG)    realtime(s) Avg Frame Size Throughput(MB/s)
================ =========== ============== ================
   639903.106318     600.002       4289.297         1066.502


Throughput(Buffers/s) Cycles/Byte       Buffers
===================== =========== =============
            17064.032      41.043  10238449.701


DPCs(count/s) Pkts(num/DPC)   Intr(count/s) Pkts(num/intr)
============= ============= =============== ==============
  3081479.450         0.085     2302796.649          0.113


Packets Sent Packets Received Retransmits Errors Avg. CPU %
============ ================ =========== ====== ==========
    66199455        156432871           4      0     99.998

You can see that there are additional ERROR lines in the text file, that means I cannot still use the for /f "skip=48" command to filter the right line out, so, I'd like to know, if any batch command or for /f parameters can be used to count up from last line so that it will be always fix line number, for example on above text content, if I count up from last line and the 16^th line is what I want, then I can keep to export the 16^th line (count from the last line), then to capture the 4^th string for the number.

I tried to google but cannot find useful information.

Answer 1

Here it is a different and simpler approach. Instead of get the line two lines below Throughput(MB/s), we can get the line three lines above Throughput(Buffers/s) that can be get via a single findstr command.

@echo off
setlocal EnableDelayedExpansion

rem Get a CR+LF (NewLine) variable
for /F %%a in ('copy /Z "%~F0" NUL') do set ^"NL=%%a^
%Don't remove this line%
^"

rem Get the appropriate line and get its 4th token
findstr "!NL!!NL!!NL!Throughput(Buffers/s)" LOM_1.log | for /F "tokens=4" %%a in ('more') do echo %%a

If you want to process several *.log files, just insert the findstr ... | for ... commands into a cycle that process the files.

EDIT 2024/03/21: Get Errors number

I was reading your request in the comments below phuclv's answer about get the "Errors" number. Get such a number is very simple in the Batch file:

If the Packets Sent Packets Received Retransmits Errors Avg. CPU % is the last section in the file, just do this:

for /F "tokens=4" %%a in (LOM_1.log) do set "Errors=%%a"

and that is it! ;)

Answer 2

Here's one example of how I may try to achieve the task of retrieving those throughput values from each log file:

@Echo Off
SetLocal EnableExtensions DisableDelayedExpansion

PushD "P:\ath To\Log Directory" 2>NUL || Exit /B

(   For /F "Tokens=1-2 Delims=:" %%G In ('%SystemRoot%\System32\findstr.exe
     /INR "Throughput(MB/s)" "lom_*.log" 2^>NUL') Do Call :Sub "%%G" "%%H"
) 1>"%~dp0result.log"
Pause

PopD
EndLocal
GoTo :EOF

:Sub
Set /A "_l=%~2 + 1"
For /F "Tokens=4" %%I In ('%SystemRoot%\System32\more.com +%_l% "%~1"'
) Do Echo %%I& GoTo :EOF

You should obviously adjust the log file path on line four, and note that I have assumed the pattern of log file names is lom_*.log. The resulting file will reside in the same directory as your batch file.

Please note, if you are only processing one file per batch file run, rather than a directory full of them, you should only need to change 1> to 1>> on line eight.

Answer 3

I misread your data somewhat, noting now that the required Throughput(MB/s) is token 6 and the required data is in token 4.

Nor sure whether you want to process multiple files - if not, the outer loop on %%y is not needed and substitute the filename for %%y in the remaining loop.

%%b will acquire token 4 and %%c token 6 in the processed lines

@ECHO OFF
SETLOCAL
rem The following setting for the directory and filenames are names
rem that I use for testing and deliberately includes spaces to make sure
rem that the process works using such names. These will need to be changed to suit your situation.

SET "sourcedir=u:\your files"
SET "destdir=u:\your results"
SET "outfile=%destdir%\outfile.txt"

(
FOR %%y IN ("%sourcedir%\q78165354*.txt") DO (
 SET "required="
 SET "grabnext="
 FOR /f "tokens=4,6delims= " %%b IN ('findstr /v /L /c:"=" "%%y"') DO IF /i "%%c"=="Throughput(MB/s)" (SET "grabnext=y") ELSE (
  IF NOT DEFINED required IF DEFINED grabnext SET required=%%b&ECHO %%b %%y
 )
)
)>"%outfile%"
GOTO :EOF

rem Always verify against a test directory before applying to real data.

I used files named q78165354*.txt containing your data for my testing.

Answer 4

It's much easier in PowerShell, if the content isn't fixed then you can use this to match the 2^nd line after Throughput(MB/s) and grab the last word:

(sls -Co 0,2 'Throughput(MB/s)' -S file.txt).Context.PostContext.Split(' ')[-1]

If you want to get exactly the last token in line 43 then use this:

(gc file.txt | select -Skip 42 -F 1).Split(' ')[-1]

(some command).Split(' ') will split the result of some command by space ' ' into an array then [-1] will get the last element in the array

Full explanation and unaliased versions below

---

Anyway your question doesn't seem to match the title. If you want to the N^th line counting from the bottom which is similar to tail -<N> file.txt | head -1 on *nix then it's completely different. To get the 17th line from the last line simply use this

 gc file.txt -Ta 17 | select -F 1

You can call it from cmd if you really can't avoid cmd like this:

powershell -C "Get-Content file.txt -Tail 17 | Select-Object -First 1"
powershell -Command "(gc file.txt | select -Skip 42 -F 1).Split(' ')[-1]"

But just avoid cmd in new code, everything is easier in PowerShell

---

Here are the full unaliased version of the above commands

# Get the 43th line by skipping the first 42, then
# split the string by space ' ' and get the last word
(Get-Content file.txt | Select-Object -Skip 42 -First 1).Split(' ')[-1]

# Same as above but counting up skipping the last 16 lines instead
(Get-Content file.txt -Tail 17 | Select-Object -First 1).Split(' ')[-1]

# Find the line containing 'Throughput(MB/s)' and 2 lines of context after that,
# matching the string literally instead of regex, then extract the last word
# on the 2nd context line
(Select-String -Context 0,2 'Throughput(MB/s)' `
    -SimpleMatch file.txt).Context.PostContext.Split(' ')[-1]

Answer 5

You can read the file, use find to determine the line number of Througput(MB/s) then add one to that to get the line number where your 4th token is the result:

@echo off & set "tput="
for /F "delims=[]" %%i in ('find /N "Throughput(MB/s)" "LOM_1.log"') do set /a ln=1+%%i
for /f "usebackq skip=%ln%tokens=4delims= " %%i in ("LOM_1.log") do if not defined tput set "tput=%%i"
echo %tput%

Append >> result.log to the last line, if you require it to be stored in the file.