I'm new here, so pardon me if my question is not in the proper format to produce a response. Please let me know if I need to add anything:
I'm currently working on a project from home for a mate at a company that uses a really old legacy system that displays output to a screen. The system they have is dated, and no longer supported (the company who set it up is out of business). I was told they need to keep the system for the monthly reports, but want to display it's output in a more modern way so that when people come in they can see what's happening without embarrassment. Tough to explain, but basically I've been asked to capture information from a screen and output to another.
I'm about 3/4 of the way through a sample, written in VB.Net... and found the image comparison to be far too slow for the changes taking place using getpixel for comparison. Speed was also an issue when saving pictures to a memory stream for comparison (going through x,y looking for match)... so i've switched over to lockbits/marshal copy to byte arrays for comparison. Basically - data displayed in multiple areas always matches one of about 20 images I have saved. I can compare the images using captures and saved images (place diff on top of the existing saved image and the output is blacked out 100%). I know my saved images are matching 100% and previous methods of comparison from g.fromimage match fully, but the methods I was using is too slow. Now that I'm using byte arrays, I have been seeing a lot of inconsistency and have found that the byte arrays are changing randomly when scraped from the same images (image loaded to compare to, and stored to array... image on screen stored to array, both saved to to text AND nothing else being done over and over produces differences.).
With a brand new project open I have reproduced this issue. Loading an image into a picturebox at start/unchanged throughout - g.copyfromscreen to a bitmap and then stored to a byte array and exported to a text file is different when done over and over. I can't figure out why this is happening. It's almost like I've accidentally discovered a random generator haha. The areas I'm comparing are 5x72 so not massive. Right now to simplify, I'm loading a picture into a pixturebox and clicking a button to store the data from that picture into a byte array. Then I am saving an area of the picture to .png and a byte array text conversion of the same area to a text file. Clicking the button over and over produces different byte arrays to text outputs but png files that match 100%. I'm not moving the image around, I'm not reloading the picturebox... I don't see why the byte arrays are changing.
Module Module1
Public baGB1Small((5 * 72 - 1) * 3) As Byte ' To be populated with small image.
Public baGB2Small((5 * 72 - 1) * 3) As Byte ' To be populated with small image.
Public baGB3Small((5 * 72 - 1) * 3) As Byte ' To be populated with small image.
Public baGB4Small((5 * 72 - 1) * 3) As Byte ' To be populated with small image.
End Module
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
Dim filename As String = DateTime.Now.ToString("yyyy_MM_dd_HH_mm_ss_fff")
'-- Below saves a chunk of the screen with the picturebox in it for comparison.
Dim BMPScrNew As New Bitmap(5, 420, Imaging.PixelFormat.Format24bppRgb)
Dim GrabRect As New Rectangle(478, 170, 5, 420)
Using g As Graphics = Graphics.FromImage(BMPScrNew)
g.CopyFromScreen(GrabRect.Location, New Point(0, 0), GrabRect.Size)
End Using
BMPScrNew.Save("C:\TestSaveLoc\SCR1-" & filename & ".png", Imaging.ImageFormat.Png)
'-- Below breaks the main saved image into chunks for comparison.
'-- Chunk 1:
Dim BMPSN As New Bitmap(5, 72)
Dim GRNew As New Rectangle(0, 7, 5, 72)
BMPSN = BMPScrNew.Clone(GRNew, Imaging.PixelFormat.Format24bppRgb)
BMPSN.Save("C:\TestSaveLoc\B1-" & filename & ".png", Imaging.ImageFormat.Png)
Dim BMD As Imaging.BitmapData = BMPSN.LockBits(New Rectangle(0, 0, BMPSN.Width, BMPSN.Height), Imaging.ImageLockMode.ReadOnly, Imaging.PixelFormat.Format24bppRgb)
Runtime.InteropServices.Marshal.Copy(BMD.Scan0, baGB1Small, 0, baGB1Small.Length)
BMPSN.UnlockBits(BMD)
'-- Chunk 2:
BMPSN = New Bitmap(5, 72)
GRNew = New Rectangle(0, 89, 5, 72)
BMPSN = BMPScrNew.Clone(GRNew, Imaging.PixelFormat.Format24bppRgb)
BMPSN.Save("C:\TestSaveLoc\B2-" & filename & ".png", Imaging.ImageFormat.Png)
BMD = BMPSN.LockBits(New Rectangle(0, 0, BMPSN.Width, BMPSN.Height), Imaging.ImageLockMode.ReadOnly, Imaging.PixelFormat.Format24bppRgb)
Runtime.InteropServices.Marshal.Copy(BMD.Scan0, baGB2Small, 0, baGB2Small.Length)
BMPSN.UnlockBits(BMD)
'-- Chunk 3:
BMPSN = New Bitmap(5, 72)
GRNew = New Rectangle(0, 171, 5, 72)
BMPSN = BMPScrNew.Clone(GRNew, Imaging.PixelFormat.Format24bppRgb)
BMPSN.Save("C:\TestSaveLoc\B3-" & filename & ".png", Imaging.ImageFormat.Png)
BMD = BMPSN.LockBits(New Rectangle(0, 0, BMPSN.Width, BMPSN.Height), Imaging.ImageLockMode.ReadOnly, Imaging.PixelFormat.Format24bppRgb)
Runtime.InteropServices.Marshal.Copy(BMD.Scan0, baGB3Small, 0, baGB3Small.Length)
BMPSN.UnlockBits(BMD)
'-- Chunk 4:
BMPSN = New Bitmap(5, 72)
GRNew = New Rectangle(0, 253, 5, 72)
BMPSN = BMPScrNew.Clone(GRNew, Imaging.PixelFormat.Format24bppRgb)
BMPSN.Save("C:\SWAutoer\TestSaveLoc\B4-" & filename & ".png", Imaging.ImageFormat.Png)
BMD = BMPSN.LockBits(New Rectangle(0, 0, BMPSN.Width, BMPSN.Height), Imaging.ImageLockMode.ReadOnly, Imaging.PixelFormat.Format24bppRgb)
Runtime.InteropServices.Marshal.Copy(BMD.Scan0, baGB4Small, 0, baGB4Small.Length)
BMPSN.UnlockBits(BMD)
BMPSN.Dispose()
BMPScrNew.Dispose()
'-- Writes output to a textbox on the screen (for testing only)
Output1.AppendText(Environment.NewLine & DateTime.Now)
Output1.AppendText(Environment.NewLine & "FirstCheck:" & CDWB3(filename))
filename = Nothing
End Sub
Public Function CDWB3(vfpath As String) As String
Dim BMPScrNew As New Bitmap(5, 420, Imaging.PixelFormat.Format24bppRgb)
Dim GrabRect As New Rectangle(478, 170, 5, 420)
Using g As Graphics = Graphics.FromImage(BMPScrNew)
g.CopyFromScreen(GrabRect.Location, New Point(0, 0), GrabRect.Size)
End Using
BMPScrNew.Save("C:\TestSaveLoc\SCR2-" & vfpath & ".png", Imaging.ImageFormat.Png)
Dim WasFound As Integer = 0
Dim FoundAtY As Integer = 0
Dim Checkst As Stopwatch = Stopwatch.StartNew
Dim StoppedAt As Integer = 169
For vNewY = 0 To 348 ' Goes down one pixel through the main scraped image to compare a chunk that is 5x72 to the saved arrays of smaller chunks.
Dim BMPSN As New Bitmap(5, 72)
Dim GRNew As New Rectangle(0, vNewY, 5, 72)
BMPSN = BMPScrNew.Clone(GRNew, Imaging.PixelFormat.Format24bppRgb)
BMPSN.Save("C:\TestSaveLoc\B1Loc-" & vfpath & ".png", Imaging.ImageFormat.Png)
Dim BMD As Imaging.BitmapData = BMPSN.LockBits(New Rectangle(0, 0, BMPSN.Width, BMPSN.Height), Imaging.ImageLockMode.ReadOnly, Imaging.PixelFormat.Format24bppRgb)
Dim vmBuffer((BMPSN.Width * BMPSN.Height - 1) * 3) As Byte
Marshal.Copy(BMD.Scan0, vmBuffer, 0, vmBuffer.Length)
BMPSN.UnlockBits(BMD)
If vNewY = 7 Then '-- THIS IS WHERE bagb1small should always match.
Dim writer1 As New IO.StreamWriter("C:\TestSaveLoc\MainBuff-" & vfpath & ".txt")
writer1.WriteLine(Convert.ToBase64String(vmBuffer))
writer1.Flush()
writer1.Close()
writer1.Dispose()
Dim writer2 As New IO.StreamWriter("C:\TestSaveLoc\bagb1small-" & vfpath & ".txt")
writer2.WriteLine(Convert.ToBase64String(bagb1small))
writer2.Flush()
writer2.Close()
writer2.Dispose()
End If
StoppedAt = StoppedAt + 1
Dim B1Found As Boolean = True
Dim B2Found As Boolean = True
Dim B3Found As Boolean = True
Dim B4Found As Boolean = True
For i = 0 To vmBuffer.Length - 1 Step 3
Dim BadCount As Integer = 0
If vmBuffer(i) <> baGB1Small(i) Then
B1Found = False
BadCount += 1
End If
If vmBuffer(i) <> baGB2Small(i) Then
B2Found = False
BadCount += 1
End If
If vmBuffer(i) <> baGB3Small(i) Then
B3Found = False
BadCount += 1
End If
If vmBuffer(i) <> baGB4Small(i) Then
B4Found = False
BadCount += 1
End If
If BadCount > 3 Then Exit For
Next
BMD = Nothing
vmBuffer = Nothing
If B1Found = True Then
WasFound = 1
Exit For
End If
If B2Found = True Then
WasFound = 2
Exit For
End If
If B3Found = True Then
WasFound = 3
Exit For
End If
If B4Found = True Then
WasFound = 4
Exit For
End If
BMPSN.Dispose()
Next
BMPScrNew.Dispose()
Checkst.Stop()
CDWB3 = WasFound.ToString & "," & StoppedAt
End Function
With the above code, a picture is loaded into a picturebox at a set position on the screen in the form's load event. What I expect to happen is clicking the button will produce the same output every time (byte array matches last click's byte array) since the image on the screen is not changing at all. This is not the case.
My image outputs (file saves to png) match every time I click the button. The byte array output to text doesn't match. Chunk 1 is being found sometimes, chunk 2-4 other times (chunk 1 not seen on screen), and sometimes none at all... This is with a static image on the screen that I am scraping from and comparing to - this should not be the case. Chunk 1 should always be found at the same position every time and the byte arrays should be the same every time.