question

MattiaFanti-4051 avatar image
0 Votes"
MattiaFanti-4051 asked XingyuZhao-MSFT edited

Windows.Media.Ocr won't scan anything if width and/or height are too small

I'm using Windows.Media.Ocr that will recognize text through a transparent picturebox. It works with the follow code:

    Dim scaledSoftwareBitmap As Windows.Graphics.Imaging.SoftwareBitmap
         Dim softwareBmp As Windows.Graphics.Imaging.SoftwareBitmap
         Using bmp As Bitmap = New Bitmap(PictureBox1.Width, PictureBox1.Height)
             Using g As Graphics = Graphics.FromImage(bmp)
                 Dim pt As Point = Me.PointToScreen(New Point(PictureBox1.Left, PictureBox1.Top))
                 g.CopyFromScreen(pt.X, pt.Y, 0, 0, bmp.Size, CopyPixelOperation.SourceCopy)
                 Using memStream = New Windows.Storage.Streams.InMemoryRandomAccessStream()
                     bmp.Save(memStream.AsStream(), System.Drawing.Imaging.ImageFormat.Bmp)
                     Dim decoder As Windows.Graphics.Imaging.BitmapDecoder = Await Windows.Graphics.Imaging.BitmapDecoder.CreateAsync(memStream)
                     softwareBmp = Await decoder.GetSoftwareBitmapAsync(decoder.BitmapPixelFormat, BitmapAlphaMode.Ignore)
                     Dim transform = New Windows.Graphics.Imaging.BitmapTransform() With {
                         .ScaledWidth = CUInt(PictureBox1.Width * 2),
                         .ScaledHeight = CUInt(PictureBox1.Height * 2),
                         .InterpolationMode = Windows.Graphics.Imaging.BitmapInterpolationMode.NearestNeighbor
                     }
                     Dim pixelData As Windows.Graphics.Imaging.PixelDataProvider = Await decoder.GetPixelDataAsync(Windows.Graphics.Imaging.BitmapPixelFormat.Bgra8, Windows.Graphics.Imaging.BitmapAlphaMode.Premultiplied, transform, Windows.Graphics.Imaging.ExifOrientationMode.RespectExifOrientation, Windows.Graphics.Imaging.ColorManagementMode.ColorManageToSRgb)
                     scaledSoftwareBitmap = New Windows.Graphics.Imaging.SoftwareBitmap(Windows.Graphics.Imaging.BitmapPixelFormat.Bgra8, CInt(transform.ScaledWidth), CInt(transform.ScaledHeight))
                     Dim pixels As Byte() = pixelData.DetachPixelData()
                     Dim buffer As Windows.Storage.Streams.IBuffer = pixels.AsBuffer()
                     scaledSoftwareBitmap.CopyFromBuffer(buffer)
                 End Using
             End Using
         End Using
         Dim ocrEng = OcrEngine.TryCreateFromLanguage(New Windows.Globalization.Language("en-US"))
         Dim languages As IReadOnlyList(Of Windows.Globalization.Language) = ocrEng.AvailableRecognizerLanguages
         For Each language In languages
             Console.WriteLine(language.LanguageTag)
         Next
         Dim r = ocrEng.RecognizerLanguage
         Dim n = ocrEng.MaxImageDimension
         Dim ocrResult = Await ocrEng.RecognizeAsync(scaledSoftwareBitmap)
         RichTextBox1.Clear()
    
         Dim wordList As List(Of cText) = New List(Of cText)()
         Dim lines As IReadOnlyList(Of OcrLine) = ocrResult.Lines
         For Each line In lines
             For Each word In line.Words
                 Dim nY As Double = CLng(word.BoundingRect.Bottom / 10) * 10
                 wordList.Add(New cText() With {.Text = word.Text, .LocY = nY, .LocX = word.BoundingRect.Left})
             Next
         Next
    
         wordList.Sort(New WordComparer())
    
         Dim oldLocY As Double = 0
         For Each item As cText In wordList
             If (item.LocY > oldLocY And oldLocY <> 0) Then
                 RichTextBox1.Text += Environment.NewLine
             End If
             RichTextBox1.Text += (item.Text + " ")
             oldLocY = item.LocY
         Next


but it seems not to scan anything if the width and or height are a bit small, like width:89 and height:27 eventhough the text is really visible on it. . It seems to start to scan when height is atleast 50 and width at least 40. Is there any other property that I might be missing or is this a limitation of the ocr engine? Thanks


dotnet-visual-basic
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

Castorix31 avatar image
0 Votes"
Castorix31 answered XingyuZhao-MSFT edited

I tested by resizing the final SoftwareBitmap and it seems to work with a small PictureBox height (tested with 32, which did not work with original SoftwareBitmap )
I tested *2, but maybe it can be better with other sizes (width cannot be too big or I got memory errors...)
or other parameters for the transformation

Add :

 Dim scaledSoftwareBitmap As Windows.Graphics.Imaging.SoftwareBitmap


Add after "softwareBmp = Await decoder.GetSoftwareBitmapAsync()" :


 Dim transform = New Windows.Graphics.Imaging.BitmapTransform() With {
                         .ScaledWidth = CUInt(PictureBox1.Width * 2),
                         .ScaledHeight = CUInt(PictureBox1.Height * 2),
                         .InterpolationMode = Windows.Graphics.Imaging.BitmapInterpolationMode.NearestNeighbor
                     }
 Dim pixelData As Windows.Graphics.Imaging.PixelDataProvider = Await decoder.GetPixelDataAsync(Windows.Graphics.Imaging.BitmapPixelFormat.Bgra8, Windows.Graphics.Imaging.BitmapAlphaMode.Premultiplied, transform, Windows.Graphics.Imaging.ExifOrientationMode.RespectExifOrientation, Windows.Graphics.Imaging.ColorManagementMode.ColorManageToSRgb)
 scaledSoftwareBitmap = New Windows.Graphics.Imaging.SoftwareBitmap(Windows.Graphics.Imaging.BitmapPixelFormat.Bgra8, CInt(transform.ScaledWidth), CInt(transform.ScaledHeight))
 Dim pixels As Byte() = pixelData.DetachPixelData()
 Dim buffer As Windows.Storage.Streams.IBuffer = pixels.AsBuffer()
 scaledSoftwareBitmap.CopyFromBuffer(buffer)

Replace :

    'Dim ocrResult = Await ocrEng.RecognizeAsync(softwareBmp)
     Dim ocrResult = Await ocrEng.RecognizeAsync(scaledSoftwareBitmap)




· 6
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hey castorix, I'm updating the code I'm using since now I got your amazing code that scan by row and not by column, please tell me if I implement that code with this last code of you in a wrong way. The ide doesn't return any error, but It still doesn't scan. Look at the gif 164697-2.gif


0 Votes 0 ·
2.gif (581.0 KiB)
Castorix31 avatar image Castorix31 MattiaFanti-4051 ·

I just tested on a Button click :

164679-vb-ocr.gif


0 Votes 0 ·
vb-ocr.gif (93.9 KiB)

Did you try the code I've just edited?
did you try using width:89 and height:27 ?
is you app dpi aware?
is your autoscale form property on dpi?

0 Votes 0 ·
Show more comments