Windows.Media.Ocr won't scan anything if width and/or height are too small

Question

I'm using Windows.Media.Ocr that will recognize text through a transparent picturebox. It works with the follow code:

   Dim scaledSoftwareBitmap As Windows.Graphics.Imaging.SoftwareBitmap
        Dim softwareBmp As Windows.Graphics.Imaging.SoftwareBitmap
        Using bmp As Bitmap = New Bitmap(PictureBox1.Width, PictureBox1.Height)
            Using g As Graphics = Graphics.FromImage(bmp)
                Dim pt As Point = Me.PointToScreen(New Point(PictureBox1.Left, PictureBox1.Top))
                g.CopyFromScreen(pt.X, pt.Y, 0, 0, bmp.Size, CopyPixelOperation.SourceCopy)
                Using memStream = New Windows.Storage.Streams.InMemoryRandomAccessStream()
                    bmp.Save(memStream.AsStream(), System.Drawing.Imaging.ImageFormat.Bmp)
                    Dim decoder As Windows.Graphics.Imaging.BitmapDecoder = Await Windows.Graphics.Imaging.BitmapDecoder.CreateAsync(memStream)
                    softwareBmp = Await decoder.GetSoftwareBitmapAsync(decoder.BitmapPixelFormat, BitmapAlphaMode.Ignore)
                    Dim transform = New Windows.Graphics.Imaging.BitmapTransform() With {
                        .ScaledWidth = CUInt(PictureBox1.Width * 2),
                        .ScaledHeight = CUInt(PictureBox1.Height * 2),
                        .InterpolationMode = Windows.Graphics.Imaging.BitmapInterpolationMode.NearestNeighbor
                    }
                    Dim pixelData As Windows.Graphics.Imaging.PixelDataProvider = Await decoder.GetPixelDataAsync(Windows.Graphics.Imaging.BitmapPixelFormat.Bgra8, Windows.Graphics.Imaging.BitmapAlphaMode.Premultiplied, transform, Windows.Graphics.Imaging.ExifOrientationMode.RespectExifOrientation, Windows.Graphics.Imaging.ColorManagementMode.ColorManageToSRgb)
                    scaledSoftwareBitmap = New Windows.Graphics.Imaging.SoftwareBitmap(Windows.Graphics.Imaging.BitmapPixelFormat.Bgra8, CInt(transform.ScaledWidth), CInt(transform.ScaledHeight))
                    Dim pixels As Byte() = pixelData.DetachPixelData()
                    Dim buffer As Windows.Storage.Streams.IBuffer = pixels.AsBuffer()
                    scaledSoftwareBitmap.CopyFromBuffer(buffer)
                End Using
            End Using
        End Using
        Dim ocrEng = OcrEngine.TryCreateFromLanguage(New Windows.Globalization.Language("en-US"))
        Dim languages As IReadOnlyList(Of Windows.Globalization.Language) = ocrEng.AvailableRecognizerLanguages
        For Each language In languages
            Console.WriteLine(language.LanguageTag)
        Next
        Dim r = ocrEng.RecognizerLanguage
        Dim n = ocrEng.MaxImageDimension
        Dim ocrResult = Await ocrEng.RecognizeAsync(scaledSoftwareBitmap)
        RichTextBox1.Clear()

        Dim wordList As List(Of cText) = New List(Of cText)()
        Dim lines As IReadOnlyList(Of OcrLine) = ocrResult.Lines
        For Each line In lines
            For Each word In line.Words
                Dim nY As Double = CLng(word.BoundingRect.Bottom / 10) * 10
                wordList.Add(New cText() With {.Text = word.Text, .LocY = nY, .LocX = word.BoundingRect.Left})
            Next
        Next

        wordList.Sort(New WordComparer())

        Dim oldLocY As Double = 0
        For Each item As cText In wordList
            If (item.LocY > oldLocY And oldLocY <> 0) Then
                RichTextBox1.Text += Environment.NewLine
            End If
            RichTextBox1.Text += (item.Text + " ")
            oldLocY = item.LocY
        Next

but it seems not to scan anything if the width and or height are a bit small, like width:89 and height:27 eventhough the text is really visible on it. . It seems to start to scan when height is atleast 50 and width at least 40. Is there any other property that I might be missing or is this a limitation of the ocr engine? Thanks

Answer 1

I tested by resizing the final SoftwareBitmap and it seems to work with a small PictureBox height (tested with 32, which did not work with original SoftwareBitmap )
I tested *2, but maybe it can be better with other sizes (width cannot be too big or I got memory errors...)
or other parameters for the transformation

Add :

Dim scaledSoftwareBitmap As Windows.Graphics.Imaging.SoftwareBitmap

Add after "softwareBmp = Await decoder.GetSoftwareBitmapAsync()" :

Dim transform = New Windows.Graphics.Imaging.BitmapTransform() With {
                        .ScaledWidth = CUInt(PictureBox1.Width * 2),
                        .ScaledHeight = CUInt(PictureBox1.Height * 2),
                        .InterpolationMode = Windows.Graphics.Imaging.BitmapInterpolationMode.NearestNeighbor
                    }
Dim pixelData As Windows.Graphics.Imaging.PixelDataProvider = Await decoder.GetPixelDataAsync(Windows.Graphics.Imaging.BitmapPixelFormat.Bgra8, Windows.Graphics.Imaging.BitmapAlphaMode.Premultiplied, transform, Windows.Graphics.Imaging.ExifOrientationMode.RespectExifOrientation, Windows.Graphics.Imaging.ColorManagementMode.ColorManageToSRgb)
scaledSoftwareBitmap = New Windows.Graphics.Imaging.SoftwareBitmap(Windows.Graphics.Imaging.BitmapPixelFormat.Bgra8, CInt(transform.ScaledWidth), CInt(transform.ScaledHeight))
Dim pixels As Byte() = pixelData.DetachPixelData()
Dim buffer As Windows.Storage.Streams.IBuffer = pixels.AsBuffer()
scaledSoftwareBitmap.CopyFromBuffer(buffer)

Replace :

   'Dim ocrResult = Await ocrEng.RecognizeAsync(softwareBmp)
    Dim ocrResult = Await ocrEng.RecognizeAsync(scaledSoftwareBitmap)