question

njsokalski avatar image
0 Votes"
njsokalski asked ·

How do I use an image from ImageCropper with Windows.Media.Ocr?

I am attempting to use an image from the ImageCropper in Windows Community Toolkit (https://docs.microsoft.com/en-us/windows/communitytoolkit/controls/imagecropper) with Windows.Media.Ocr. This is my first time using either one of these, and my OcrResult object claims the image has no text (the Text property has a value of ""). My app displays the ImageCropper with it's image with no problem, so the original image is definitely there, but I cannot seem to get the cropped image, therefore preventing me from using it in the Ocr. I'm sure this code is very simple once I see it, but right now, I'm struggling to find it. Can anybody help me? Thanks.

windows-uwp
· 1
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Do you mean you can't continue to use Ocr since you failed to get the cropped image? If it is, you can refer to the source code of ImageCropper to get the cropped image. If it isn't, can you show more details about your unexpected behavior and the code snippet what you have tried?


0 Votes 0 ·
njsokalski avatar image
0 Votes"
njsokalski answered ·

Here is the code I am using to get an image from the camera (cePuzzle is a CaptureElement & puzzlecapture is a MediaCapture & icPuzzle is an ImageCropper):

 Private Async Sub btnStartStopScanning_Click(sender As Object, e As RoutedEventArgs) Handles btnStartStopScanning.Click
 Select Case Me.puzzlecapture.CameraStreamState
 Case CameraStreamState.NotStreaming
 Me.btnStartStopScanning.Content = "Stop Scanning"
 Me.btnLookForPuzzle.IsEnabled = False
 Me.cePuzzle.Visibility = Visibility.Visible
 Me.icPuzzle.Reset()
 Await Me.puzzlecapture.StartPreviewAsync()
 Case CameraStreamState.Streaming
 Me.btnStartStopScanning.Content = "Start Scanning"
 Me.btnLookForPuzzle.IsEnabled = True
 Me.cePuzzle.Visibility = Visibility.Collapsed
 Await Me.puzzlecapture.StopPreviewAsync()
    
 'Create a temporary file for the ImageCropper from the camera
 Dim sf As StorageFile = Await ApplicationData.Current.LocalFolder.CreateFileAsync("temppuzzleimage.jpeg", CreationCollisionOption.ReplaceExisting).AsTask()
 'Capture the image from the camera
 Await Me.puzzlecapture.CapturePhotoToStorageFileAsync(ImageEncodingProperties.CreateJpeg(), sf)
 'Place the image in the ImageCropper
 Await Me.icPuzzle.LoadImageFromFile(sf)
 'Case CameraStreamState.BlockedForPrivacy
 'Case CameraStreamState.Shutdown
 End Select
 End Sub

This code works fine, it takes the image & puts it in the ImageCropper, so I can see it & it lets me select a region to crop. However, when I use the following code (which may be incomplete * the OCR is currently commented out, since I have been trying things which were obviously incorrect) it does not seem to work (there is no code there to create the SoftwareBitmap because I was probably doing it wrong as well, so most of this code probably needs rewritten or is missing):

 Private Async Sub btnLookForPuzzle_Click(sender As Object, e As RoutedEventArgs) Handles btnLookForPuzzle.Click
 Dim sf As StorageFile = Await ApplicationData.Current.LocalFolder.CreateFileAsync("tempcroppedimage.png", CreationCollisionOption.ReplaceExisting)
 Dim stream As Streams.IRandomAccessStream = Await sf.OpenAsync(FileAccessMode.ReadWrite)
 Await Me.icPuzzle.SaveAsync(stream, BitmapFileFormat.Jpeg)
 'Dim bmp As SoftwareBitmap
    
 'Dim ocr As OcrEngine = OcrEngine.TryCreateFromUserProfileLanguages()
 'Dim result As OcrResult = Await ocr.RecognizeAsync(bmp)
 'For Each textline As OcrLine In result.Lines
 ' System.Diagnostics.Debug.WriteLine(textline.Text)
 'Next
 'System.Diagnostics.Debug.WriteLine(result.Text)
 End Sub

Everything I have tried would give an OcrResult that has found no results (result.Text is ""). What should I be doing? Thanks.

· 1 ·
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Windows.Media.Ocr API allows for application developer to extract text in the specific language from an image. So first, does your cropped image have text in it? In addition, does the text show the same language with your specified language(e.g. you specify English, but the text shows Chinese, then the Text of OcrResult will return "").

0 Votes 0 ·
njsokalski avatar image
0 Votes"
njsokalski answered ·

I tried changing the second method (btnLookForPuzzle) to the following:

 Private Async Sub btnLookForPuzzle_Click(sender As Object, e As RoutedEventArgs) Handles btnLookForPuzzle.Click
     Dim sf As StorageFile = Await ApplicationData.Current.LocalFolder.GetFileAsync("temppuzzleimage.png")
     Dim stream As Streams.IRandomAccessStream = Await sf.OpenAsync(FileAccessMode.ReadWrite)
     Await Me.icPuzzle.SaveAsync(Stream, BitmapFileFormat.Png)
     Dim decoder As BitmapDecoder = Await BitmapDecoder.CreateAsync(stream)
     Dim bmp As SoftwareBitmap = Await decoder.GetSoftwareBitmapAsync()
     Dim ocr As OcrEngine = OcrEngine.TryCreateFromUserProfileLanguages()
     Dim result As OcrResult = Await ocr.RecognizeAsync(bmp)
     For Each textline As OcrLine In result.Lines
         System.Diagnostics.Debug.WriteLine(textline.Text)
     Next
     System.Diagnostics.Debug.WriteLine(result.Text)
 End Sub

This finds some little pieces of text, but not nearly everything, and a large amount of what it does find is incorrect. Most of what it should find is single digits, almost none of which it finds, and some of the text it finds incorrectly. The text is solid black on a white & yellow background, so the contrast is very distinct, and the font is very basic (it is not some fancy script or anything like that). However, this also gives the following exception the second time the method is called:

Exception thrown: 'System.IO.FileLoadException' in System.Private.CoreLib.dll
WinRT information: The file is in use. Please close the file before continuing.

I'm sure that this is also something that can be easily fixed, but since I am not experienced with working with files, I'm not sure how to fix it. How do I fix this exception, and why is the OCR having so much trouble finding basic text? Thanks.

· 1 ·
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

When I tested it, I didn't receive the same issue and could print the correct text. Can you provide your cropped image for us to test? In addition, here is an official sample about Ocr, you can also test it.


0 Votes 0 ·
njsokalski avatar image
0 Votes"
njsokalski answered ·

Here is one of the images I tried scanning (Note: The bottom of the image I am sending is cropped off because of the size limit, but the part that was cropped off is the same as the part near the bottom of the image):

7654-temppuzzleimage.png



The text that I was expecting it to find would be similar to the following:

EASY
VOLUME 256
6 1 4 5

In place of "EASY" it found several seemingly random characters that didn't even look similar to me, in most cases, it did find "VOLUME 256", and it never found anything else (the numbers, which are actually the part that I care about it finding). I found it very surprising that it would find "VOLUME 256" but not "EASY", since it looks like they are the same (or at least a similar) font, and "EASY" is actually a little bit larger. The numbers are a little bit thinner, so that is my best guess as to not finding them, although they still look pretty clear. Any ideas?


· 1 ·
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

When I used this image directly, I can reproduce your issue, the "Easy" can't be recognized. But if I took this image with mobile phone, then recognized it, the "Easy" can be recognized. So this issue has something to do with image. You can go to the feedback(Start menu -> Feedback Hub) to raise this question.

0 Votes 0 ·