question

njsokalski avatar image
0 Votes"
njsokalski asked ·

Repeated Camera Captures

I am using CaptureElement & MediaCapture for the camera, and I want to automatically continuously capture images that I will process (until the user stops the camera, of course). I want to take a picture, process it, take another, process it, etc. I do not need to keep the images once they have been processed, they are only temporary while processing. However, I am not sure how to do this. I am receiving Exceptions about the file being in use and things needing initialized (I don't know if these things are related or not). I have used CaptureElement & MediaCapture plenty of times before, but the idea of repeated captures is new for me. What would be the best way to do this? Thanks.

Here is the code involved in my problem. The XAML elements referenced in this are ceCard (a CaptureElement), imgCropped (an Image), and btnStartStopScanning (a Button). The VB.NET code (in MainPage.xaml.vb) is:

 Public NotInheritable Class MainPage : Inherits Page
  Private ReadOnly endpt As New CustomVisionPredictionClient() With {.ApiKey = App.PredictionKey, .Endpoint = App.EndPoint}
  Private cardcapture As MediaCapture
    
  Public Sub New()
  InitializeComponent()
  End Sub
    
  Private Async Sub MainPage_Loaded(sender As Object, e As RoutedEventArgs) Handles Me.Loaded
  'Prepare for previewing
  Try
  Me.cardcapture = New MediaCapture()
  Await Me.cardcapture.InitializeAsync(New MediaCaptureInitializationSettings() With {.StreamingCaptureMode = StreamingCaptureMode.Video, .VideoDeviceId = (Await DeviceInformation.FindAllAsync(DeviceClass.VideoCapture)).FirstOrDefault(Function(di) di.EnclosureLocation.Panel = Windows.Devices.Enumeration.Panel.Back).Id})
  Catch ex As Exception
  End Try
  Try
  Me.ceCard.Source = Me.cardcapture
  Catch ex As Exception
  End Try
  End Sub
    
  Private Async Sub btnStartStopScanning_Click(sender As Object, e As RoutedEventArgs) Handles btnStartStopScanning.Click
  Select Case Me.cardcapture.CameraStreamState
  Case CameraStreamState.NotStreaming
  Await Me.cardcapture.StartPreviewAsync()
  Me.btnStartStopScanning.Content = "Stop Scanning"
  Try : Me.ScanCard()
  Catch ex As Exception
  End Try
  Case CameraStreamState.Streaming
  Await Me.cardcapture.StopPreviewAsync()
  Me.btnStartStopScanning.Content = "Start Scanning"
  End Select
  End Sub
    
  Private Async Sub ScanCard()
  'Create a temporary file for the image from the camera
  Dim sf As StorageFile = Await ApplicationData.Current.LocalFolder.CreateFileAsync("originalcardimage.jpeg", CreationCollisionOption.ReplaceExisting).AsTask()
  'Capture the image from the camera
  Await Me.cardcapture.CapturePhotoToStorageFileAsync(ImageEncodingProperties.CreateJpeg(), sf)
  'Get the size of the image
  Dim imagewidth As Double = CType(Me.cardcapture.VideoDeviceController.GetMediaStreamProperties(MediaStreamType.Photo), VideoEncodingProperties).Width
  Dim imageheight As Double = CType(Me.cardcapture.VideoDeviceController.GetMediaStreamProperties(MediaStreamType.Photo), VideoEncodingProperties).Height
  Await Dispatcher.RunAsync(Core.CoreDispatcherPriority.Normal, Async Sub()
  Me.GenerateCard(Await Me.endpt.DetectImageWithNoStoreAsync(App.ProjectId, App.PublishedName, File.OpenRead(Path.Combine(ApplicationData.Current.LocalFolder.Path, "originalcardimage.jpeg"))), imagewidth, imageheight)
  If Me.cardcapture.CameraStreamState = CameraStreamState.Streaming Then Me.ScanCard()
  End Sub)
  End Sub
    
  Private Async Sub GenerateCard(result As ImagePrediction, imagewidth As Double, imageheight As Double)
  If result.Predictions.Any(Function(pm) pm.TagName = "ColorCard" AndAlso pm.Probability >= 0.8) Then
  'Get the card with the highest probability
  Dim pm As PredictionModel = result.Predictions.OrderByDescending(Function(model) model.Probability)(0)
  'Crop the image from the camera
  Dim croppedbmp As WriteableBitmap = Await BitmapFactory.FromStream(File.OpenRead(Path.Combine(ApplicationData.Current.LocalFolder.Path, "originalcardimage.jpeg")))
  croppedbmp = croppedbmp.Crop(CInt(pm.BoundingBox.Left), CInt(pm.BoundingBox.Top), CInt(pm.BoundingBox.Width), CInt(pm.BoundingBox.Height))
  Me.imgCropped.Source = croppedbmp
  End If
  End Sub
 End Class

The most significant parts of this code are the ScanCard() and GenerateCard(...) methods where I call File.OpenRead(...). My basic goal is to repeat the process of capturing an image (like I do with CapturePhotoToStorageFileAsync(...)) and then looking at it (like I do in GenerateCard(...)). I think calling File.OpenRead(...) in multiple places is causing the problem, but I am not sure where or how to close (or whatever I need to do) to avoid the following Exception:

Exception thrown: 'System.IO.FileLoadException' in System.Private.CoreLib.dll
WinRT information: The file is in use. Please close the file before continuing.

What do I need to change or add? Thanks.



Update -1:

You are correct in that I (supposedly) do not need to create a file, but I am not sure what the process would be to get directly from the CaptureElement to a SoftwareBitmap (or, in my case, it is actually a WriteableBitmap that I need). You will notice that the method I use to capture the image is CapturePhotoToStorageFileAsync (which requires a StorageFile, which requires a file). You will also notice that after opening the file (in the GenerateCard method) I convert it to a WriteableBitmap, which is when I do stuff with it. If I had a way to get from the CaptureElement to a WriteableBitmap without using a file, that would be great, but I am not sure how to do that. I noticed that there is a CapturePhotoToStreamAsync method, but I was not quite sure how to use this (streams have always been one of my weak points, I'm never sure if I created them correctly, whether I closed them correctly, if I am using them correctly, etc.). If you could show me the best way to get from a CaptureElement to a WriteableBitmap without using a file (if using CapturePhotoToStreamAsync is the right way, can you show me the required steps so that I don't forget anything?), it would be greatly appreciated. Thanks!



Update -2:

I think that you have mostly answered my question, although now I have an (almost opposite question). I created the following method to capture a WriteableBitmap:

 Private Async Function CaptureWriteableBitmap() As Task(Of WriteableBitmap)
  Dim softwarebmp As SoftwareBitmap = (Await (Await Me.cardcapture.PrepareLowLagPhotoCaptureAsync(ImageEncodingProperties.CreateUncompressed(MediaPixelFormat.Bgra8))).CaptureAsync()).Frame.SoftwareBitmap
  Dim writeablebmp As WriteableBitmap = New WriteableBitmap(softwarebmp.PixelWidth, softwarebmp.PixelHeight)
  softwarebmp.CopyToBuffer(writeablebmp.PixelBuffer)
  Return writeablebmp
 End Function

However, you may have noticed that in my original posting (on line 45) I call the DetectImageWithNoStoreAsync method (this is a method from Azure's CustomVision service), which is where I originally used the File.OpenRead method to get a FileStream (the 3rd parameter of DetectImageWithNoStoreAsync is of type Stream). So I now need to figure out how to get from a WriteableBitmap to a Stream. I have tried methods such as WriteableBitmap.ToStream, but couldn't seem to get that to work. I am not quite sure how to get the Stream to pass (or even what kind of stream to pass, since it is obviously not a FileStream). What kind of Stream should I be passing, and how can I get it from the WriteableBitmap (or the SoftwareBitmap that I create in my CaptureWriteableBitmap method)? Thanks!

windows-uwp
· 3
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi, I noticed that you mentioned that you received an error. Can you provide a minimal runnable demo so that we can reproduce your problem? Did you call AdvancedPhotoCapture.FinishAsync or release MediaCapture during the repeated photo taking process?

0 Votes 0 ·

Hello, in the problem description, you said that you do not need to keep the image, so do you have to create a file? Is it possible to save the captured image as SoftwareBitmap as described in this document? Eliminate the process of storing to local files, and save to SoftwareBitmap for more advantages in continuous photos.

0 Votes 0 ·

Hello, you can try writeableBitmap.PixelBuffer.AsStream(). In addition, Answer is the place to provide solutions. If you have supplementary explanations to the questions, please write comments below the corresponding questions or answers. If the length of comments is limited, you can modify your problem description. I have modified your problem description and added your previous reply.

0 Votes 0 ·

1 Answer

RichardZhang-MSFT avatar image
0 Votes"
RichardZhang-MSFT answered ·

Hello,​

Welcome to our Microsoft Q&A platform!

According to our discussion, for take photos continuously without creating a local file, you can convert the photos to SoftwareBitmap, the specific content is in this document.

If you just need WriteableBitmap, you don't necessarily need to stream. After obtaining SoftwareBitmap, we can directly convert to WriteableBitmap.

 Dim bitmap As WriteableBitmap = New WriteableBitmap(captureBitmap.PixelWidth, captureBitmap.PixelHeight)
 captureBitmap.CopyToBuffer(bitmap.PixelBuffer)

Thanks.

·
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.