question

SangramMohanty-9763 avatar image
0 Votes"
SangramMohanty-9763 asked MartinJaffer-MSFT commented

Why I have been charged to stop active dataflow debug sessions in Azure data factory ?

I have been charged to stop active dataflow debug sessions in Azure data factory.

I had stopped some active data flow debug sessions that were showing under Azure data factory. I used the below command to stop them : -

Stop-AzDataFactoryV2DataFlowDebugSession -ResourceGroupName adf -DataFactoryName WikiADF -SessionId <sessionid>

Next day, I was a sudden spike in my bill. Earlier on 30th April it was Rs 37/- and all of a sudden it spiked to Rs 1500/-. I don't understand why Azure had charged me so much only to stop some active dataflow debug sessions. Beneath it data flow debug sessions were standard clusters i.e. 4+4 vCore clusters that gets suspended after 1 hour.

Also, I had created these session during my trial period and Azure that deducted those price from it's trial amount. But does stopping an old sessions cost too much and that too those were created during my trial period ?

Kindly, answer me on this.

azure-data-factoryazure-cost-management
· 10
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thank you Martin,
I have created a Support request for the same. Please let me know if any further action I need to take.

1 Vote 1 ·

If there are any learnigns you feel comfortable sharing, it would be great to help educate the community. Or you can just let me know the ticket number.

0 Votes 0 ·

I have created support request ID 2205100030000179 for the same issue.

0 Votes 0 ·

Hello @SangramMohanty-9763 and welcome to Microsoft Q&A.

For billing issues, it is better to open a support ticket. (Because I do not have the means to provide remediation in case of an error, and you would have to go to support.) If you do not have a support plan, I can provide you a 1-time free support voucher. Let me know whether you need the voucher or not.

0 Votes 0 ·

Hi Martin,
Thank you for your helpful gesture. Definitely, I want to accept 1-time free support voucher. I will provide all my questions in the same support ticket ?

0 Votes 0 ·

I have emailed you receipt and instructions on redeeming the support ticket @SangramMohanty-9763

0 Votes 0 ·

Hi Martin,
I have not received any convincing reply from assigned support engineer for this ticket. My next billing date is approaching fast i.e. 20th may 2022 and my case is still not resolved.

Please let me know how I can escalate it so that things get resolved faster.

0 Votes 0 ·

I took a look at the ticket @SangramMohanty-9763 , I think I can explain better. I put in answer, because too big for comment. I also tried to address your additional questions.

0 Votes 0 ·

Okay @SangramMohanty-9763 I just read some of the case notes. I think I can make a more relatable explanation.

Okay, so you have your computer. You pay for the electricity it uses. When on, it uses electricity. You know how when you walk away from a laptop, after a while the laptop eventually turns off? This period of time between walking away and it turning off, is called Time To Live (TTL). You can choose how much time this should be (for real, and in this example). If you come back to the laptop before the time runs out, and type something, the timer is reset. Every time you do something on the laptop, the timer is reset.

Well, the same idea applies to the Data Flow clusters.

If the Time to Live is set to 0, then the cluster should shut down when it finishes the job.
If the Time to Live is set to 1 hour, then the cluster stays on and waits for a new job. If it gets a new job, then it runs, and when the run finishes, the countdown is reset. The timer will not shut down the cluster mid-job. If no new job comes along before the hour runs out, then the cluster shuts down.

So it wasn't "Shutting down the cluster" that you got charged for, it was the idle time the cluster waited after the job finished.

Shutting off a cluster is not an act that should cause additional charge.

How to set the Time to Live on a new cluster:
202472-image.png

0 Votes 0 ·
image.png (221.4 KiB)

<<Continuation to above comment from @MartinJaffer-MSFT - As part of moderation I have to break it down into two comments due to comment size limitation>>

  1. in the studio, go to the Management page
    1. In the Management page, select Integration Runtimes

    2. In the Integration runtimes panel, select + New

    3. In the setup panel choose Azure, Selfhosted

    4. In the setup panel choose Azure

    5. In the setup panel choose the "Data flow runtime" tab at the top

    6. The Time to Live is the last of 3 items.

How to set the Time to Live on an existing cluster:
1. Step 1 & 2, same as above in creating new cluster. Get to the Integration Runtimes page.
2. Click the name of the relevant integration runtime.
3. In the popup select "Data flow runtime" tab at the top.

Note: the default AutoResolveIntegrationRuntime cannot be edited, but any you create can be edited.

I'm not sure what you mean by suspended cluster. I might just be having a dumb moment, but could you give some context please?

I see there was mention of Managed Vnet integration runtime. My examples are mainly focused on Data Flow, but I think the idea can be reused.

0 Votes 0 ·
SangramMohanty-9763 avatar image
0 Votes"
SangramMohanty-9763 answered SangramMohanty-9763 edited

Hi Martin & All,
Please check my existing time to live for data flow cluster.
202578-image.png




The snippet says the idle time of this cluster is 0 minutes. Then why did I got charged so much ?
Earlier I had given my necessary information like subscription id. Can you please check why I got charged so much on 18th April & 12th April.

You mentioned shutting down a cluster doesn't charge. But I didn't created so many clusters on 30th April. But I had shut them all. Then I found accumulation cost in my bill had spiked. It happened for both the dates 12th April & 18th April. Why did this happen then ? Why it didn't happen before ? What I remember is Azure charged me some money during trial period for these dates. But those costs were adjusted during my trial period billing cycle.

The clusters were created during my trial period.


image.png (54.8 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

SangramMohanty-9763 avatar image
0 Votes"
SangramMohanty-9763 answered MartinJaffer-MSFT commented

Please answer my below questions : -

  1. Why Azure charged me so much on 30th April ? I have deleted unused sessions from Azure data factory on 30th April. There were multiple unused sessions( with time to live = 0 mins) created on 12th April & 18th April that were deleted. Once I removed all those unused sessions, I saw a spike in my Azure bill. Please check accumulated cost in my bill with subscription id : 5a583eac-ccd6-4c26-acc1-bac90b9606f3.

  2. Does Azure charge for used data flow sessions ? I only deleted unused data flow sessions not active data flow sessions on 30th April.

  3. Why Azure is charging me now while I created those data flow sessions during my trial period 20th March to 19th April 2022 ?

  4. If Azure is charging for Time to Live for those clusters then why didn't it charge me before. Why it happened once I deleted those data flow sessions ?

Kindly, provide me a detailed analysis of my charge details that I had consumed(day by day basis, activity by day) ?. I am unable to track my activity by day bill in billing dashboard.




· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

I cannot answer those questions, and have emailed the engineer working your support case.

0 Votes 0 ·