How to get AzDO Build Pipeline run to fail when there are http 400-range errors in my J-Meter tests

Abrams, Paul PJA [NC] 21 Reputation points
2021-09-23T18:32:43.203+00:00

I can run J-Meter tests in my AzDO pipelines, and I can see Http 404s and such in the .jtl I upload in my build artifact.
How can I get AzDO to report these as failures? I've made sure "Fail on standard error" is true, and "Continue on error" is false in the command for running my jmx file. I've also tried various Jmeter property settings, but nothing seems to help.
Here's the latest command I've tried:

D:\somecrazypath\apache-jmeter-5.2.1\bin\jmeter -n -t JMETER-POC\JMETER_Sample.jmx -l results-$(Build.BuildId)\JMETER_Sample_test_results.jtl -e -o results-$(Build.BuildId)\ -Jjmeter.save.saveservice.print_field_names=true -Jjmeter.save.saveservice.response_code=true -Jjmeter.save.saveservice.response_data.on_error=true -Jjmeter.save.saveservice.assertion_results_failure_message=true -Jjmeter.save.saveservice.assertion_results=true -Jjmeter.save.saveservice.output_format=csv -Jjmeter.save.saveservice.timestamp_format=yyyy/MM/dd -Jjmeter.save.saveservice.output_format=csv
-Jjmeter.save.saveservice.assertion_results_failure_message=true
-Jjmeter.save.saveservice.default_delimiter=| 

Even though the resulting .jtl file has plenty of http errors, for some reason AzDO always interprets the run as a "Pass".

I need it to fail when there are failures.

Not Monitored
Not Monitored
Tag not monitored by Microsoft.
36,001 questions
{count} votes

2 answers

Sort by: Most helpful
  1. KarishmaTiwari-MSFT 18,447 Reputation points Microsoft Employee
    2021-09-24T23:32:18.523+00:00

    Microsoft Q&A currently do not support Azure DevOps.

    The Azure DevOps team and community are active and answering questions on : https://developercommunity.visualstudio.com/spaces/21/index.html

    Please post your question there and the experts will help you there.

    You can also post your question here: https://stackoverflow.com/questions/tagged/azure-devops. Thanks

    0 comments No comments

  2. TravisCragg-MSFT 5,676 Reputation points Microsoft Employee
    2021-09-25T00:56:25.593+00:00

    Azure DevOps is not supported on Microsoft Q&A. The best place to ask these questions are on Stack Overflow under the azure-devops tag.

    0 comments No comments