I can run J-Meter tests in my AzDO pipelines, and I can see Http 404s and such in the .jtl I upload in my build artifact.
How can I get AzDO to report these as failures? I've made sure "Fail on standard error" is true, and "Continue on error" is false in the command for running my jmx file. I've also tried various Jmeter property settings, but nothing seems to help.
Here's the latest command I've tried:
D:\somecrazypath\apache-jmeter-5.2.1\bin\jmeter -n -t JMETER-POC\JMETER_Sample.jmx -l results-$(Build.BuildId)\JMETER_Sample_test_results.jtl -e -o results-$(Build.BuildId)\ -Jjmeter.save.saveservice.print_field_names=true -Jjmeter.save.saveservice.response_code=true -Jjmeter.save.saveservice.response_data.on_error=true -Jjmeter.save.saveservice.assertion_results_failure_message=true -Jjmeter.save.saveservice.assertion_results=true -Jjmeter.save.saveservice.output_format=csv -Jjmeter.save.saveservice.timestamp_format=yyyy/MM/dd -Jjmeter.save.saveservice.output_format=csv
-Jjmeter.save.saveservice.assertion_results_failure_message=true
-Jjmeter.save.saveservice.default_delimiter=|
Even though the resulting .jtl file has plenty of http errors, for some reason AzDO always interprets the run as a "Pass".
I need it to fail when there are failures.