The Internet Explorer Testcenter welcome page: Clarification & Corrections

Earlier this week, we published a number of new tests for web standardson our IE Test Center page, together with a harness to run them automatically in your browser as well as cross-browser pass rate statistics for these new tests. We quickly received web community feedback that the pass rate data in the first table was prone to misinterpretation.

What the Test Center Pass Rates Include:

The pass rates on the site are specific to only the 104 new test cases we just submitted to the W3C in conjunction with the first IE9 Platform Preview. As part of our regular involvement with the Working Groups across a number of W3C standards, we have identified a number of interoperability issues that are not yet covered by the standards body's existing test suite.  We should note that the test results are only for the Windows versions of all the browsers in the test results table.

Pass Rates and Vendor-specific Feature Implementations

These tests were written and submitted as standard test cases, not IE9 Platform Preview demo pages.  Thus, the cases don’t include any vendor-specific prefixes. By contrast, our IE9 Test Drive samples include extra code to support the vendor-specific versions of certain features (e.g. -moz-border-radius for Mozilla’s implementation of CSS3’s border-radius).  Standards test cases do not and cannot include proprietary properties, methods, or markup.

In the case of border-radius, Opera 10.50 and recent Google Chrome builds support the un-prefixed version of the property. The latest major releases of Firefox and Safari do not. The pass rates for the current Firefox and Safari versions simply reflect that they require vendor specific prefixes and do not yet support the web standard markup. This wasn’t meant to imply that those browsers don’t have support for rounded corners using some markup but simply reflects the fact that they require -moz/-webkit CSS declarations to render them. From an interoperability standpoint, web authors cannot use the same markup across these browsers to achieve the same results today.

Test Case Fixes

Commenters on this blog and beyond also reported some bugs in a handful of the test cases we submitted for review on Tuesday.  Thank you!  This is exactly why there is a test case review process.  Feedback on the test cases helps the web development community.  We welcome it. We have updated the tests to reflect the initial feedback to ensure the cases are useful to the web standards community. The known issues and related pass/fail updates are described below:

  • CSS3 Selectors
    • The :nth-child-selector and CSS comments test case assumes that comments can occur between the multiplier and the n term i.e. that in the (an+b) expression, a and n are separate tokens. This was a bug in the test case. The IE9 Platform Preview should ignore this selector and thus fail the test case. Firefox 3.6, Opera 10.50, Safari 4.0, and Chrome 4.0 all pass.
  • DOM Level 2 Style
    • The https://samples.msdn.microsoft.com/ietestcenter/domstyle/showdomstyletest.htm?./invalidMedia.htm test case verifies that @import rules inside an @media block are ignored but the case had a bug and checked for that result improperly. The first IE9 Platform Preview now fails this test.  Firefox 3.6 passes this test.  Opera 10.5 still fails the test. Safari 4.0 and Chrome 4.0 still pass this test.
    • Both the Syntax for backgroundRepeat and negative border width test cases had Javascript in them that mistakenly used an IE-only Javascript extension allowing elements to be referenced directly using their ID attribute value, which is supported for compatibility reasons.  The JavaScript in the test case was changed. The IE9 Platform Preview still passes both test cases.  Firefox 3.6 now passes both of these test cases.  Opera 10.50 still passes both tests.  Safari 4.0 still passes the backgroundRepeat test and still fails the negative border width test.  Chrome 4.0 still fails both test cases.

We thank the community for the feedback we’ve received so far. We have updated our tests, and the pass rate data, and submitted these updates to the standards body.

This is part of the standards process: a conversation leading to consensus. The one request we have is to please have these conversations within the W3C working groups instead of on various blogs.  We have a link to each working group’s mailing list in the table on this blog post.  We can have a healthy discussion about each test case in a threaded, archived format where everyone interested can participate and anyone can follow-along.

What’s Next

As we continue developing and testing IE9, we will continue submitting our test cases to the W3C and the wider community. Conformance and interoperability can only be measured and achieved with the help of solid, complete test suites. Modern browsers should be able to demonstrate their conformance to web standards.  Test suites that identify cross-browser incompatibilities help all browsers make progress.

Jason Upton
Test Manager