Icedtea-Web, unfortunately, does not pass all tests.
It is possible that some tests should be marked with the @KnownToFail annotation, however there are many tests that can fail depending on different settings.
(Note that all these lists are potentially incomplete!)
1 Test failures caused by browser configuration
The browser tests require an installed and supported browser to test in.
The browser tests require closing of the browser window repeatedly, which can cause failures in some tests. Make sure that your browser does not reopen tabs on reload while testing, it will cause inaccurate testing. If you do not have cookies enabled / your browser does not save cookies on reload, the cookie-related tests will likely fail.
There is a known issue testing in Firefox in recent versions, whereby the browser prompts that its entering safemode after being repeatedly closed, and will unfortunately cause tests to not run. Setting
toolkit.startup.max_resumed_crashes to -1 in Firefox's about:config should mitigate this issue.
2 Tests that always fail
Updated 2012/10/19. These reproducers have at least one @KnownToFail test method. This indicates a known issue that needs to be fixed.
If these tests pass before their fix, it most likely indicates a weakness in the test.
custom: AdditionalJarsInMetaInfIndexList signed: CacheReproducer ClasspathManifestTest signed2: MultipleSignaturesTest MultipleSignaturesTestSamePackage simple: encodingTestsĚŠČŘŽÝÁÍÉĚÉŘŤÝÚŮÍÓÁŠĎŽŹŇ SingleInstanceServiceTest
3 Tests that fail due to matching any exception
Tests can fail due to exceptions being thrown when the browser window is being closed. It is probably not feasible to cause all of the possible exceptions that can result to not occur. The browser tests that fail when any exception is thrown should thus be revised.
Updated 2012/10/19. Individual tests in reproducers that can fail due to matching any exception:
AppletTestSignedTests AppletTestSignedFirefoxTestXslowX AppletTestSignedFirefoxTest MultipleSignaturesTestTests multipleSignaturesTestHtmlApplet ClasspathManifestTest BrowserAppletRemoteTest MultipleSignaturesTestTests multipleSignaturesTestHtmlApplet InternalClassloaderWithDownloadedResourceTest launchInternalClassloaderWithDownloadedResourceAsHtmlAppletHack ParametrizedJarUrlTests parametrizedAppletInBrowserWithParamTest EncodingTestTest testEncodingTest4Iso88592 AppletTestTests doubleChrome
4 Tests that fail but need further investigation
Updated 2012/10/19. These tests have been observed to fail sometimes but are yet unexplained (If you have clues as to why, please add this information!):
DeadLockTestTest testDeadLockTestTerminated2 java.lang.AssertionError - stdout should contains: 3 Deadlock sleeping, didn't, so framework have consumed to much during termination testSimpletest1lunchFork java.lang.AssertionError - launched JVMs must be exactly 2, was 1 expected:<2> but was:<1> testSimpletest1lunchNoFork java.lang.AssertionError - launched JVMs must be exactly 1, was 0 expected:<1> but was:<0> InternalClassloaderWithDownloadedResourceTest launchInternalClassloaderWithDownloadedResourceAsJnlpAppletHack java.lang.AssertionError - Stdout should contains Good simple javaws exapmle but didn't CacheReproducerTest clearCacheUnsucessfully java.lang.AssertionError - Stderr should contain Can not clear cache at this time, but did not ParallelAppletsTest testParallelAppletsTest1x2e java.lang.AssertionError - counting... 8 was expected exactly 1 but was found 0 expected:<1> but was:<0>