Reproducer Failures

From IcedTea

Jump to: navigation, search

Icedtea-Web, unfortunately, does not pass all tests.

It is possible that some tests should be marked with the @KnownToFail annotation, however there are many tests that can fail depending on different settings.

(Note that all these lists are potentially incomplete!)

Contents

1 Test failures caused by browser configuration

The browser tests require an installed and supported browser to test in.

The browser tests require closing of the browser window repeatedly, which can cause failures in some tests. Make sure that your browser does not reopen tabs on reload while testing, it will cause inaccurate testing. If you do not have cookies enabled / your browser does not save cookies on reload, the cookie-related tests will likely fail.

There is a known issue testing in Firefox in recent versions, whereby the browser prompts that its entering safemode after being repeatedly closed, and will unfortunately cause tests to not run. Setting toolkit.startup.max_resumed_crashes to -1 in Firefox's about:config should mitigate this issue.

2 Tests that always fail

Updated 2012/10/19. These reproducers have at least one @KnownToFail test method. This indicates a known issue that needs to be fixed.

If these tests pass before their fix, it most likely indicates a weakness in the test.

custom:
AdditionalJarsInMetaInfIndexList

signed:
CacheReproducer
ClasspathManifestTest

signed2:
MultipleSignaturesTest
MultipleSignaturesTestSamePackage

simple:
encodingTestsĚŠČŘŽÝÁÍÉĚÉŘŤÝÚŮÍÓÁŠĎŽŹŇ
SingleInstanceServiceTest

3 Tests that fail due to matching any exception

Tests can fail due to exceptions being thrown when the browser window is being closed. It is probably not feasible to cause all of the possible exceptions that can result to not occur. The browser tests that fail when any exception is thrown should thus be revised.

Updated 2012/10/19. Individual tests in reproducers that can fail due to matching any exception:

AppletTestSignedTests
  AppletTestSignedFirefoxTestXslowX
  AppletTestSignedFirefoxTest
MultipleSignaturesTestTests
  multipleSignaturesTestHtmlApplet
ClasspathManifestTest
  BrowserAppletRemoteTest
MultipleSignaturesTestTests
  multipleSignaturesTestHtmlApplet
InternalClassloaderWithDownloadedResourceTest
  launchInternalClassloaderWithDownloadedResourceAsHtmlAppletHack
ParametrizedJarUrlTests
  parametrizedAppletInBrowserWithParamTest
EncodingTestTest
  testEncodingTest4Iso88592
AppletTestTests
  doubleChrome

4 Tests that fail but need further investigation

Updated 2012/10/19. These tests have been observed to fail sometimes but are yet unexplained (If you have clues as to why, please add this information!):

DeadLockTestTest
   testDeadLockTestTerminated2
     java.lang.AssertionError - stdout should contains: 3 Deadlock sleeping, didn't, so framework have consumed to much during termination
   testSimpletest1lunchFork
     java.lang.AssertionError - launched JVMs must be exactly 2, was 1 expected:<2> but was:<1>
   testSimpletest1lunchNoFork
     java.lang.AssertionError - launched JVMs must be exactly 1, was 0 expected:<1> but was:<0>
InternalClassloaderWithDownloadedResourceTest
   launchInternalClassloaderWithDownloadedResourceAsJnlpAppletHack
     java.lang.AssertionError - Stdout should contains Good simple javaws exapmle but didn't
CacheReproducerTest
   clearCacheUnsucessfully
     java.lang.AssertionError - Stderr should contain Can not clear cache at this time, but did not
ParallelAppletsTest
   testParallelAppletsTest1x2e
     java.lang.AssertionError - counting... 8 was expected exactly 1 but was found 0 expected:<1> but was:<0>
Personal tools