Recommendations for organisations referring to the specification (informative)

As the owner of the Test Assertions to which the present document refers, the HbbTV Association would like to draw the attention of organizations who plan to refer to the present document with a view to operating a conformance regime, ("Referring Organizations") to the following points.

1. HbbTV's goal is to achieve a commonly implemented standard across all its deployed markets and territories and as such strongly recommends organisations referring to this specification implement a common Test Assertions list, adding a level of tolerance to possible errors instead of removing applicable test assertions from the list. Creating different Test Assertion lists across multiple markets has the potential to introduce market fragmentation and is therefore discouraged.
2. 100% compliance of devices with the complete set of test assertions might not be achieved in the early years of a given specification version and therefore, it is advisable for Referring Organizations to adopt
1. A Test Challenge Process. This is an important element of any compliance process and HbbTV Association operates its own Test Challenge process for the assertions listed in this document
2. Temporary Tolerances, e.g. for performance requirements that a hardware implementation may not be able to meet
3. Waivers and Allowances, especially for non-critical aspects of a device or service implementation
3. HbbTV Association strongly advises and encourages that Referring Organizations to contribute to and feedback on issues with test assertions, or the specifications to improve them in the future. HbbTV Association operates its own Test Challenge process for the assertions listed in this document and is encouraging any Referring Organization to contribute to such test challenge process to help improve the reliability of the tests.
4. Even full compliance to any defined test suite does not guarantee full interoperability in any market implementation and further action is required to ensure services and implementations are fully interoperable:
1. Devices in the market will need to be tested against real available applications from on air services to improve interoperability.
2. HbbTV Association recommends service testing and interoperability plug-fests to augment compliance testing to the applicable test assertions.
3. Mandating any test list, several years ahead of service availability, will not guarantee interoperability at later dates.
5. Immediate conformance to any published test case cannot be achieved. Device Manufacturers will need to be given reasonable and sufficient time to implement and/or fix issues.
1. The industry accepted normal practice is that manufacturers require eighteen months lead time (aka sunrise period) to provide an implementation once a new test assertion has been published.

Conditions of publication

1. HbbTV keeps copyright of the Test Assertions referred in this document.
2. The HbbTV Association provides this document to support its Specification by outlining the scope of testing defined (by way of a published list of Test Assertions) for the benefit of users of its Specification. It is the objective of the Association in doing so to increase the interoperability of services and implementations and welcomes feedback on ways to improve interoperability.
3. The publication by the HbbTV Association of the test assertions is made on a voluntary basis to help and inform market participants. Fulfilment of all published applicable test assertions is good practice to improve interoperability but is not alone sufficient to ensure the compliance of equipment against a certain version of the specification.
4. HbbTV Association makes no warranty that the published test assertions are correct and necessary, and accept no liability in conjunction with published test assertions, nor with any subsequent testing or other certification activity or product based on the published test assertions that a third party may develop, apply to a third party, or be exposed to.
5. HbbTV Association's Published Test Assertions include only assertions for which an approved test case exists and it has proved to be a valid test case; other assertions may exist and not be included in the published Test Assertions. A test case may not be in the list of approved test cases because has been found to include defects and has been removed from the list (defined as a Challenge). Possible reasons for removal of tests cases from the list include, but are not limited to: IPR claims, difficulties for the market devices to pass the test cases or errors, or defects in the tests themselves.
6. The published test assertions may change from time to time at the initiative of the HbbTV Association at the sole discretion of the HbbTV Association, without any explanation or justification of its decisions, without any ex-ante or ex-post notice. HbbTV Association will accept no liability on the occurrence and possible implications of these changes. HbbTV will not make any judgements or adjudications on issues arising between Refering Organisations, Service Operators or Device Implementers. HbbTV Association has no expectations and does not enforce that legacy devices of any age are updated to keep up with such possible changes.
7. Certain categories of tests, e.g. those that apply to independent HbbTV specifications, have been designed in the knowledge that a mutually acceptable commercial relationship is likely to be required between service provider and device manufacturer and as such mandation of those test cases by any referring organisation does not override this principle.