Insights and Lessons Learned

You are viewing an archival website from the original trustmark pilot in 2013-2016. If you are looking for more recent content about the trustmark framework, please visit the Trustmark Initiative website.

This page summarizes the insights that we have gained through our pilot project about the concepts of a trustmark framework and a trustmark marketplace.

1. Monolithic trust frameworks are ill-suited as a basis for achieving wide-scale trust and interoperability within the Identity Ecosystem. The scope of requirements for trust and interoperability varies greatly across trust frameworks (e.g., protocols, attributes, participating communities of interest, capabilities, services, policies, legal agreements, etc.), which means that on a broad scale, there is no single framework that meets the requirements of all participants. Even within a single trust framework, not all participants have the same set of trust requirements. In addition, trust frameworks’ requirements tend to evolve over time. As a result, onboarding into a monolithic framework is not only complex and costly, but also disadvantageous to early adopters, which often must “re-onboard” into the framework as it evolves.

2. The concept of “identity interfederation” — establishing trust and interoperability between the members of multiple identity federations by way of agreements between the federation operators — is limited in its applicability due to several unavoidable challenges. Here are a few reasons why.

No two trust frameworks are the same, which means that trying to map one framework’s trust and interoperability requirements onto another framework is hard.
Constant evolution of trust and interoperability requirements within trust frameworks further complicates this mapping process.
Inter-federation arrangements create transitive trust (i.e., trust relationships that flow through one or more intermediaries), and transitive trust is diluted trust.
Under law, contractual obligations between a federation participant and a federation operator usually cannot be transferred or assigned to 3rd parties, which makes inter-federation legal agreements difficult or impossible to execute.

3. For the purposes of communication and collaboration, no community of interest within the Identity Ecosystem is an island unto itself. Police need to communicate with first responders. First responders need to communicate with health care workers. The insurance industry needs to communicate with the finance industry. Etc. So regardless of where we draw the boundaries between communities of interest, there is always additional value to be gained by expanding that boundary and enabling members of a community to talk with a broader set of participants who may belong to another community. This means the traditional idea of having one identity framework or federation per community is of limited value in the long run. A better long-term strategy is to design a trust and interoperability architecture that erases the traditional boundaries of identity federations and enables trust and interoperability to emerge at the broadest possible scale.

4. Creating wide-scale trust and interoperability within the Identity Ecosystem requires a framework that can help us address and solve the problems associated with monolithic trust frameworks and identity federations. Based on the known problems with monolithic frameworks, we believe that a viable way forward must have the following characteristics.

It must allow for trust frameworks to be componentized, thereby enabling “partial interoperability” to exist between trust frameworks where it is appropriate.
It must allow for trust frameworks to be defined, evolved over time, and extended where necessary via the assembly of modular trust and interoperability components. (These modular components are TDs.)
It must allow for the organic growth of a trustmark marketplace that can exist alongside existing monolithic trust frameworks. Over time, this marketplace would relieve Identity Ecosystem participants (IDPs, SPs, etc.) from having to rely on a single 3rd party for all aspects of trust. It would also allow for the emergence of 3rd party trustmark provider businesses that can specialize in the issuance of specific trustmarks, according to their capabilities. Once established, such a marketplace could become a thriving, competitive arena that responds to and serves the evolving trust and interoperability requirements of the Identity Ecosystem in a manner that is much more agile than today’s rigid, monolithic trust frameworks can provide.
It must allow for the proliferation of a scalable model for establishing the necessary legal agreements and contracts that underpin trust in the Identity Ecosystem – and we believe our trustmark legal framework can meet this requirement.

5. The trustmark framework may be an elegant concept for encouraging the proliferation of componentized trust in the Identity Ecosystem, but defining the “right” trustmarks and the “right” level of trustmark granularity is critical for real-world adoption. Obviously, for the trustmark concept to work, we need to identify and develop trustmarks that are useful and reusable across many communities. But in addition, the size of the trustmarks (i.e., the scope of their requirements) matters. If trustmarks are too coarse-grained, then the trustmark concept will tend to collapse into a small set of large trustmarks that closely resemble today’s monolithic trust frameworks, and we won’t achieve the key benefits of reusability and wide applicability. Conversely, if trustmarks are too fine-grained, then the trustmark marketplace could become too complex as we witness the proliferation of a very large number of trustmarks. It seems that the “right” level of trustmark granularity can be characterized as “just fine-grained enough to allow us to precisely identify and codify the overlaps between trust frameworks and their source requirements, but no finer”. These areas of overlap can then naturally evolve into a set of “common” trustmarks that are broadly recognized and accepted by many or all trust frameworks. Each trust framework can then adopt and reuse the appropriate set of common trustmarks, and supplement it with a set of “custom” trustmarks that meet its unique needs.

6. Trustmarks can be a helpful tool for managing the complexity that results from the growth and evolution of trust frameworks’ requirements over time. Growth and evolution of trust frameworks is inevitable throughout the Identity Ecosystem, and as trust frameworks evolve, their participants face the challenges of adapting to meet new requirements and managing the increased complexity that is often created by the new requirements. But for a trust framework that is already componentized, evolution and adoption of new requirements can be as simple as the introduction of a few new trustmarks or the deprecation of a few existing trustmarks. Componentization and trustmarks therefore enable trust framework providers to minimize the complexity and confusion that naturally arises due to evolution and change over time.

7. Significant overlap exists between the requirements of many trust frameworks and other source policy documents that are used within the Identity Ecosystem. We have identified overlaps in several areas, including technical specifications for interoperability, cryptographic and security requirements, and privacy requirements.

8. There are no obvious limitations on the scope of requirements that can be represented using the trustmark framework. Throughout our work so far, we have not yet encountered a well-defined, concrete requirement to which a trustmark cannot be applied. And conversely, whenever we have encountered a requirement or concept to which trustmarks do not seem applicable, we have found the requirement to be ill defined or poorly understood. The obvious corollary to this lesson is that for a trust framework requirement to be reusable within other trust frameworks, it must first be well defined and clearly expressed in writing.

9. In the near term, adoption of trustmarks within the existing identity software infrastructure will require the development of “bridging” technologies that enable trustmark-based trust and interoperability information to be leveraged by legacy products that do not natively support trustmarks. As with any technology, an effective implementation strategy requires consideration of the “last mile”, and in a trustmark framework, the “last mile” is the technique or set of techniques by which trustmarks are bound to existing service endpoints, such as SAML IDPs and SPs.

10. There is substantial overlap in trust and interoperability requirements across communities and across trust frameworks, e.g., FBI CJIS Security Policy vs. NIST 800-53 Controls vs. ISO 27001/27002 Controls. Cross-comparison of requirement sets and decomposition of requirements into generic reusable modules is inherently difficult and messy, and tends to result in trustmarks of small granularity. The number of trustmarks increases in proportion to the number of requirement sets considered. But by doing the cross-comparison work, we can illuminate the precise differences and minimize the effort required to participate in multiple communities. An alternative to rigorous cross-comparison is to simply define coarse-grained trustmarks for specific requirement sets (e.g., FICAM IDP Certification, Cloud Security Alliance Audit, FBI CJIS Security Audit, etc.) to “unlock” previous audit and assessment work. This process is simpler and faster than rigorous cross-comparison, and may be a valuable strategy to encourage the proliferation of trustmarks, but it offers little benefit in terms of minimizing the effort required to participate in multiple communities.

11. The trustmark concept provides a framework that communities can use in thinking rigorously about how to define and structure (componentize) their trust and interoperability requirements, particularly with respect to other communities’ requirements, and the evolution of requirements over time. But the trustmark concept, and the broader goal of cross-community identity trust and interoperability, will have limited success unless communities begin to embrace certain “best practices” in defining their requirements. Some of these best practices are as follows. (This may not be a complete list.)

Recognize the likely non-uniqueness of most of your community’s trust and interoperability requirements, and adopt requirements from other communities or existing, well-accepted standards where possible.
Recognize the difference between “common” (non-unique) requirements and “community-specific” (unique) requirements, and avoid conflation of these two categories of requirements.
Recognize and clearly define the various “conformance targets” (e.g., IDP, SP, AP) that exist within the community, and avoid conflation of requirements for multiple conformance targets.
Recognize the need for requirements to evolve over time, and structure requirements documentation accordingly.
Practice modular documentation of requirements across all these dimensions.

12. Trustmarks need not represent an entire trust framework to provide value. It is possible for a set of widely accepted, trustmark-defined requirements to grow organically over time within a wider ecosystem of requirements. For example, the health community could begin using trustmarks by adopting an “informed consent” trustmark and promoting its use within the context of existing trust frameworks and relationships. Additional trustmarks could then emerge over time and become well accepted within the health community, as community members slowly replace “custom” trust components with common components that offer a suitable replacement.

13. The use of trustmarks as a tool for rigorous assessment of trust criteria can help to illuminate the extent to which existing trust relationships are not so rigorously defined. For example, Agency A may claim to require trustmarks X, Y, and Z, and it may trust Agency B based on a longstanding business relationship even though Agency B does not have trustmarks Y and Z. We use the terms “reputational trust” and “residual risk” to characterize the trust decision that Agency A has made and the inherent risks that accompany that decision. Our experience through this pilot project tells us that reputational trust and the acceptance of residual risk is widespread. In the context of this reality, we believe that trustmarks offer a useful framework for precisely quantifying and defining an acceptable amount of reputational trust and residual risk, as well as defining a formal roadmap towards decreased residual risk over time.

14. With respect to trustmark assessments, the primary benefit of trustmarks is in their potential for reuse of the assessment results. There is no real advantage or efficiency gained by initially assessing an agency’s policies against a set of trustmarks vs. against a comparable monolithic policy. The benefits only come when the same agency is to be assessed against another trust framework or trust policy for which the requirements are also composed of an overlapping set of trustmark definitions.

15. Even in our pilot of relatively limited scope, we defined over 600 TDs corresponding to the trustmarks that an organization can receive. The number of TDs may grow quite large as we begin to consider and cross-compare an increasing number of requirement sets representing the trust needs of various communities. This scale of TD “proliferation” underscores the need for appropriate software tooling to help manage the inherent complexity of all aspects of the trustmark framework, from TD and TIP publication to trustmark assessment to trustmark usage by trustmark relying parties.

16. In addition to performing many trustmark assessments, a trustmark provider must also issue and manage many trustmarks. This includes cryptographic signing of trustmarks, publication of trustmarks at well-defined locations online, maintenance of appropriate trustmark status data (e.g., to notify affected parties if a trustmark becomes revoked), and managing the “reassessment” cycle for renewal of trustmarks that have expired. As noted previously for trustmark assessment, these additional trustmark lifecycle management tasks are highly impractical, even at a small scale, without appropriate software tooling.

17. Trustmark framework software tooling would be impractical to develop and maintain without an appropriate amount of regularity and standardization within TDs and trustmarks. This fact underscores the importance of the TFTS, which provides normative specifications that govern the structure of all significant trustmark-related artifacts.

18. It is common for a trustmark assessment to determine that an agency meets some, but not all, of the assessment criteria for a trustmark. This reality presented us with two options: (1) refuse to issue a trustmark except in cases where an agency meets 100% of the assessment criteria, or (2) develop a framework for handling cases of less-than-full conformance via issuance of “provisional trustmarks” with “exceptions”. We chose the latter option, with the following caveats.

To receive a provisional trustmark, an agency must meet “most” of the assessment criteria for the trustmark, and any unmet criteria must be deemed as relatively insignificant to the meaning of the trustmark.
A provisional trustmark is valid only for a relatively brief period of time, e.g., six months.
A provisional trustmark must include notes documenting any exceptions, so that trustmark relying parties can easily determine whether to trust the provisional trustmark.
The trustmark recipient must agree to resolve the problem and undergo reassessment for a non-provisional trustmark ASAP.

Even though this strategy introduces some ambiguity into the trustmark framework, we feel that it will help in the long run, as it favors practicality over perfectionism and also provides a concrete framework for documenting where agencies need to improve their operations.

19. It is relatively common, when doing a trustmark assessment, to discover one of the following conditions for any given requirement.

The agency does not know whether it currently meets the requirement as stated, but it can provide audit results documentation to demonstrate that is meets a similar requirement.
The agency believes that it likely meets the requirement, or can demonstrate that it meets the requirement operationally (e.g., via testing of live systems), but does not have formal documentation in place stating that it meets the requirement.
The agency has formal documentation in place stating that it meets the requirement, but cannot demonstrate that it meets the requirement operationally.

We expect that over time, as agencies become accustomed to undergoing periodic assessments, they will adapt and be better prepared for the assessment process.

20. Trustmark assessment mostly moves at the pace of coordination and cooperation. If a TD’s assessment process is clearly defined, then there should be little or no ambiguity for the assessor, and this should cause the assessment process to flow smoothly and quickly. But the assessor is often at the mercy of the assessed; if an organization undergoing an assessment does not value the assessment and prioritize the assessment accordingly, then the assessment process will drag.

21. Our trustmark legal framework appears to be acceptable to organizations, at least within the law enforcement community. We make this claim based on having executed trustmark recipient agreements with 11 agencies to date. Each executed agreement with an agency pertains to all trustmarks issued by GTRI to that agency.

22. Our trustmark framework separates the concept of trustmark issuance to an agency from trustmark binding to an agency’s operational endpoints; this separation is necessary to give trustmarks greater operational modularity and potential for reuse with multiple endpoints.

23. Our technical approach to binding trustmarks to operational endpoints is based on the need to bind trustmarks to operational endpoints (e.g., SAML 2.0 metadata Entity Descriptor objects) in a manner that does not preclude existing, non-trustmark-aware tools (e.g., SAML endpoint implementations) from processing them, yet also allows for the development of trustmark-aware tools for processing trustmark-bound endpoints more intelligently in the future.

24. While we first conceived of our trustmark framework concept as a potential solution to the scalable trust and “inter-federation” challenge, we have now identified at least five distinct “use cases” or applications of the trustmark framework. These five use cases are as follows.

Scalable Trust for Federated Identity — This is the original use case for which we designed the trustmark framework, and the basic use case that we explored through this project.
Trust and Assurance for High-Value Attributes — This use case represents a “corollary” to the first use case of scalable trust for federated identity. In this case, trustmarks can be used to underpin trust in IDPs and APs that assert high-value attributes about end-users. In this application of the trustmark framework, any attribute conveyed by an IDP or AP would have a corresponding TD that defines precise criteria and conditions for asserting the attribute. Upon receiving a signed assertion or claim from an IDP or AP, an RP would first check to ensure that the IDP or AP possesses a trustmark to indicate that it has been assessed for proper assertion of the attribute. This level of rigor for attribute trust may not be necessary for many attributes, but in certain high-value and high-risk applications (e.g., in law enforcement and counter-terrorism information sharing applications), it may be advantageous to implement such a rigorous-yet-flexible infrastructure.
Information Sharing and Safeguarding — The concept of this use case is that the trustmark framework can be leveraged to enable rapid, agile information sharing and safeguarding relationships among agencies that need to securely share data, e.g., within the ISE community. The ICIF Information Sharing Agreement Builder Capability that we discuss in Section 3.7 represents our first attempt to leverage the trustmark framework for this use case.
Assessment, Certification, and Accreditation Management — The world of security assessment, certification, and accreditation is becoming increasingly complex and increasingly rigorous, particularly for U.S. government agencies and other governments (e.g., state and local agencies and others), due in part to the rise in adoption of the NIST Risk Management Framework (RMF). The RMF is a robust and flexible model for designing, deploying, and managing the risks inherent to information systems within today’s macro environment, in which information systems are proliferating, the need to engage in trusted information resource sharing across organizational boundaries is increasing, and cyber threats are a constant. The RMF acknowledges that subtle differences must exist in security controls across different use cases, due to differences in the risks inherent in those use cases. It also acknowledges the need for continuous monitoring of information systems and the organizations that deploy and manage those systems, to constantly guard against new threats that may emerge over time. These two traits – acknowledgment of differential requirements across systems based on differential risks, and acknowledgment of the need for continuous monitoring – set the RMF apart from other approaches to security and risk management. Despite its inherent positive qualities, the RMF also introduces new challenges, as it exposes certain inherent complexities of effective security and risk management in today’s world. Organizations that wish to embrace and leverage the RMF must manage and track their RMF lifecycle processes across all their systems, as well as over time. In addition, organizations with hierarchical management structures and security and risk management policies that “flow-down” from higher levels to lower levels within the organization (or from other policy sources external to the organization, e.g., prevailing laws and statutes) must account for the added complexities introduced by those factors. In other words, faithfully implementing and lifecycle managing the RMF and all of the security controls that apply to various types of information systems impacted by the RMF can be a significant burden, especially in large enterprises with many spans of control. These complexities and challenges underscore the need for appropriate tools and frameworks to help organizations manage the RMF process. We believe that the trustmark framework can help fill this need, and we have begun to explore opportunities to pilot the trustmark framework within the context of the RMF.
Trust Negotiation and Conveyance in the Internet of Things — The Internet of Things (IoT) represents something of an emerging “grand challenge” for scalable peer-to-peer trust. The IoT presents many of the same basic challenges that exist in the Identity Ecosystem today (e.g., lack of centralized oversight, heterogeneity of trust policy criteria, etc.); however, due its device-centric nature and project exponential growth in the coming years, the sheer size of the IoT may soon dwarf the size of the Identity Ecosystem in terms of the number of “trusted” entities. The problem of “trust negotiation” between IoT devices is a significant challenge that must be overcome to enable certain IoT use cases, and we believe that the trustmark framework is well poised to provide the necessary infrastructure for addressing this problem. During 2015-2016, we explored this concept as part of an internally funded GTRI research project. Our initial findings indicate that with the right software tooling and select modifications to its design, the trustmark framework could someday fill this need. We plan to pursue additional opportunities in this area as they arise.