Archetype Quality II

In my previous post, I proposed a high level diagram representing the archetype development processes. Pondering on this process further, I am generally happy with the high-level steps identified although the implementation step should really outside the scope of determining quality criteria for the archetype itself; thus four identified processes remain:

  • Requirements gathering & analysis:
  • Design & build:
  • Collaboration & verification; &
  • Publication, maintenance & distribution.

The next logical steps are to:

  1. Identify all of the sub-processes involved including the full set of activities required to create, produce, maintain and distribute the final archetype, including the people, materials, tools and procedures;
  2. Identify all of the points within these sub-processes at which we can define quality criteria; and
  3. Identify the individual quality indicators with which we can measure or asses each of the quality criteria.

In addition, I'm increasingly of the opinion that we should utilise some of the ideas outlined by Kalra et al to assist us to be systematic in our identification of the quality criteria and indicators. Kalra et al proposed high level quality criteria (or statements that demonstrate ideal practice) in the following categories:

  • Business Requirements
  • Clinical Requirements
  • Technical Requirements
  • Information Governance Requirements
  • Repository Requirements

I've modified these criteria categories to:

  • Design requirements/methodology - identification of quality criteria related to the design of an archetype where it impacts on any or all of these four processes. Examples include: inclusive design (maximal data set for universal use case as the ideal, except where over-inclusiveness makes the model impractical, confusing or unsafe; minimisation of overlap with other models; minimisation of duplication of data elements in multiple archetypes; direct terminology bindings and use of terminology reference sets; precision and detail; granularity; include another archetype fragment; mandatory vs optional; metadata; authorship and references/evidence;
  • Business requirements - identification of quality criteria ensuring that the archetypes support the required activities for data storage, exchange, querying, comparative analysis, secondary use, and knowledge-based activities such as decision support for any or all of these four processes.
  • Stakeholder requirements - identification of quality criteria ensuring that stakeholder requirements are represented in the final archetypes. The term stakeholder is deliberately used here, rather than only referring to clinicians, as while the archetype methodology enables clinicians to actively participate they are not the only domain experts who should be contributing to the model. The quality criteria will possibly predominantly focused on the archetype content, but the archetypes will not only be used to support direct clinical care but also a range of other purposes. For example, to support distributed care, clinical decision support, data aggregation and reporting, comparative data analysis, population health planning etc. All potential stakeholders who will either use the data created by the archetypes or implement them should be included in the stakeholder category.
  • Technical requirements - identification of quality criteria ensuring that the archetypes are technically 'fit for purpose'. Examples include ensuring that they conform to the openEHR reference model specifications; represent existing standards specifications or data sets, as appropriate; and technical attributes such as data constraints, occurrence/cardinality and null flavors are represented accurately.
  • Information governance requirements (including repository activity) - identification of quality criteria to ensure strong governance of each archetype through it's life-cycle and maturity within a repository, and processes to support the publication and distribution of meaningful sets of archetypes. Examples include  archetype version management; identification of relationships, including dependence, to other archetypes or knowledge artifacts such as terminology subsets; currency of archetype; endorsement or certification by professional colleges or jurisdictions; implemented use in real systems; implementation guidance; identification of  gaps/overlaps of coverage in sets of archetypes; design documentation; sample data; transformations; and overall quality and safety assessment.

[Note: I have specifically excluded establishing quality criteria for the Clinical Knowledge Repository itself - this should be a separate process which determines governance policy and process for a range of clinical knowledge artifacts within a repository application - including archetypes, templates, terminology subsets, transformation to other semantically equivalent artifacts and inclusion of models from other modeling approaches.]

Each of these categories of requirements will need to be considered in each of the four processes of archetype development. A candidate framework for consideration might therefore be represented in the following table:

Design requirements/ methodology Business requirements Stakeholder requirements Technical requirements Information governance requirements
Requirements gathering & analysis √ √  √  √  X
Design & Build √ √  √  √  X
Collaboration & Verification  √  √   √   √   √
Publication, Maintenance & Distribution X  X   √   √    √

Quality criteria and the indicators that will be used to measure or assess them should be identified for each area in the table represented by a tick (√).

Some simple examples of criteria that can be applied to the 'Collaboration & Verification' process have been outlined in a previous post,. demonstrating practical indicators to measure and assess criteria related to the collaborative review process, evidence-basis and 'fit for purpose'.

Reference: Kalra D, Tapuria A, Freriks G, Mennerat F, Devlies J (2008). Management and maintenance policies for EHR interoperability resources [36 pages] (Q-REC Project IST 027370 3.3). The European Commission: Brussels. (Last accessed May 28, 2011))