Know your data?

Recently I was reminded of some work we did a number of years ago. It involved a large research database, painstakingly collected over 20 years. The data was defined across a number of specialisations within a single clinical domain and represented in 83 data dictionaries stored in an Excel spreadsheet.

Data was collected based on a series of questionnaires, and we were told that successive data custodians had, true to human nature, made slight tweaks and updates to the questionnaires on multiple occasions. The data collected was actually evolving!

The only way to view the data was to open each of the 83 spreadsheets, painstakingly, one by one.

We were engaged to create archetypes to represent both the legacy data and the data that the research organisation wished to standardise to take forward.

So the activity of converting these data dictionaries - firstly to archetypes for each clinical concept, and then representing each data dictionary as a template - resulting in considerable insight into the quality and scope of the data that hadn't been available previously.

For example, the mind map below is an aggregation of the various ways that questions were asked about the topic of smoking.

Interestingly, what it showed was that no one individual in the organisation had full oversight of the detailed data in all of the data dictionaries.The development of the archetypes effectively provided a cross section of the data focusing on commonality at a clinical concept level and revealed insights into the whole data collection that was a major surprise to the research organisation. It triggered an internal review and major revision of their data.

Some of the issues apparent in this mind map are:

• A number of questions have been asked in slightly different ways, but with slight semantic variation, thus creating the old 'apples' vs 'pears' problem when all we wanted was a basket of apples;
• Often the data is abstracted and recorded in categories, rather than recording the actual, valuable raw data which could be used for multiple purposes, not just he purpose of the rigid categories;
• Some questions have 'munged' two questions together with a single True/False answer, resulting in somewhat ambiguous data; and
• Some questions are based on fixed intervals of time.

No doubt you will see other issues or have more variations of your own you could share in your systems.

And we have repeatedly seen a number of our clients undergo this same process, where archetypes help to reveal issues with enormously valuable data that had previously been obscured by spreadsheets and the like. The creation of archetypes and re-use of archetypes as consistent patterns for clinical content has an enormous positive impact on the quality of data that is subsequently collected.

And while harmonisation and pattern re-use within one organisation or project can be hard enough, standardisation between organisations or regions or national programs or even internationally has further challenges. It may take a while to achieve broader harmonisation but the benefits of interoperability will be palpable when we get there.

In the meantime the archetypes are a great way to trigger the necessary conversations between the clinicians, domain experts, organisations, vendors and other interested parties - getting a handle on our data is a human issue that needs dialogue and collaboration to solve.

Archetypes are a great way to get a handle on our data.

Preventing health data 'black holes'!

It really is not surprising that scientific data is disappearing all the time. But, oh, think of the value of this data - in terms of cost and knowledge. Irreplaceable. The original article from Smithsonian.com is here: The vast majority of raw data from old scientific studies may now be missing.

Some excerpts:

A new survey of 20-year-old studies shows that poor archives and inaccessible authors make 90 percent of raw data impossible to find.

...

When a group of researchers tried to email the authors of 516 biological studies published between 1991 and 2011 and ask for the raw data, they were dismayed to find that more 90 percent of the oldest data (from papers written more than 20 years ago) were inaccessible. In total, even including papers published as recently as 2011, they were only able to track down the data for 23 percent.

...

"Some of the time, for instance, it was saved on three-and-a-half inch floppy disks, so no one could access it, because they no longer had the proper drives," Vines says. Because the basic idea of keeping data is so that it can be used by others in future research, this sort of obsolescence essentially renders the data useless.

...

And preserving data is so important, it's worth remembering, because it's impossible to predict in which directions research will move in the future.

Seems to me that the openEHR approach to data definitions is an excellent candidate for preventing the health data 'black hole' too!

Non-proprietary, open data specifications are a key component for future-proofing irreplaceable clinical and research data.

Archetypes/DCMs MIA in SDOs

Curious to note that there is very little apparent interest in detailed clinical information models (DCMs) of any brand or flavour in the major Standards Development Organisations (SDO's) - they are effectively Missing In Action when compared to the likes of CDA and IHE profiles. The ISO 13972 DCM specification took a long and tortuous time to travel through the ISO TC 215 processes. Engagement with 13972 during its development, and from what I can observe now it is on the verge of publication has been rather sparse.  A further piece of work is now starting in ISO regarding quality criteria for DCMs but it also seems to be struggling to find an audience that understands it, or even cares.

I don't quite understand why the concept of DCMs has not been a big ticket item on the radar of the SDOs for a long time as it is a major missing piece of any standards-based framework. Groups like CIMI are raising awareness, alongside the openEHR work, so momentum is gathering, but for some reason it seems to keep a very understated profile compared to new opportunities like FHIR in HL7.

The work of messages, documents, profiles and terminologies are clearly important for interoperability, but standardisation of clinical content models working closely with terminologies can potentially make the work required to develop messages, documents, and profiles orders of magnitude easier.

Let me test a metaphor on you. Think of each message, document or profile as a sentence and each archetype or DCM as a word, a building block that is one component of each sentence. By focussing on the building a specific sentence, we are working backwards by trying to determine the components, and the outcome is still just that single sentence. However if we start by standardising the words/archetypes, then once they are stable it is relatively simple to construct not only one sentence for a specific purpose, but the potential is a much greater output in which many more additional sentences can be created using a variety of words in different combinations. If we manage the words (archetypes) as core building blocks and get them right, then we allow a multitude of possible sentences (messages/documents/profiles) to proliferate.

The ‘brand’ of archetype/DCM solution does not concern me so much as raising awareness that clinician-led, standardised clinical content is a significant missing and overlooked piece of the international eHealth foundations puzzle.

Standards Development?

https://twitter.com/psujorge/status/438909282499977216

Technical/Wire...Human/Content

In a comment on one of my most recent posts, Lloyd McKenzie, one of the main authors of the new HL7 FHIR standard made a comment which I think is important in the discourse about whether openEHR archetypes could be utilised within FHIR resources. To ensure it does not remain buried in the rather lengthy comments, I've posted my reply here, with my emphasis added.

Hi Lloyd,

This is where we fundamentally differ: You said: "And we don’t care if the data being shared reflects best practice, worst practice or anything in between."

I do. I care a lot.

High quality EHR data content is a key component of interoperability that has NEVER been solved. It is predominantly a human issue, not a technical one - success will only be achieved with heaps of human interaction and collaboration. With the openEHR methodology we are making some inroads into solving it. But even if archetypes are not the final solution, the models that are publicly available are freely available for others to leverage towards 'the ultimate solution'.

Conversely, I don't particularly care what wire format is used to exchange the data. FHIR is the latest of a number of health data exchange mechanisms that have been developed. Hopefully it will be one that is easier to use, more widely implemented and will contribute significantly to improve health data exchange. But ultimately data exchange is a largely technical issue, needs a technical solution and is relatively easy to solve by comparison.

I'm not trying to solve the same problem you are. I have different focus. But I do think that FHIR (and including HL7 more broadly) working together with the openEHR approach to clinical modelling/EHRs could be a pretty powerful combo, if we choose to.

Heather

We need both - quality EHR content AND an excellent technical exchange format. And EHR platforms, CDRs, registries etc. With common clinical archetypes defining the patterns in all of these uses, data can potentially start to flow... and not be blocked and potentially degraded by the current need for transforms, mappings, etc.

Archetypes: health data bridges

What do we want for our health data - silos of information models for different purposes or ones that bridge multiple use cases? From a series of emails shared on the HL7 Patient Care email list in the past few days...

Grahame Grieve (FHIR, HL7):

"Heather, you need to keep in mind the difference between FHIR and clinical models: it's not our business to say not to exchange data that people do have because some user in an edge case might not understand it. We define an exchange standard, not a clinical UI standard..."

and

"Heather, do not lose sight of the difference between a clinical standard for what care/records should be, and FHIR, which is an IT standard for how care records are."

My response:

"...I am concerned about developing another standard that you state clearly is only designed for exchange and not for what care records should be. If we are not designing to try to harmonise data requirements for health information exchange, how clinical care records are and how clinical care records should be, then we are building siloes of data structures again, that will require mappings and transforms ad infinitum. I’d hate to see us end up with a standard for exchange that can’t be implemented for persistence"...

If we end up with models for exchange, models representing current data in systems (whether or not they represent good clinical practice and models that are regarded as the roadmap for good data, then what have we got? Three sets of data models that perpetuate the nightmare of non-interoperability.

Our openEHR archetypes are attempting to bridge all of these. Use them in whatever context you choose - messages, document exchange, EHR persistence, CDS, secondary use, aggregation and analysis, querying etc. The 'secret sauce' is the use of a second layer of modelling - the template, that allows the correct expression of the archetype appropriate for the context of use.

Mappings and transformations are acceptable where we don't have any choice, especially with legacy data, but they open us up to vulnerabilities from errors, misinterpretation and ambiguity, concerns re data integrity and possible overt data loss. Given the choice, lets work towards creating high quality data that can be re-used in multiple contexts safely.

The Questionnaire Challenge

How should we model questionnaires in our health data? This is something that @ianmcnicoll and I have grappled with for years. We have reached a conclusion in recent times, and our approach, perhaps rather controversially, is not to model them! Yes, you read me right - as a general principle, don't archetype questionnaires.

Now of course there will be some situations where there are standardised and ubiquitous questionnaires and perhaps it will be reasonable to lock these down as fixed data elements and value sets in an archetype, and maybe even govern them within a CKM environment.

But... Consider the number of questionnaires out 'in the wild' at any point in time. Should each of these be archetyped?

Let's think it through. If there are 5000 questionnaires in the world (and clearly there are way more questionnaires than that out in the health ecosystem) then we would need a corresponding whopping 5000 archetypes. And, as we all know, no two questionnaires will be alike even if they have a common parent document - it is always human nature to 'tweak' each one for local use because 'our situation' is unique. It's just the way it is. The consequences are that any data captured using the myriad of archetypes, even though they may be similar, the data will not be interoperable. We will have a huge number of archetypes with a huge variation in content and intent - not a lot of upside from my point of view.

Another alternative could be to define a generic archetype pattern for a questionnaire, and re-use that. In fact we tried this back in 2007 with our work in the NHS - you can see a reasonable example here. The resulting questionnaire pattern is pretty simple and relies on using the templating layer to document the questions, and either templating or use of a terminology subset to record the answers. The equivalent FHIR resource appears similar in principle and intended use. This kind of pattern provides a common framework for a questionnaire but really doesn't give us a lot more interoperability for questionnaire data - the actual questionnaire content will vary enormously and the results can still be chaotic.

So, still somewhat clunky and awkward - not an elegant solution at all.

Then we got to thinking: Do we always need to actually record the questions and their answers in the EHR? This is a critical question. Sometimes the answer is yes, but most often I think we will find that we don't need to record the actual question and (often) check box response. What we really want to record is the outcome, the real health information meaning.

Think about the practical aspects of this...

Clinicians have a systematic questioning process for history taking - we all have a similar pattern but ask subtly different questions - resulting in zillions of permutations and combinations and levels of granularity. Questions could range from: "Have you had any abdominal symptoms?" to "Have you had any nausea, vomiting, reflux, abdominal pain, diarrhoea, constipation etc" to whatever combination is relevant for a given clinician in a given clinical situation. Many will ask the same kind of question slightly differently. Every resulting questionnaire will be slightly different.

And what do they record? They don't record each question and corresponding Yes/No answers in their paper health records. They record the positive responses or the relevant negatives, and/or maybe a quick note that there were no positive responses to systematic questioning about current symptoms, problems, past history, family history etc.

So we need to ask ourselves: Do we need to record the exact question, the potential alternative answers and the actual answer?

If the answer is yes, then it is a very good reason to lock in the questionnaire in an archetype, or at least a template.

If the answer is no, then what is the best way to record the relevant data - the relevant positives and the relevant negatives. For example, with abdominal pain - record the details about their diarrhoea and colicky pain in the right lower abdomen, PLUS that the patient has NEVER had an Appendicectomy.

So I'm suggesting that we need to record the positive presence of something identified in the questionnaire, for example a symptom, diagnosis or previous procedure, and the positive absence of related things. In this case, record the details about the diarrhoea and abdominal pain as the positive presence of symptoms using the Symptom archetype and positive exclusion of a previous Appendicectomy procedure in the Exclusion of Procedure archetype. We don't need to record the actual question and corresponding Yes/No answer.

So our current approach in Ocean is to use the software UI as the means to display the checklist or questionnaire, but only record in the electronic health record any relevant answers - both the positive presence of symptoms, signs, diagnoses, procedures and tests etc, and also the positive exclusion of any of these things - all using standardised archetypes.

Lets face it, it is not often that we ever go back to look at the raw questionnaire data again. So now we tend not to record the raw data (with some exceptions, where it may be required or useful) but use a transform so that a patient's or clinician's positive tick in a box for 'Past History of Epilepsy?' will be converted into a positive statement of 'Epilepsy' within an EHR, using the Diagnosis archetype. Any additional 'other' free text or 'details' or 'date of diagnosis' from the questionnaire can be captured using other relevant data fields for the Diagnosis archetype. The benefit from this approach is that this data can then be potentially re-used into the future as part of a comprehensive Problem List, not just buried as a ticked check box within a questionnaire from years ago, perhaps never to see the light of day ever again.

Consider the questionnaire as what it really is - just a clinical communication tool, a checklist. It is absolutely not the best means to record, persist and re-use good quality health data. What we really want to record in a consistent way are those critical pieces of health information in a formal archetype so that the data can be utilised for long term health records, decision support, exchange or analysis. Recording the check boxes answers from a questionnaire don't really do that job!

Stop the #healthIT 'religious' wars

I seemed to trigger an interesting discussion over on Grahame Grieve's blog when I posted the following question to him on Twitter. https://twitter.com/omowizard/status/432869451739185152

Grahame responded a la blog, but not really answering the essence of my question.

However, below is a great comment copied directly from the discussion thread, and submitted by Koray Atalag (from openEHR NZ). In it he has succeeded in expanding my twitter-concise thoughts rather eloquently:

"Hi guys,

I’m interested in getting these two awesome formalisms as close as possible. In what way – have no clue at the moment apart from existing mutual understanding and sympathy at a personal level. Well that’s where things start I guess  Ed Hammond once said, as he was visiting us in Auckland last year, convergence in standards is a must, mappings just become complex and hard to maintain. I kind of agree with that. Where I’d like to see openEHR and FHIR is really dead simple – Share what they are the best. Here is how I see things (and apologise if you find too simplistic): 1) Archetypes are THE way to model clinical information – anyone argue with that? 2) FHIR IS the way to exchange health information over the wire; modern, non-document/message oriented, heaps of interest from vendors etc. 3) openEHR’s Model Driven Development methodology can be used to create very flexible and highly maintainable health information systems. So this is a different territory that FHIR covers. Inside systems vs. Outside. A growing number of vendors have adopted this innovative approach but it’d be dumb to expect to have any significant dominance over the next decade or so.

So why not use openEHR’s modelling methodology and existing investment which includes thousands of expert clinicians’ time AND feed into FHIR Resource development – I’d assume Archetypes will still retain lots of granularity and the challenge would be to decide which fall under the 80%. I take it that this proportion thing is not mathematical but a commonsense thingy.

As with anything in life there is not one perfect way of doing health IT; but I feel that FHIR based health information exchange with propriety (and from the looks increasingly monolithic) large back-end HIS and openEHR based health information systems working with rich and changeable clinical data (note some Big Data flavour here will prevail in this rapidly changing environment.

So I’m interested – probably mapping as a starting step but without losing time we need to start working together.

The non-brainer benefits will be: 1) FHIR can leverage good content – I tend to think a number of Published or Under Review type archetypes have been in use in real life for a while and that’s probably what Heather was referring to by clinical validation. A formal clinical validation is a huge undertaking and absolutely unnecessary I guess unless you’re programming the Mars Colonisation Flight health information systems!

2) openEHR can learn from FHIR experience and use it as the means to exchange information (I haven’t yet seen EHR Extracts flying over using modern web technologies). There is an EHR service model and API but I’d say it is not as mature as rest of the specs.

3) Vendors (and the World for that matter) can benefit from 1) mappings; and then 2) better FHIR Resources in terms of more effectively managing the semantic ‘impedance mismatch’ problem. An example is medication – I’d assume an HIS data model for representing medication should have at least the same granularity as the FHIR Resource it ought to fill in (practically only the mandatory items). If any less you’re in trouble – but having a sound model will ease the HIS internal data model matching and help with deciding which part is 80% vs. 20%.

4) Needless to say vendors/national programmes using pure openEHR vs. FHIR + something else will benefit hugely. Even ones using CDA – remember some countries are using (or just starting as in New Zealand) Archetypes as a reference library and then creating payload definitions (e.g. CDA) from these. So having this openEHR – FHIR connection will help transition those CDA based implementations to FHIR. Interesting outcome 

5) I think in the long run vendors can see the bigger picture around dealing with health information inside their systems and perhaps start refactoring or rebuilding parts of it; e.g. clinical data repositories. An HIS with sound data model will likely to produce better FHIR instances and definitely have more capability for using that information for things like advanced decision support etc.

All for now…"

And then the conversation continues, including Thomas Beale from @oceaninfo and Borut Fabjan from @marandlab. I know of many others who have expressed a strong desire for openEHR clinical content and FHIR to be more aligned and collaborative.

FHIR seems here to stay - it is gathering fantastic momentum. The openEHR methodology for developing clinical content is also gathering momentum, including national program adoption in a number of countries and in clinical registries.

Chuck Jaffe (CEO of HL7) emphasised the need for collaboration between standards at last week's Joint Inter-Ministerial Policy Dialogue on eHealth Standardization and Second WHO Forum on eHealth Standardization and Interoperability at the World Health Organisation in Geneva.

So let's do it.

We all live in the real world and need to be more proactive in working together.

It is time to stop the 'religious wars', especially the long-time, tedious 'not invented here' argument between openEHR and HL7 and the ever-ongoing lets 'reinvent the wheel' approach to EHR content that occurs more broadly.

I call on all participants in the eHealth standards world to get the 'best of breed' standards working together.

Archetypes in the Real World

I've spent the past week in Ljubljana, Slovenia. Ian McNicoll (@ianmcnicoll) and I were been training clinicians about archetypes and clinical knowledge governance, ready for the launch of their national CKM. A highlight was a side trip to visit the state-of-the-art Paediatric Intensive Care Unit in Ljubljana. The electronic health record has been running there now for two years, with electronic processes gradually taking over. I was escorted by the clinician in charge of the ICU, Professor Kalan. The purpose of the visit – for him to meet someone who facilitated the archetypes used to run his EHR and for me to see our collective international archetype work implemented and used for real clinical purposes, largely under the expert clinical informatics guidance from Ian.

I was thrilled and a little taken aback, all at once. It is one thing to sit in an office researching clinical models and then to remotely collaborate with our international archetype community. But it is another to see real-time data being collected half a world away from home and knowing that we all had a small part in this, especially to support such critical care for a newborn baby. In the photo, above, you might just be able to spot a humidicrib surrounded by all of the equipment.

The majority of these archetypes were built by the international openEHR community for various projects and now utilised under the CC-BY-SA license by the EHR company to develop their clinical system. There are some local  archetypes in use as well – added for practical and pragmatic purposes - but these are very much in the minority. These same international archetypes are also being used in the EHR repository in the Northern Territory, Australia, and are underpinning their current work on shared antenatal care and hearing health programs. Soon this work is to be extended for renal failure and heart disease. And across more than 20 sites in Australia we have an infection control system that is using both archetypes and some that have been built specifically to support infection control activities and outbreak management. These shared archetypes are also underpinning work in UK, Brazil, Japan and Sweden.

Next week Ian and I are in Norway to support the Norwegian national archetype effort – training their clinicians and informaticians about archetypes, and especially governance principles at a national level.

There are now 5 CKMs in existence:

• the openEHR international CKM;
• Australia;
• City of Moscow;
• UK clinical community; and the brand new
• Slovenian eHealth program CKM.

The international CKM will continue to gather quality archetypes from all sources and coordinate international review and modelling activities. The intent is for this CKM to be the first port of call for those looking for an archetype.

The national-, organisation- or program-based CKMs will be focussed on supporting local health IT activities and will leverage the international pool of archetypes by a virtual 'read only' reference capability as well as hold specific local archetypes or modifications of the international archetypes that will support local implementation.

Above all the aim is to create high quality, computable, clinical content definitions that have been developed and ratified by the clinicians themselves. In turn this will support collection of good quality data that can be used for a variety of purposes – ranging from the health record itself; through querying and knowledge-based activities such as decision support; aggregation, analysis and research; and secondary use, including population health activities.

I have said it before, but let me say it again…

IT. IS. ALL. ABOUT. THE. DATA.

In the discussions about standards, the standardisation of data is usually missed.

Seeing this little baby in a humidicrib in amongst all of the 'machines that go beep' has invigorated me again.

Let's continue, and even accelerate, our collaboration on the development of archetypes. This will enable us to gather the data we need to provide the kind of healthcare our patients deserve.