Is your quality system fact or fiction?

You might hear quality and compliance people say things like "if it is not documented, then it does not exist" but do you hear them say "well, it is obviously documented, but does it exist?" as often?

The lack of documentation is a big problem in many enterprises, but I would say that the amount of documention that does not reflect the reality is an even bigger, though more hidden, problem. It is more convenient to look for a specific document or to review an existing document to see if it conforms to certain requirements or standards than to go around asking each and everyone if the contents of a document reflects their reality. I would say that accuracy is the quality which is hardest to measure and ensure.

Estimating the Value of Content

If you empty your pockets and there's a lighter, a pen, the keys to your home and your wallet - what would you care about putting in a safety deposit when you go swimming?

Estimating the value of content is key to efficient content management. Depending on how valuable the content is, it needs to be managed differently. Content that is estimated to be very valuable for the business should be considered as assets and managed with the same care as other kinds of assets. Content that is more or less worthless should be terminated - if it worth the effort to terminate it. Simply speaking.

Seth Gottlieb shares some very sharp insights on this subject in his post "CMS Business Case":

"There has been an enormous amount of writing and discussion about building a business case for a CMS and I don't have much to add other than to say that most of what I have heard is totally wrong ...//...In my opinion, the business case discussion should be around the content itself - not the technology used to manage it"

"At cmf2007, Bob Boiko's keynote talked about how we are not yet in the information economy because we have a hard time determining the value of content and the markets for trading information are primitive. Content managers are put into the subservient role of having to post everything that they are given. I would tend to agree with him. I do not feel like companies are any better at deciding what content to keep than the parent of a prolific three year old artist. In fact, I feel like the parent has the edge because he has a finite amount of refrigerator door space."

Very well said by both Seth and Bob Boiko.

Finally, here are some of the posts on this blog related to the subject of content value:

• Perspectives on the Value of Content
• 8 Insights about ECM for the CIO
• Looking At Content Quality From Different Perspectives
• A Process Perspective On Content Production
  

Looking At Content Quality From Different Perspectives

"One of the most obvious, yet surprisingly overlooked, components of a search strategy is the creation of quality content...//... The key question, however, is what is "content?" And what are the characteristics of good content?" (Search Engine Watch)

What exactly is content quality? There is much talk about high quality content, but very few dare, care or know how to define what they mean with it. Or, when talking about high quality web content, they point to what search engines define as high quality content, which does not at all need to be perceived as high quality content by your intended users.

I wrote a post some time ago about measuring the quality of a content product and pointed to three main content quality attributes – accuracy, timeliness and completeness. I put relevance as a forth quality attribute since I discussed content quality from a producer perspective. However, from a user (as apposed to producer) perspective, relevancy must be said to be the most important quality attribute of all. There are plenty of additional attributes to consider if I am to cover all dimensions of content quality, but it is fairly logical that - for a user- the content must:

1. be useful for the user’s objective or task (e.g. relevant)
2. be available when the user needs it (e.g. timely)
3. include all that the user needs to know, no more no less (e.g. complete)
4. come from a source that make you trust that it is true (trustworthy)

Content that is not perceived as relevant by the user is practically useless for the user. From a content producer perspective, content relevancy means knowing exactly what the user wants, something which is very hard to figure out. Even if you are both the producer and the user of the content, it is hard to know what will be relevant to you simply because your objective and usage context (the usage scenario) might differ from time to time. Given that it is practically impossible for a content producer to know what each and every user will consider as relevant in each and every usage scenario, it is more usable to talk about typical users, or roles, and typical usage scenarios. I recently read a nice post by Aurora Brown called “Redefining Quality Content For The Web” where she exemplifies how the perception of content quality changes with what perspective you perceive it from.

“From a user's perspective, quality (or my preferred term, valuable) content, whatever the medium, is characterized by one essential factor: it fulfils the user's needs at the time they're looking for.”

“From a search engine's perspective, quality content seems to be determined by the most on-topic (relevant) and trusted material in the online community. Relevancy is affected by the number of links to a page's content (like 'votes' for the page), the content's significance, how well the search engine algorithm understands the meaning of the page (semantics) and how well the content matches the searcher's topic. Trust is the other part of the equation, and is determined by the types of sites that link to a webpage.”

"From a web owner's, web masters, or website owner's perspective, quality content is simply content that garners links, offers value to users, is properly optimized and draws traffic that can be used for brand recognition or monetization.”


The point to make here is that content producers need to focus on defining (and measuring) what content quality really is so that they can ensure that the content they provide to their intended users is perceived as high quality content, and that this requires considering many different perspectives when doing so.

Approaching Information Quality

Information quality is often a neglected topic, both from the business side and the IT side. Perhaps, it is because information quality is a complex concept that includes many management techniques and quality practices.

Neglecting information quality issues may e.g.:

• lead to extra time and resources to manage and resolve information assets
• initiate a loss of credibility in services
• delay deployments of new applications and integrations
• be an originator of compliance problems
• cause customer and partner dissatisfaction

When starting a quality effort it is often necessary to manage issues from two sides - existing quality issues needs to be sorted out and the occurrence of new issues needs to be minimised.

A quality study often reveals many existing and often serious quality issues. One of the things to deal with is simply where to start hence the actions need to be prioritised. A successful approach can be based on an evaluation of information assets (where quality levels are compared to business importance) and a shortlist of actions (selected by comparing business benefit to business effort).

Realising the actions often requires buy-in from sponsors and stakeholders. The approach is more solid if it can be supported by a business case. The business case should clearly state increased costs and risks as well as possibly lower revenue and confidence. The quality of the business case itself is reliant on e.g. how well the information assets are linked to IT services and business processes.

Minimising the occurrence of new issues may be done at different levels depending on the ambitions of the sponsors and stakeholders. Below is a list of some common actions to consider for the long run:

• Create a information quality strategy to define a practical quality approach and a realistic vision
• Establish a governance of information quality to unite business and IT professionals
• Define appropriate metrics & tools for business focus and results
• Assess master data and information architecture to deal with inconsistencies and sharing problems
• Realise an information quality process and instructions to enable continuous improvements
• Manage awareness and change concerning quality work to secure implementation

A common misconception is that quality is owned by and the responsibility of one group. However, a successful quality effort should lead to the realisation that quality is, and should be, the responsibility of everyone.

The Need For Content Governance

I think most people who have worked with content management in an organization have experienced the following:

• Content products are developed and produced but never or rarely used
• The content that is produced is not always accurate, complete and/or timely when released
• Content defects are not discovered, managed and resolved in a structured way once the content has been released
• The efficiency of content products is not measured
• Content products becomes outdated but are not archived or retired
• Content is not maintained and updated
• Content products do not comply with branding standards and guidelines
• Content does not comply with laws and regulations
• Content falls into the wrong hands and get
• Taxonomies grow wild and become inconsistent and more or less unusable over time

A lot of the content within an enterprise is of great importance or even critical to the enterprise and should therefore be managed with the same respect as other kinds of assets. The reason is, simply speaking, that bad content quality can hurt the enterprise badly. Despite this, many content workers seem to put their faith into technology and simply assume that content will automatically remain complete, correct, fresh and relevant once they have produced and released it. Or even worse, they don't care what happens to it after it has been released the first time.

I have experienced this many times. Often it has to do with the fact that the people that produced the content are no longer responsible and accountable for the content. The content was produced and delivered by a project and then it was left to its own devices once the project closed. If the project deliverables were handed over to someone before the project closed, it was probably handed over to someone in an IT maintenance organization, an organization that was set up to maintain and govern IT solutions – not content. Having the wrong person responsible for managing the content is often worse than having no one responsible for it, because then people tend to believe it is managed when it is not.

So how do you avoid this? Well, first of all it is absolutely essential to have disciplined and skilled people that have the right attitude. This is by far the hardest part. If you are failing here, especially if people's attitude is wrong, then your chances of succeeding are small if not minimal. But if you manage to get disciplined and skilled people with the right attitude by your side, then it is just about establishing and implementing a functioning governance model and getting the right processes, resources, tools and infrastructure in place to manage the content throughout its life-cycle.

When discussing governance models and frameworks, it is important to understand that there is no single model or framework that will suit all organizations. Besides that no organization and enterprise is identical to another, it is often hard to draw distinct lines between different areas of responsibilities. For example, it is hard to draw a distinct line between the management of internet-specific content, intranet-specific content, documents and digital assets. Consequently, it is a better approach to define what needs to be done and what resources and roles are needed to do the work instead of finding a model for how to organize these roles. It is also pretty simple to identify what needs to be done to get well functioning governance, such as:

• Management support
• Policies, standards and guidelines to follow
• Functioning processes
• Clearly defined roles and responsibilities
• Skilled and disciplined people with the right attitude
• Supporting tools and infrastructure
• Routines for auditing and follow up

It is nice to see that so many organizations are getting better and better at governing their IT business. Now they just need to set the same level of ambition for governing their content.

Some Thoughts About the Quality of Online Content Products

According to my experience, many companies do not treat online publishing with the same seriousness as they treat offline publishing (such as CD-ROM & DVD productions) and print publishing. And I believe that the answer is simple - it can be found in the relatively low cost of publishing and distributing content products online compared to offline and in print.

In print publishing, the cost of sending a content product to print can be enormous. And to correct any errors, such as major spelling errors or incorrect contact information in a printed catalogue, the content product needs to be printed again. You simply don’t afford to discover errors after the content product has been published. However, the same is not true when publishing a content product online. In most cases, any errors can easily and at almost no cost be corrected after the content product has been published.

It seems as when there are no apparent costs connected to quality flaws, it tends to lead to sloppiness in the production and publishing process. Still, there are always indirect, less obvious costs of publishing content products with quality flaws online. A mistake in the information about a product in an online product catalogue can get the same negative consequences as if it would happen in an offline product catalogue. The consequences of such an error might get even worse online if the online catalogue is used for e-commerce where there is no human “intermediary” (like the cashier) that can discover and correct any errors before a purchase is made. And, then there’s the damage that an error – for example in an online campaign - can do to the brand. And bear in mind that the online content product might reach a bigger audience than the offline or printed version.

My point is that a company should be just as afraid of making an error in an online content product as if they make it in a content product that is to be published offline or in print. And it should put as much effort in ensuring that the quality is as desired before publishing a content product online as they would when publishing a similar content product offline or in print.

Is User Experience Also a Quality?

You might wonder why the concept of user experience is not mentioned in my previous post. Peter Merholz presents some really interesting thoughts in his post "User Experience is a Quality, Not A Discipline" from 2005:

• "Usability is simply a quality. It's an important quality, but just one of many. And it definitely doesn't warrant being a "discipline."
  
• "User experience is not a discipline, or an approach, it's a thing, a quality, an emergent property between a person and a product or service.
  
• "This puts me in direct opposition with Jesse's diagram. Those aren't elements of user experience. Those are elements of web design."
  
• "User experience should not be just about interactive systems -- it's a quality that reflects the sum total of a person's experiences with any product, service, organization."

I basically agree with this reasoning, especially that usability is a quality and that Jesse James Garretts model "The Elements of User Experiences" is focused on web design. Which Jesse also declares in his book with the same name: "This book is about the user experience of one particular kind of product: Web sites". But I don't agree with Peter that the user experience is simply one quality among many others.

A user experience of a product, service or organization is the sum of all its qualities as perceived by the individual user who experiences it. It also includes the value dimension. Furthermore, each user experience is affected by all previous experiences the user has had with the product, service or organization in question. In a sense, the user experience is the overall quality of a product, service or organization as perceived and experienced by an individual.

Hence, I have not made user experience to a quality among the others in my post "The User Experience Onion". A user experience is not an individual layer of the onion, it is the sum of all layers, the onion itself. And the user experience is created when the content product ("the onion") is consumed by an individual user.

If you are interested, I borrowed the onion metaphor from the animated movie Shrek.

SHREK: For your information, there's a lot more to ogres than people think.
DONKEY: Example?
SHREK: Example? Okay, um, ogres are like onions.
DONKEY: [Sniffs] They stink?
SHREK: Yes. No!
DONKEY: They make you cry?
SHREK: No!
DONKEY: You leave them out in the sun, they get all brown, start sprouting' little white hairs.
SHREK: No! Layers! Onions have layers! Ogres have layers! Onions have layers.You get it? We both have layers. [Sighs]
DONKEY: Oh, you both have layers. Oh. [Sniffs] You know, not everybody likes onions. Cake! Everybody loves cakes! Cakes have layers.
SHREK: I don't care... what everyone likes. Ogres are not like cakes.
DONKEY: You know what else everybody likes? Parfaits. Have you ever met a person, you say, "Let's get some parfait," they say, "Hell no, I don't like no parfait"? Parfaits are delicious.
SHREK: No! You dense, irritating, miniature beast of burden! Ogres are like onions! End of story. Bye-bye. See ya later.

Can We Measure the Quality of a Content Product?

When you talk about the quality of a product as a user / customer, I guess that you are talking about the overall impression you have of how the product performs. If it performs to your expectations, or even excels them, then it is of high quality. If it does not, then your are simply not satisfied with it. The same goes for content products, such as a web site, a whitepaper or a Flash animation. If it does not perform to your expectations, your leave the site, delete the whitepaper or close down the Flash animation.

“The quality of a product or service refers to the perception of the degree to which the product or service meets the customer's expectations. Quality has no specific meaning unless related to a specific function and/or object. Quality is a perceptual, conditional and somewhat subjective attribute.” (Wikipedia.org)

However, this definition of quality is very hard to use for a content producer. It might lead to that quality is not defined at all because it is too difficult to define it. And if it is not defined, it cannot be measured and assured.

To avoid this scenario, we must make a distinction between quality and value. The value of a content product is a subjective judgement by the user and how the benefits are perceived in relation to the cost (usually monetary, but could also be effort) of obtaining the content product. For example, a specific content product can be of high quality but of no value to me as a user. But it might as well be perceived as very valuable by other users.

So, seen from a creator / producer, we are definitely in need of more "universal" quality measurements - accuracy, timeliness and completeness.

Accuracy
The content is accurate if it is produced and delivered without errors or flaws. For example:

• Text is free from grammatical, syntactic and style errors
• The writing style, terminology, and style is consistent
• The content adheres to established standards and guidelines
• And so on

Timeliness
The strict definition of timeliness – sometimes referred to as “freshness” – is “occurring at a suitable or opportune time; well-timed.” The point is that content that is correct today might not be correct tomorrow. For a content product to be correct, it needs to be both accurate and timely. In many contexts, such as when making a business decision, timeliness is the most important quality factor. In other, it is less important. But it is always a factor that affects the quality of the content.

Completeness
Completeness is "the state of being complete and entire; having everything that is needed" or
"having all necessary or normal parts, components, or steps". It is about providing all the relevant content needed in one place, in one content product.

Obviously, the concept of relevance is closely related to the concept of completeness. Relevance is a subjective measurement of how well the content satisfies the user's need. It is about saying just enough and not forcing the user to filter out things that are irrelevant from the content. The content shall include all content that is relevant and exclude all content that is irrelevant.

Depending on what the requirements are, additional measures can be used to determine the quality of a content product, such as:

• Readability – measure of the accessibility of the content and/or associated layout, indicating how effectively it will reach a given reading audience
• Usability – how easy and convenient it is to use the content product
• Reliability – if the content is worthy of reliance or trust, if it is perceived as a reliable source
• Availability – if the content is ready for use by an authorized person
• Accessibility – how accessible the content is to people with disabilities
• Compliance – how well the content complies with relevant laws, rules and regulations

Simply put - if you can define it, you can measure it.

To sum up, a content producer can define, measure and ensure the quality of a product. To define, measure and ensure the value of a content product is another - and much harder - thing. But let's not make it unnecessary hard for ourselves by mixing quality and value.