Quality Assurance – Trust Is Good But Control Is Better

The company or organization that claims to have no data quality issues is as trustworthy as a person that claims to never have made a mistake. Errors are not only an intrinsic part of human nature, machines also fail. The quality assurance process that should be implemented depends a lot on the specific context. Marcoumar is a popular blood thinner but is also used to poison rats, so if you don’t want to kill your patient you had better make sure that there is no doubt nor mistake regarding the right dose to be administered.

At JOT we don’t administer medicine, but nevertheless we believe that a continuous quality control process is of utmost importance. Every single day we manage and analyze millions of clicks and further data from a multitude of sources. With the Big Data hype, many companies get so excited about the volume and potential benefits of acquiring Big Data, that they neglect the most basic data validation checks.

Since there are already several articles and white papers about practices such as data profiling and data cleansing in general, I want to focus on two other aspects related to quality assurance:
• Errors – risk or opportunity?
• Common human errors and how to handle them

 

Errors – risk or opportunity?

Let’s start with the first one: Are errors always “bad”? Several of mankind’s breakthrough discoveries have been made by mistake or through negligence. Take penicillin, for example. Does that mean we shouldn’t focus so much on quality controls, limit ourselves to risk management and wait for the outcome? Of course not. Quality control processes always go hand in hand with risk management, since the consequence of having a typo in a keyword is not comparable with having a typo in an account tracking template, for example. In other words: In areas where mistakes have a bigger magnitude, the quality controls need to have a higher priority.

In some contexts, however, we have seen that using data which is not 100% “perfect” might give us extra opportunities. In theory, when we use the Adwords exact match type for keywords, misspellings plural/singular forms, and stemmings are all covered. In practice, however, sometimes adding these so-called close variants will generate more traffic than just this one keyword which should already represent all the variants. One part of a good quality control process is to know which kind of errors you can accept where, when, and to what degree.

Once you have defined your risk measures for all your business areas, including the worst possible outcome for every type of error, you need to set up the actual quality controls. At JOT we usually apply a combination of automated quality checks and random sampling. When daily quality control has not discovered a single error in a month, this doesn’t mean that you can just stop it. You might have to think about whether some part of the process chain has changed and is escaping your control. There might not really be any errors but – working similarly to how a scientist works – we never stop doubting and checking, and looking at the data from the other side.

As we don’t only work with massive data, let’s look at some simple guidelines that might be interesting to anyone working in digital marketing or even elsewhere.

 

Common human errors and how to handle them

Get a pair of fresh eyes to double check your work
At JOT we regularly rotate projects within the team because we believe that after a while a new impulse, a different point of view is beneficial. Don’t fall into the trap of thinking that just because the person who handled the project before you is more senior, you don’t need to check all the campaign settings and all relevant parameters before you jump into your new project.

Don’t get overconfident
Even if you have been working with a tool for 10 years and are considered an expert, you should never stop previewing and testing before executing massive changes. Haste makes waste.

Audit Internally
We have the tendency to see what we want to see. Let’s say you are setting up some automated rules. Of course you will carefully check (and preview) the conditions before actually enabling a rule. Yet we still sometimes overlook an important detail, and the erroneous rule will be executed again and again until we realize too late and start investigating where the sub-optimal performance is coming from. For important objects or changes, let somebody else double-check.

Be careful with file formats and conversion errors
At JOT our team is composed of a multitude of nationalities and we have noticed that it is of utmost importance that all of us use the same regional settings. Otherwise file sharing might turn into a disaster. The same thing applies when working with, let’s say, the Adwords platform. Whether you need to use a comma or a decimal point to separate decimal places is not a trifling matter, especially when you’re changing CPCs for a million keywords and you don’t have tight budget limits set (budget control in line with your defined risk levels is another topic of quality control). What about uploading keyword files in 17 different languages? Make sure you use a file format and conversion that correctly works for all of them.

Establish naming conventions
I cannot insist enough on the importance of using well-defined naming conventions and standard objects even in such a rapidly changing environment as digital marketing. Many, especially creative people, show a higher resistance towards rigorously applying such conventions to all objects they are creating. However, if you want to set up an automated quality control process, you need to know what to expect within each account or campaign directly from its name.

Having said all this, try to find a healthy balance between rigorous quality control processes and getting so absorbed by it that you barely have any time left for your actual business.

And if you ever need any help, you know where to find us!

Miriam Graue, New Business Project Manager

Leave a Reply

Your email address will not be published. Required fields are marked *