What Makes a Good Analyst?

There is far too much information to process: what is the analysts’ best way to wade through it all?

When I worked in security intelligence in Canada over a 32-year career the collection, processing and analysis of information was the bread and butter of what we did. We received intelligence requirements from our government agencies and figured out the best way to meet those requirements. Sometimes that was through human sources; at others it was through a ‘technical’ one (SIGINT, imagery, etc.).

Even back then we were swimming in data. The analogy we used was ‘drinking through a firehose’. That was then. I cannot imagine how many orders of magnitude the problem has grown today. The challenge is not to find information: that part is easy. The real task is to find the nuggets of truth and accuracy in an ocean of noise.

UNRELIABLE SOURCES

Intelligence agencies have ways of determining whether a source is lying or not. These include obtaining similar information from multiple sources, what we call ‘corroboration’. There is also an attempt to see whether the source is fabricating the intelligence by subjecting him/her to a polygraph.

Neither of these is perfect (many doubt the usefulness of the polygraph at all). Examples from the 2003 US invasion of Iraq serve as a painful reminder how difficult this is.

The US tried to make the case that the Iraqi regime of Saddam Hussein had weapons of mass destruction to justify the need to remove him. The main human source was named Curveball, a chemical engineer who lied about the Iraqi government’s biological weapons program (an accomplishment he is proud of as it helped to end a brutal dictatorship).

There was also intercepted communications that purported to back up the US conviction that there was a program. It turned out that facilities managers lied about their progress out of fear that a truthful rendering would end up badly for them. All in all, the US intelligence community came to the wrong conclusions, the invasion took place and Iraq is where it is now.

DETERMINING HOW GOOD THE INFORMATION IS

As the vast majority of analysts do not work for intelligence agencies, how can they get better at determining how good the information is upon which they rely for their work? While there is no foolproof system (and never will be) there are steps that can be taken to make better decisions on what to use and what to reject:

• what is the source’s track record (if this can be determined)?
• what, if any, agenda does the source have?
• how plausible is the information? If something is too good to be true perhaps it is not true.
• what are other sources saying?
• does this piece of information fit with your working theory? If so, are you choosing it because you want your ideas to be seen as supported?

These questions are hard to answer at times, especially in an era of misinformation and disinformation. Furthermore, there are no guarantees that you will have the best, most accurate information available when you need it. Hence, you have to make do with what you have in light of probable time constraints and delivery pressures.

Analysis is tough and takes time to get good at. As long as you are open to new ideas and are willing to question your own conclusions you will get better at it. After all, the world needs people to wade through the maelstrom of data and make sense of it, providing good advice to those who require it.