The International Association for the Measurement and Evaluation of Communication met in Sofia recently for its 2024 Global Summit on Measurement, exploring “Innovation, Implementation and Insights: The Global Communication, Data, Measurement and Evaluation Journey.”

It remains sadly rare to see measurement and evaluation addressed in this kind of depth, despite the continued discussions about the PR profession’s difficulties in proving its value to management, and we attended the Summit—and spoke with several speakers and attendees—to identify some of the major themes this year.

1. AMEC is First and Foremost a Community

More than any other of our profession’s leading organizations, AMEC is a community, bound together by a genuine enthusiasm for a topic that is all too often relegated to the sidelines—or to empty promises—by the rest of the industry. Indeed, so little progress has been made on measurement and evaluation that there are times when AMEC feels like a support group.

Stuart Bruce, author of the PR Futurist Blog, captures this perfectly: “I don’t often learn much at AMEC as often speakers have already shared their knowledge in books, blog posts, articles, podcasts, and videos,” he says. “However, what I do massively benefit from is increased confidence that the counsel we’re providing to clients is the right advice. It makes me double down on persuading clients to accept challenging advice.”

By convening people who continue to push the issue of measurement to the forefront, AMEC has a power and a value that goes beyond the content of the conference—as well as providing a glimpse into the future of the debate around evaluation.

“I’ve been fortunate to attend every AMEC Communications Evaluation Summit since they first went international, 15 years ago in Berlin,” says Richard Bagnall, now an independent consultant and recent recipient of PRovoke’s SABRE Award for Outstanding Individual Achievement. “The pace of change in the industry is phenomenal and there are many exciting reasons to be optimistic about the future of PR measurement.”

Reputation management and measurement guru Sandra Macleod, group CEO of Echo Research, said the event had highlighted “greater understanding of measuring communications and reputation more generally, along with technological advancements driving increased sophistication among both clients and providers.

“The sector is maturing, reflecting substantial growth and development.”

And she suggested that the measurement discussion may be poised for disruption: “The industry must transition from a boutique/cottage industry approach to a more coordinated and value-focused model; stay tuned for greater consolidation ahead. The emphasis should shift from merely providing instant, cheap access to delivering enhanced value and strategic insights.”

2. Academics Drive New Thinking

One of the challenges facing the profession as a whole is the sometimes-inscrutable nature of new thinking that emerges from academia. The chasm between PR academics and practitioners sometimes appears unbridgeable.

It is now 40 years since University of Maryland professor Jim Grunig developed the “four models of public relations” and argued for the primacy of “two-way symmetrical communications.” His argument was persuasive, but I don’t believe I have ever heard those four words used by an actual practitioner (though some doubtless employ the model without realizing it).

In any event, AMEC provides one of the few forums in which academic thinking can penetrate the actual professon.

“The AMEC Global Summit is not a scientific conference, but there is clear value in including renowned academics in the program, showing the importance of applied science in the broad field of communications,” says Maya Koleva, director of research and insights at Commetric.

And some of the most interesting sessions at this year’s event came from the academic community.

Bagnall, for example, cites “a particularly powerful presentation” from distinguished professor Jim Macnamara of the University of Technology, Sydney, “one of the titans of the sector,” who talked about the MEL (Measurement-Evaluation-Learning) model, in which evaluation is part of the planning cycle and used for continuous improvement and not a separate process—as it often is in practice.

Says Koleva, ““He also highlighted that establishing a theory of change is crucial for linking communication activities to desired outcomes and impacts. It helps in understanding the process and causation.

Another highlight was a presentation by Larry DeGaris of the Medill Spiegel Research Center at Northwestern University, who focused on sponsorship as a communications and marketing platform, rather than as a channel, and proposed an integrated measurement framework, with an emphasis on context and authenticity.

3. Practitioners Innovate and Lead

But crucially AMEC also makes room for practitioners who are elevating the importance of measurement within their own organization, most of whom appear to be using their exposure the leaindg-edge thinking at AMEC to drive new approaches that set C-suite expectations rather than simply meeting the (often limited) expectations of CEOs, CFOs and marketers.

Bagnall points to several initiatives over the past few years: the Barcelona Principles, an Integrated Evaluation Framework, the Measurement Maturity Mapper, and the “Say No to AVEs” campaign, which have all gained traction among both agency and client-side leaders.

“With the principles, frameworks, and best-practice established, it’s now a question of global adoption and implementation.” To that end, he says, “each year more and more organizations and brands come to the summit to share their measurement journey.”

Highlights in Sofia included Abby Scott and Sarah Myles from McDonald’s, who came to talk about the future or reputation measurement, LinkedIn’s Nicole Moreo on best practice and new metrics, and Golin’s Jonny Bentwood on data and disruption, as well as Andre Manning—a veteran of communications roles at Royal Phillips, Booking.com and Tata Steel—who discussed the three times that measurement had “saved him” in the boardroom.

From the agency side of the business, Jonny Bentwood, global president, data and analytics, at Golin, provided what Bruce called “an engaging and enlightening presentation” focusing on new tools available to practitioners as well as on the importance of data storytelling.

“Without a compelling narrative, even the most robust data insights and KPIs can drown in noise,” said Bentwood. “Succinct storytelling is our ace, and leaning into design-focused visualizations is our winning strategy.” He pointed to the firm’s award-winning work with McDonald’s as an example.

And Allison Spray, managing director, data + analytics, at H+K Strategies was joined by Kyle Mason, head of external monitoring, corporate relations, Shell to discuss more of the emerging data tools available to practitioners and to provide guidance on whether agencies on their clients should “build or buy” such tools.

4. The Future is AI

Spray and Mason’s presentation was driven in part by the massive investments that PR agencies, holding companies, vendors, and even some clients are making in artificial intelligence tools that have the potential to revolutionize not only PR evaluation but also planning and predictive capabilities. Not surprisingly, the impact of AI was one of the themes that ran through many presentations.

“Tech developments and AI were topics that were explored in depth,” says Bagnall. “They bring enormous benefits to the sector, increasing speed and capabilities in ways that seemed impossible even a year ago.”

But he warned against viewing AI exclusively as a labor-saving device. “It is critical that we use the time benefits to our advantage, allowing us to increase our curiosity, improve our work, and up our insights not ‘dumb down’ our services based on the ‘plagiarized soup’ of AI generated averageness. The importance of consultants and experts to make sense of the tools, enrich the data and provide context and relevance to the insights will be critical in this new world.”

Bentwood agreed. “AI is set to revolutionize our efficiency,” he said. “The challenge lies in harnessing the time it liberates. We should channel curiosity and drive to advance our work, rather than merely enjoying the extra hours. Beyond efficiency, AI opens doors to previously unattainable feats. A focus on detecting disinformation, with allies like [disinformation specialist] Cyabra, exemplifies this potential.”

There were other warnings too, amid the generally optimistic tone of the discussion.

Rob Key, CEO of Converseon, talked about the critical importance of data quality in AI and the challenges with unstructured data and human bias. Poor data quality, he said, can significantly hinder AI initiatives and lead to inaccurate sentiment analysis and other classifications, impacting decision-making and product development. He called on the wider research community, PR measurement and media intelligence sector to take responsibility for the accuracy and governance of AI models, even when using third-party platforms.

“Human biases and varying interpretations of data, such as sentiment or emotional tone, complicate the development of reliable models,” said Koleva, commenting on Key’s presentation. Strategies to improve data quality include having multiple humans review and agree on data classifications, using precision and recall metrics, and employing topic discovery tools to uncover unknown patterns in data.

Commetric senior consultant Sofia Tzamarelou also talked about the danger of cognitive bias, especially in the era of generative AI, warning that biases might stem from preconceived notions, past experiences, and cultural influences, which can affect how data is interpreted and what decisions are made.

She suggested methods such as devil’s advocacy and “what-if” analysis; continuous questioning and critical thinking; and diverse teams and perspectives. “Encourage collaboration with colleagues from different backgrounds and areas of expertise,” she suggested. “By discussing and debating various viewpoints, you can identify biases that may not be apparent within a homogenous group.”

5. “Reach” is Vanity

One of the most successful initiatives to emerge from AMEC in recent years is the campaign to eliminate AVEs (attempts to define PR results in terms of “advertising equivalent: costs) as a measure that is used by respectable practitioners.

While there are still clients who demand AVEs from agencies or third-party providers—usually in an attempt to compare the output of PR in terms that relate to spending on other marketing activities—and still agencies that succumb to those demands, AVEs have become increasingly rare in recent years.

And so much of the attention at AMEC has turned to “vanity metrics” such as impressions and media reach which—I think it’s fair to say based on the awards entries we see—remain the most widespread form of “measurement,” an impression reinforced by somewhat depressing research into the current state of measurement presented by Hotwire’s global head of data and analytics Matt Oakley.

“There are still occasions where people confuse media coverage measurement with PR evaluation,” says Bagnall. “They substitute media metrics, with overclaims of comms effectiveness. There can be too strong a reliance and emphasis on some media metrics that are losing their relevance.

“In the same way that the old metrics like number of articles and ‘column inches’ are no longer relevant, ‘reach’ and ‘impressions’ are losing their relevance too,” Bagnall says. And, for obvious reasons, “It can still be too tempting to trot out the largest numbers that are available.”

If professionals do talk about reach, he says, they should remember that “article level” reach is more realistic than domain level and that even then, more than half of people rarely read beyond the headline and perhaps the first couple of paragraphs.

“Reach these days is little more than a broad index and certainly not a number of people that have seen, heard or taken in your messages and perspectives<” Bagnall says. “The days of seeing reports with claims suggesting that X billion people saw coverage as a validation of success need to be behind us.”

As AMEC global managing director Johna Burke quipped, more than once during the event, ‘Your impressions don’t necessarily mean anyone was impressed.’

6. Focus on Reputation

When organizations do go beyond reach and other vanity metrics to assess the actual effectiveness of their communications efforts, there is often a tendency to focus on short-term impacts: the way in which a particular PR-driven promotion can impact sales over a period of days or weeks, for example.

But it is clear from discussions with AMEC attendees that the evaluation community still wants to see a heavier emphasis on long-term relationship-building benefits of PR, and on corporate reputation.

Says Bentwood, “Emphasis was placed on the importance of long-term brand building over short-term tactical results.”

He added: “Building a healthy trust ‘bank balance’ is essential for crisis navigation.” (Golin’s founder, Al Golin, was one of the first to discuss this idea of a “trust bank,” sometimes referred to as “a reservoir of goodwill). “A comprehensive framework for reputation management includes gaining goodwill, commanding respect, and building engagement, affinity, and trust.”

Roy Persson, head of corporate reputation & PR at National Research Group was joined by two senior executives from McDonald’s communications measurement and insights function, director Sarah Myles and manager Abby Scott, to discuss the development and implementation of the company’s Global Reputation & Impact Tracker.

The tracker is a multi-stakeholder research intiative that assesses reputational health and informs decision making across McDonald’s Global Impact functions, including communications, government relations, public policy, sustainability and social impact.

“The presentation had a strong focus on the difference between reputation and brand,” says Bruce, “and why it’s essential to ensure why broader reputations and relationships aren’t sacrificed to brand or marketing communications.”

Bruce also highlighted a presentation by India’s Amith Prabhu, founding dean at the School of Communications & Reputation and the co-founder of PRAXIS. Prabhu spoke about the need to focus on much broader and more holistic metrics that encompass the importance of reputations and relationships rather than just marketing-related sales metrics.

His GREAT framework focuses on Goodwill, Respect, Engagement, Affinity and Trust.

Other highlights included The P Word’s Kostav Petrov, who took attendees through the major reputational risks in the era of the permacrisis, and Rachel Phillips, director of Ipsos Corporate Reputation, who shared research—including insights from the organization’s PR Council—on the trends, issues and challenges confronting CCOs in the year ahead.

Image created by Copilot AI.