← MBA Hub
MBUS 825 — Session 5

Make It Count: Measuring Marketing Effectiveness

Queen's Smith AMBA 2027 · December 5 · In person, Toronto · 8:30–12:30 · Andrea Hunt, MBA
Metrics Roadmap Budgets Down AI-Savvy Leader Dial-an-Executive
Block 1 — Session at a Glance: Make It Count

Define Success, Measure the Full Picture, Spend Less Better, Lead the AI

First in-person session. CLLO 7: "Define what successful marketing looks like and how an organization should measure the effectiveness of its marketing efforts." Three readings, one argument: marketers who measure only campaigns get treated as a cost centre, and in an era of less that is fatal.

Syllabus

Leveraging data and analytics in value creation

Descriptive → predictive → prescriptive (Session 1) applied to measurement, plus experiments for causality (Session 4).

Syllabus

Predictive analytics and customer segmentation

Propensity, churn and CLV models. RFM and clustering for value-based segments.

Syllabus

KPIs for brand health and effectiveness

Magill & Moorman's five-stage roadmap plus Stengel's FRMU composite.

Syllabus

Budgeting realities and AI

Gartner's "era of less" and De Cremer's case that AI fails when leaders defer.

Block 2 — Do Your Marketing Metrics Show You the Full Picture? (Magill & Moorman, HBR 2022)

The Metrics You Report Signal the Role You Play

"Incomplete metrics signal a marketing function with limited impact." Build a metrics roadmap that foreshadows what you will measure and therefore how you'll contribute.

Stage 1
Campaigns

Clicks, visits, ROI. Execs' reaction: "Great, glad we didn't waste the money."

→
Stage 2
Sales & satisfaction

Revenue growth, NPS. Where most marketers are stuck.

→
Stage 3
Brand

Awareness, consideration, preference, quality, differentiation, affinity, equity value

→
Stage 4
Capabilities

Market knowledge, lower CAC, CLV, churn reduction

→
Stage 5
Enterprise

Brand in HR and talent, insight in strategy, ABM for sales, so growth rate, multiples and enterprise value

Three Lenses to Classify Any Metric

  • Income statement (sales) vs balance sheet (brand equity as an asset)
  • Historical (last quarter) vs forward-looking (expected CLV)
  • Single-period (a campaign) vs multi-period (a testing capability that pays off for years)
The CMO Survey finding (26 metrics, Aug 2021): campaign and sales/satisfaction metrics dominate, while brand and capability metrics are "very infrequently measured." Why: attribution fights over sales credit eat all the bandwidth, enterprise-wide NPS crowds out its drivers, and some marketers have settled for being sales support.
Block 3 — KPIs for Brand Health & Effectiveness: A Working Dashboard

Leading Indicators (Brand) + Lagging Indicators (Revenue) + Causal Proof

FamilyKPIsLeading or lagging?Watch-out
Brand healthAided/unaided awareness, consideration, preference, FRMU (familiarity, regard, meaning, uniqueness), share of search, NPSLeadingMeasure among non-customers too (Prospects, Winbacks), not just loyals
Customer economicsCAC, CLV, CLV:CAC, retention and churn, share of walletMixedCAC from last-click flatters performance channels
EfficiencyROAS, CPA, CPM, cost per lead, martech utilizationLaggingROAS without incrementality overstates (eBay)
IncrementalityLift from holdouts, geo-tests, MMM-estimated contributionCausalThe truth serum: the only way to separate "would have bought anyway"
FinancialRevenue growth, gross margin, price premium, brand value (royalty relief)LaggingThe CFO's language. Translate the others into these.
CapabilityTest velocity, data quality, time-to-insight, first-party data coverageLeadingAlmost never reported, yet the biggest long-run lever

Measurement Methods: Triangulate, Don't Pick One

Attribution (MTA / last click)

Fast, granular, biased

Good for in-channel optimisation. Systematically over-credits bottom-funnel and ignores brand. Walled gardens and privacy limits are eroding it.

Marketing mix modelling

Holistic, slow, correlational

Covers offline and brand plus macro effects. Needs history and spend variation. EY's "good-enough math" across the funnel.

Experiments

Causal and focused

Holdouts and geo-lift calibrate MMM and attribution. Facebook's 500M-observation study found only randomized tests got ad effects right.

Brand tracking

The leading indicator

FRMU or equity scores linked by elasticity to revenue (Stengel). Makes brand performance-accountable.

Predictive Analytics & Segmentation (Syllabus)

  • Propensity models: who will buy or upgrade, for targeting and suppression (don't pay to reach sure-things).
  • Churn models: who will leave. Optimise on expected value saved, not accuracy (Fantini's false-negative point).
  • CLV models: how much a customer is worth, which sets allowable CAC by segment.
  • Clustering / RFM: behaviour-based segments that refresh dynamically. Pair them with qualitative "why" (Collins) before building creative.
Block 4 — Marketing When Budgets Are Down (McIntyre, Gartner, HBR 2023)

From the "Era of More" to the "Era of Less"

12.1% → 9.1%
Marketing budget as % of company revenue: 2016 peak → 2023 (9.5% in 2022)
42%
Martech utilization. More than half of every martech dollar is unused, and martech was 29.2% of budget at its 2018 peak.
75%
Of CMOs pressured to cut martech. "Unthinkable just a few years ago."
SignalData (Gartner CMO Spend Survey 2023, 400+ CMOs)
Resources>70% lack budget to deliver 2023 strategy. 75% asked to do more with less. 86% pressured to change how marketing works.
Digital stallsDigital share of media down 1 point to 55%
Enterprise impatience81% of boards report no progress toward digital transformation goals. 67% of CFOs say three years of digital spend missed expectations.
Purchasing powerA marketing "cost-of-living crisis": talent and media costs up, tech yield down, so each dollar buys less
Shift in statusDigital and martech moved from system of innovation (unproven, transformative) to system of record (expected to be efficient and predictable). "It's entirely possible to get fired for spending more on digital and tech."

Gartner's Three Leadership Traits

C

Clarity. Be choiceful about what will and won't be supported, and give equal thought to what's cut and why.

C

Courage. Challenge spurious assumptions, strategic sentimentality and sunk cost ("spent $100k doesn't justify $100k more"). Every investment must re-earn its case against future goals.

C

Connection. Build a shared value story with the CFO, CIO and CSO. "Translate brand speak into business speak." Return on objectives alongside return on investment.

On AI: it isn't new to marketing (it has been "hiding in your tech stack"), and with 42% martech utilization, piling generative AI on top is a fiscal risk. Think hard before new commitments. Budgets "drop like a stone at the first sign of trouble and rise like a feather."

A Practical "What to Cut, What to Protect" Screen

Protect

Spend with proven incrementality, brand building among the highest-return segment (Stengel), first-party data and the test-and-learn capability, and retention of high-CLV customers.

Cut

Unused martech seats (58% idle), always-on spend never tested with a holdout, duplicated tools and agencies, and "legacy" programs renewed out of habit.

Test before deciding

Channels with high attributed but unknown incremental ROAS (often branded search and retargeting). Run holdouts and cut if lift is low.

Re-scope

Big platform builds. Phase them and require early proof ("impatient for profit," Session 4).

Block 5 — The AI-Savvy Leader: Nine Ways to Take Back Control and Make AI Work (De Cremer, HBR Press 2024)

AI Projects Fail Because Leaders Aren't Leading Them

De Cremer opens with a manufacturer that spent millions on AI and was ready to pull the plug within a year. The diagnosis wasn't the technology: "Where were the business leaders?" They had deferred to technologists, never used the tools, communicated no vision, and gave employees no idea what AI meant for their jobs.

87%
Of digital transformation projects failing to meet objectives (on ~$6.8T of projected spend)
68%
Of Singapore managers (a top-4 digital economy) say their organization is not effective with adopted AI
~$16T
Predicted AI addition to the global economy: the prize driving the rush
Perspective 1: replace

AI as a cheap way to replace people and delegate thinking ("tech-driving-tech"). Employees become task completers. Leaders who think this way "are over time ceding their leadership to AI."

Perspective 2: augment

AI augments human intelligence to unlock creativity and innovation. It requires investment in people and leaders taking active charge. This is the AI-savvy leader.

The Nine Ways (Chapter Titles)

#PrincipleMarketing application
1Learning: get to know AI and use it yourselfThe CMO personally uses the gen-AI content and analytics tools before mandating them
2Purpose: use your purpose to ask the right questionsStart from the customer job and the brand purpose, not "where can we put AI?"
3Inclusion: work inclusively to drive human–AI collaborationCreative, analytics and agency teams co-design workflows
4Communication: a flat communication culture drives adoptionSurface failures fast, with no fear of saying the model is wrong
5Vision: be visionary in how to use AIPersonalization at scale and faster insight, framed as better customer value
6Balance: adopt AI with all stakeholders in mindCustomer privacy, creators' IP, employees, regulators (echoverse risks, Session 6)
7Empathy: a human-centred approachBrand voice and trust. AI disinformation is more credible than human-written (Session 6).
8Mission: augment, don't automate, to create jobsAI drafts, humans judge and craft the story (Balis: "don't lose that creative spark")
9Emotional intelligence: soft skills are the new hard skillsInsight (Collins's intimacy) and influence are what machines can't do
"You can't delegate digital transformation."
Barry Ross, quoted by De Cremer. And B. F. Skinner: "The real question is not whether machines think but whether men do."
Block 6 — Likely Discussion Questions, With Sharp Answers

Cold-Call Ready

Q1: Your CFO wants a 20% marketing cut. What do you cut first?
Cut what you can't prove, not what you can't see. (1) Idle martech: utilization is 42%, so consolidating seats and tools is pure efficiency. (2) Spend with high attributed but untested ROAS. Run fast holdouts on branded search and retargeting, where eBay-style "would have bought anyway" waste hides. (3) Protect brand investment in the segment with the highest equity-to-revenue elasticity and the testing capability, because they're the future. Present it as Gartner's clarity: here is what we stop and why, in return-on-objectives terms.
Startup reflex: in a cash crunch, kill vanity channels first. A two-week holdout on one channel often frees 10–20% of spend with no revenue loss.
Q2: Isn't NPS the one metric that matters?
It's useful for alignment but dangerous as the only metric. Magill and Moorman show enterprise-wide NPS "crowds out" tracking of its drivers (brand, capabilities). NPS is a lagging, current-customer metric. It says nothing about Prospects (Stengel) and nothing about causality. Use it as one outcome in a roadmap, not as the roadmap.
66°North's NPS looks strong only among people who already know it. In a familiarity-problem brand, NPS is a misleading north star.
Q3: Will AI make marketing measurement solved?
No. It makes prescriptive optimisation cheaper but doesn't fix the question-framing problem or the causality problem. Models trained on observational data inherit attribution bias, and randomized tests are still the only reliable truth. De Cremer's warning applies: if leaders defer to the model, decisions become "only as good as the algorithms allow." AI should speed up test-and-learn, not replace it.
Analytics PM angle: the most valuable AI use at Redamo or Prodigy-scale is automating experiment analysis and anomaly detection, which raises test velocity. That's augmentation, not replacement.
Block 7 — Participation Hooks, Taju's Edge, Guest & Course Links

In-Person: Be the Room's Measurement Person

Reframe

"In the era of less, the best budget defence is a holdout test. Gartner says 75% are asked to do more with less, and the fastest way is to stop paying for sales you'd get anyway."

Push Consensus

The room will say "measure brand too." Push further: brand metrics must be elasticity-linked to revenue (Stengel) or they'll be cut first again.

AI Nuance

Agree AI matters, then add that martech is 42% utilized. The constraint is adoption leadership, not tools (De Cremer and Gartner agree).

Taju's Edge: Experimentation

Real numbers from A/B work at 3M+ user scale (+16% retention, +$35 LTV per user). Speak as someone who has run the tests, including the false-positive traps.

Taju's Edge: Constrained Budgets

Building Stutern, Native and AfroShorts under tight budgets means the "era of less" has always been the default. Choicefulness is a founder's daily habit.

Taju's Edge: AI Agents

Practical perspective on AI agents in workflows. Augmentation works when the human owns the objective function. That's De Cremer's Perspective 2 in practice.

Questions for the Dial-an-Executive Guest

  • What's on your monthly marketing dashboard that the CFO actually reads?
  • Last time budgets were cut, what did you protect, and what evidence convinced the CFO?
  • Where has AI changed a marketing decision, not just saved time?
Group Assignment (Dec 16)

The 66°North marketing plan needs a measurement section: FRMU targets among US Prospects, tourist-capture conversion, DTC CLV, and a geo-test for pop-ups.

→ Session 6 (afternoon)

Influencer ROI is the hardest measurement problem: engagement isn't incrementality (Huel's ~$30 CPA is the exception that measured).

Debate (Dec 8)

Likely motions on brand vs performance, AI and the CMO, or budget cuts. Bank the Gartner and De Cremer stats.

One sentence to remember: Marketing earns budget in the era of less by proving incrementality and translating brand into business value. AI raises the bar for that leadership rather than replacing it.
MBUS 825 · Session 5 Prep · Queen's Smith AMBA 2027 · Andrea Hunt · Group Assignment (30%) due Dec 16 · AI-assisted prep notes (Claude), cite per course GenAI policy