AEO Measurement & Insights Lead

About the role

This role owns the measurement and insights infrastructure behind the AEO program. You'll build and maintain the prompt panel used to test how the brand shows up in AI-generated answers, evaluate the tools that support it, run quality assurance on the responses that come back, and turn all of it into dashboards and experiment analysis the team can act on. The core question you're answering: when someone asks an AI platform about this category, does the brand show up, how is it described, and what makes that change.

You'll own the measurement system end to end, from designing what gets tracked, to auditing the quality of what comes back, to building the reporting layer that makes results legible to non-technical stakeholders. The job takes equal parts rigor and communication. Sound experiment design and consistent QA on one side, clear and usable findings on the other.

This is a new category, so the measurement approach doesn't fully exist yet. You'll be building it, and some of it you'll figure out as you go.

You will be responsible for

• Building and maintaining a structured prompt panel to test and track brand visibility and citations across AI platforms.
• Evaluating and recommending tools for prompt panel management, response capture, and analysis.
• Running quality assurance on response data, flagging inconsistencies, tool errors, or gaps that would compromise the accuracy of findings.
• Designing and building dashboards that make performance and trends clear and actionable for both agency and client stakeholders.
• Designing and running experiments to test hypotheses and isolate what actually drives changes in visibility.
• Analyzing experiment and panel results and translating findings into clear recommendations.
• Documenting measurement methodology and QA processes so the system is repeatable, defensible, and can transfer to the client team over time.
• Partnering with agency and client stakeholders to align on what's being measured and why, and socializing findings as needed.

What You'll Be Measuring

• Visibility. Whether the brand appears in AI-generated answers across a defined prompt panel, and how often.
• Share of voice. How the brand's presence compares to competitors within the same answers.
• Accuracy and framing. How the brand and its products are described, and whether that description is correct and favorable.
• Source attribution. Which pages, domains, and third-party sources the models pull from when the brand does appear.
• Platform coverage. How results differ across ChatGPT, Gemini, Perplexity, Copilot, and AI Overviews.
• Consistency. How stable responses are when the same prompt runs repeatedly, and what that variance means for confidence in the data.
• Movement over time. Whether the work the team is doing actually changes any of the above.

The Insights You'll Deliver

• Where the brand is absent from answers it should be in, and the likely reason why.
• Which content and structural changes moved citation rates, and which didn't.
• Competitive gaps worth acting on, and what competitors are doing to earn their citations.
• Prioritized recommendations for the content and optimization team, grounded in what the data shows.
• A regular readout that both agency leadership and client stakeholders can follow without a data background.

Must-have

• Hands-on experience measuring AI search or AEO/GEO performance: prompt testing, citation and share-of-voice tracking, or LLM response analysis.
• Experience building and maintaining measurement or reporting systems (dashboards, tracking panels, or similar) in an analytics or research context.
• Strong data QA instincts. You're comfortable auditing outputs for accuracy and catching errors before they reach a report.
• Experience designing and analyzing experiments, whether A/B testing or similar comparative analysis.
• Proficiency with dashboarding and BI tooling, and comfort working with structured data.
• Clear written and verbal communication for translating technical findings into recommendations non-technical people can use.

Nice-to-have

• Background at an agency or platform delivering AEO or GEO as a service.
• Technical SEO experience, particularly around structured data and content structuring for retrieval.

Additional Information

• We Are Rosie provides strategic consulting and talent solutions for the marketing industry. We partner with more than 200 of the world’s biggest brands and agencies to help them solve unique marketing challenges with the best people, swiftly and at scale.
• Founded by former advertising executive Stephanie Nadi Olson in 2018, We Are Rosie is on a mission to redefine how marketing works, with an inclusive, human-centric approach that’s better for business and better for marketers.
• Our company has been recognized as an industry trailblazer and earned many accolades, including being named to the 2023 Global Top 100 Inspiring Workplaces, the Inc. 5000 2022, and Adweek’s Fastest Growing Agencies in 2021.

About the role

Start Date: October 1st

Length: 8 months

Hourly Rate

• $150 (based on experience)
• Location: Remote
• Hours: 40 hours/week