Portfolio quality belongs in the evaluation, but it is only the start. CMOs need to understand how an agency reasons, what evidence it creates, how it handles risk, and what the client will own.
Business case and measurement
A strong answer connects each objective to a buyer or internal use case, a requirement, an instrumented event, a downstream business metric, and an owner. Ask to see a sample measurement plan or anonymized reporting structure. A vague promise to “increase engagement” is a red flag if the agency cannot say which behavior matters or how it connects to pipeline, revenue, service, or operating efficiency.
Research and experience
Look for a method proportionate to the stakes. Relevant artifacts may include interview guides, analytics findings, journey maps, content inventories, task flows, accessibility requirements, and usability findings. The agency should explain how research changes information architecture and design decisions. Personas created without evidence and wireframes that appear before the team understands critical journeys suggest that the process is running on assumptions.
Technical architecture and CMS fit
Ask how the website development company will validate the CMS, integration pattern, component model, hosting, and deployment process. A strong response covers editor workflow, permissions, content reuse, security, scalability, performance, developer dependency, and total cost of ownership. Request a CMS editing demonstration before the architecture is locked.
SEO and migration risk
The diligence conversation should cover the URL inventory, content decisions, redirect map, canonicals, internal links, structured data, staging controls, sitemap, launch validation, and monitoring. Google warns that significant site changes can produce temporary ranking fluctuations while pages are recrawled and reindexed, so no agency can responsibly guarantee a fluctuation-free launch. Ask who owns the migration artifacts, who validates redirects at scale, and what happens when Search Console exposes an issue. A buyer who needs deeper implementation criteria can use a current technical SEO checklist to probe the proposed workstream.
Accessibility and quality assurance
Require the agency to state the targeted WCAG version and level, the templates and journeys in scope, the mix of automated and human testing, the assistive technologies or keyboard tests used, and how defects will be documented. Ask for the browser and device matrix, acceptance criteria, defect priorities, and launch authority. A scanner score presented as proof of legal compliance is not an adequate plan.
Proof, team, governance, and ownership
Review relevant web design and development work, then ask for the baseline, intervention, timeframe, data source, and limitations behind a relevant case. Meet the people who will lead the work. Confirm subcontractor roles, capacity, escalation paths, decision rights, and change control. Contract review should establish ownership and access for the domain, hosting, analytics, designs, content, code, accounts, documentation, and licensed components.
The evaluation is strongest when it follows one evidence chain:
Business objective → audience job → required capability → deliverable → instrumented event → CRM stage → pipeline or revenue measure → review owner and cadence
An agency does not need to control every link. It does need to identify where its evidence ends, where client data begins, and which assumptions could weaken the conclusion.