AI strategy and build vs buy
On this page 9 topics
Overview
Topics
- Where AI creates value
- hosted vs open-weight models
- RAG vs fine-tuning
- total cost of ownership
- lock-in
- data strategy
Learn first
4 h of practice (estimate)
Learning outcomes
- Identify where AI creates real value in a product area and where it adds cost and risk without benefit.
- Compare hosted model APIs, open-weight self-hosted models, and managed platforms by cost, control, latency, and data handling.
- Choose between prompting, retrieval-augmented generation, and fine-tuning for a use case.
- Build a multi-year AI cost and capability plan, including lock-in and exit options, and the data foundations it depends on.
Lessons
Lab and exercises
Lab
SRA-021-LAB1 · Commerce 2030 AI strategy
Outcomes: 1–4 · Time: ~2 h
- Define a portfolio of AI use cases for the marketplace, including a seller listing assistant, a support assistant, and search.
- Score each by value, feasibility, data readiness, risk, and cost; reject at least one with reasons.
- For the shortlisted cases choose hosted, open-weight, or managed, and prompting, RAG, or fine-tuning, with justification.
- Build a three-year cost model with explicit usage assumptions, price-change scenarios, and exit options.
- Check data foundations: owners, quality, sensitivity.
- Defend the strategy to a role-played CTO and CFO.
Acceptance: at least one idea rejected with reasons; costs traceable to usage assumptions.
Exercises (case-based; six exercises)
| ID | Outcome | Task |
|---|---|---|
| SRA-021-EX01 | 1 | Score six AI ideas for the marketplace and reject two; write the reasons in terms of value, risk, and cost. |
| SRA-021-EX02 | 2 | Compare a hosted API, an open-weight self-hosted model, and a managed platform for the support assistant on cost, control, latency, and data handling. |
| SRA-021-EX03 | 2 | Design an evaluation set and procedure that lets you compare two models fairly on your own task. |
| SRA-021-EX04 | 3 | For three use cases choose prompting, RAG, or fine-tuning, and state the evidence that would make you escalate. |
| SRA-021-EX05 | 4 | Build a three-year cost model for the listing assistant with usage, price change, and review effort; compute the break-even against self-hosting. |
| SRA-021-EX06 | 4 | Write the exit plan for a single-provider dependency and a data-readiness checklist for a RAG use case. |
Show solutions
- EX01 Example rejects: a fully automated price-setting assistant (high risk, unclear value) and an AI feature that duplicates a rules engine. Keep ideas with a measurable outcome, adequate data, and acceptable risk.
- EX02 Hosted: low start cost, scales with use, data leaves under provider terms; self-hosted: control and residency, GPU and staffing burden; managed: in-cloud controls at a premium. Check latency and terms by verification.
- EX03 Representative inputs with expected outputs, scoring rubric (human and automatic), blind comparison, cost and latency recorded, rerun for each candidate model.
- EX04 Support assistant over changing documentation: prompting plus RAG. Listing assistant with a consistent house style: prompting first; fine-tune only if evaluation shows style failures persist. Product search: ranking signals and retrieval first. Escalate on measured quality gaps.
- EX05 Multiply requests by tokens by unit price, add review time, evaluation and staffing; self-hosting adds GPUs, utilization, redundancy, and engineers; break-even is the volume where totals cross; show sensitivity to price drops.
- EX06 Exit plan: abstraction layer, portable prompts and evaluation suite, second provider tested, data export. Checklist: owner, freshness, quality checks, sensitivity class, access control, deletion path.
Outcomes 1 and 4 are also covered by the lab defense and the quiz.
Workplace evidence: an AI investment decision you shaped with a cost and value case.
Quiz
Outcomes: O1 finding AI value · O2 hosted, open-weight, managed · O3 prompting, RAG, fine-tuning · O4 AI cost, lock-in, data foundations.
A self-check for you: nothing is graded, saved or sent anywhere. Choose an answer, then check it. You can try again as often as you like.
Practical part
Six case exercises SRA-021-EX01 to EX06 plus the lab, defended before a role-played CTO and CFO. Pass: at least five of six exercises correct, quiz at least 8/10, and a strategy with at least one idea rejected with reasons and costs traceable to usage assumptions.
Common mistakes
AI strategy as a list of demos; fine-tuning by default; ignoring data quality and ownership; single-provider dependency without an exit plan.
Assessment
Strategy defense before a role-played CTO and CFO.
Workplace evidence
An AI investment decision you shaped with a cost and value case.
Career relevance
AI solution architecture (ARC-045) and AI vendor evaluation (ARC-047).