Data analysis

What actually changed

• WasLearning the tool — SQL, pandas, ggplot — was a real fraction of learning the craft.

  NowThe syntax layer is nearly free, which is a genuine gift, and it removes the slow apprenticeship during which people used to absorb what the data meant.

• WasA wrong query usually errored, or returned something visibly absurd.

  NowIt returns a plausible number attached to a fluent explanation of why that number makes sense — the two failure signals you used to rely on, both muted at once.

• WasYou knew your data because you had spent three days cleaning it by hand.

  NowCleaning is generated in a minute, so provenance has to be learned deliberately: read the source, count the nulls, ask who is missing.

• WasCharts took long enough that you made few of them and thought about each.

  NowYou can make forty, which makes the discipline of deciding what you were asking BEFORE plotting it more valuable, not less.

The path

In order. Each stage names what the model may and may not do while you are in it — that rule is the difference between the two arms of every study at the bottom of this page.

1. 1 Ask a question that could be answered wrongly

   Learn to state an analysis as a claim with a shape, before touching data.

   • Write the question, the number that would answer it, and what you expect that number to be, in advance.
   • Say what would change if you were wrong. An analysis nobody would act on differently either way is a chart, not an answer.
   • Define the metric in one sentence a stranger could implement. Most disputes about numbers are disputes about definitions.

   What AI may do here It may sharpen the wording. It may not choose the question — the question encodes what you care about, and the model has no stake in your decision.

   You are done when Someone else can read your question and metric definition and produce the same number independently.

2. 2 Meet the data before you model it

   Build the habit that catches most real errors: knowing what the rows are and who is missing.

   • Count rows, nulls, duplicates and date ranges before anything else. Every time.
   • Find out how each column is produced. A field named active means whatever the system that writes it means, and it usually means something surprising.
   • Plot the raw distribution before any aggregate. Means hide bimodality, outliers and dead sensors with equal enthusiasm.
   • Ask who is not in this dataset — the churned, the unlogged, the bots, the people whose consent banner blocked the beacon.

   What AI may do here Let it write the profiling code; read the output yourself, line by line. The summary of the summary is where the surprise gets smoothed away, and the surprise was the point.

   You are done when You can describe your dataset's shape, gaps and biases from memory, and name one group it systematically misses.

3. 3 Learn the statistics that change decisions

   Acquire the small set of ideas that separate a real finding from a confident coincidence.

   • Variation and uncertainty first: what an interval means, what it does not, and why a difference between two noisy numbers is usually nothing.
   • Confounding and selection: the reason your best-converting channel looks best and the reason that may be an artefact.
   • Base rates, and what a small percentage of a large denominator does to your intuition.
   • Run one real experiment end to end — hypothesis, sample size decided in advance, analysis written before the data arrives.

   What AI may do here Excellent at explaining a concept and at doing the arithmetic once you have specified the design. Do not let it choose the test: the model will produce a defensible-sounding choice, and the assumptions it silently made are the analysis.

   You are done when You have killed one of your own findings by checking it — a difference that turned out to be noise, or a comparison that was confounded.

4. 4 Communicate it so it survives contact with a decision

   Turn a correct analysis into something someone acts on without misreading it.

   • One chart, one claim. Label the window the number covers, not just the metric.
   • Say the uncertainty out loud, in the sentence, not in a footnote.
   • Write the version for the person who will only read the first line.
   • Keep the analysis reproducible — a script and a query someone else can rerun in six months.

   What AI may do here It can draft the summary and critique your chart. Read every sentence it writes about your data against your own numbers: it will confidently round, generalise and smooth in ways that are wrong in exactly the direction that sounds better.

   You are done when A decision was made from your analysis, and the person who made it could state the caveat correctly afterwards.

5. 5 Automate, with the checks in the pipeline

   Get the speed of generated analysis without inheriting its silence.

   • Turn your manual checks into assertions that run every time — row counts, ranges, null rates, totals that must reconcile.
   • Version the metric definitions where the queries can see them, so two dashboards cannot disagree.
   • Ask the model for the analysis and for the check separately, and write the check first.
   • Re-derive one important number by a different method occasionally. Two roads to the same number is the cheapest audit there is.

   What AI may do here Generate freely at this stage, because you now have something that fails loudly. The rule is simply that no generated analysis reaches a decision without passing a check you wrote.

   You are done when A pipeline of yours has caught a real data error before a human did.

Where the model genuinely helps you learn

• Writing the query, the reshape and the plot once you have decided what you are asking — the compression here is large and costs you nothing you needed.
• Explaining an unfamiliar statistical method with your own data in front of you, which is far better than a textbook example.
• Generating the checks: ask what would have to be true for this result to be an artefact, then go and test each one yourself.
• Producing the alternative explanation you are too invested to see.
• Turning your finished analysis into the three versions different audiences need, which you then verify against your own numbers.

What counts as proof now

The artefact stopped being evidence — anyone can generate one. These are the things that still say something about you.

• An analysis where you predicted the answer in advance, were wrong, and found out why.
• A finding of your own that you killed by checking it.
• A reproducible pipeline with assertions that have actually fired.
• A decision someone made from your work, with the caveat they repeated back correctly.

Where to learn it

The evidence

What each source measured, and what kind of source it is. A practitioner study and a randomised trial are both worth reading and are not the same claim.

• Gu et al. (2024), CHI Peer-reviewed

  Qualitative study of 22 analysts verifying AI-assisted analyses: responses can be misaligned with intent or seemingly correct while leading to incorrect conclusions, and verification behaviour depended heavily on the analyst's own programming and analysis background.

• Brynjolfsson, Li and Raymond (2025), Quarterly Journal of Economics Peer-reviewed

  5,179 support agents: 14% average productivity gain, 34% for novices, near zero for experts — the model spreads existing best practice rather than creating new judgement, which is exactly the part of analysis that is scarce.

• Dell'Acqua et al. (2023/2025), Organization Science Peer-reviewed

  758 consultants: better and faster work inside the model's capability, and 19 percentage points worse than unaided colleagues on a task just outside it — a business-analysis task, which is the closest thing in the literature to this job.

• Noy and Zhang (2023), Science Peer-reviewed

  Professional writing tasks, with data analysts among the 453 participants: 40% less time and 18% higher graded quality, concentrated among the weaker performers.

Related skills