From chatbot to colleague: using AI in research

Justin Andrew Johnson, Department of Applied Economics, University of Minnesota

Four stages, each one closer to the work

  • Chatbot. It sees only what you paste in. You carry every answer back by hand.
  • Autocomplete. It sees the file you have open. You type, and it finishes the line.
  • Agent in the editor. It sees the whole repository and the terminal. You set the task and review the changes.
  • The app. It sees every repository, plus the web. You run several tasks and read the results.

Stage 1: a chatbot in a browser tab

  • It sees whatever you paste in
  • Good for explaining a method, one error message, a first draft
  • It never sees your data or what happened when you ran the code

Stage 2: autocomplete in VS Code

  • It sees the file you have open
  • Good for boilerplate, plotting code, syntax you half remember
  • You still drive every line, and it runs nothing

Stage 3: an agent, working inside the editor

  • You describe the task. It reads the code, edits the files, runs them and fixes what breaks
  • It sees the whole repository and the terminal output
  • Good for data ingestion, refactors, new pipeline steps, tests
  • In our stack: “Ingest the 30 m GLC_FCS30D land-cover series into base_data, with a legend and a readme”
  • The limit: every session starts knowing nothing about your project

The catch

  • Every session, the agent arrives as a brilliant new research assistant who has never seen your project
  • It does not know which library owns what, which folders are frozen, or how you name things
  • So we write that down, in a place it reads before it starts: a CLAUDE.md file

Our devstack: many repositories, one stack

  • Studies, one folder per paper: ngfs_pnas, nff_global, nz_brazil, nz_south_africa
  • Models: seals, global_invest, gtap_invest, gtappy, gep
  • Base library: hazelbean. Every model starts with import hazelbean as hb
  • Data: base_data, one folder per dataset
  • Umbrella: earth_economy_devstack, holding docs, specs and the shared guidance for AI agents

Nested CLAUDE.md: instructions at every level

~/.claude/CLAUDE.md                            me, on this machine (never committed)
jajohns_devstack/CLAUDE.md                     the workspace: an index of every repository
earth_economy_devstack/devstack_guidance.md    the whole team: shared rules, committed
seals/seals/CLAUDE.md                          one repository: install, test, one import

One line in each repository’s CLAUDE.md pulls in the shared rules:

@../../earth_economy_devstack/devstack_guidance.md

What we write in those files

  • Reuse before you build. Search hazelbean for an existing utility before writing a new one
  • Who owns what. A table with one row per repository and what belongs in it
  • What not to touch. Folders named old, archive or deprecated are read-only, and so is external_repos
  • Our rule of thumb: the second time you correct the agent, write the correction into the file

Stage 4: the app, with no editor in front of it

  • The same agent, pointed at a folder
  • Several tasks run side by side, each on its own branch
  • It sees every repository, plus the web and your documents
  • Good for long jobs, and for work that is not only code
  • The limit: you watch less of the work, so the record has to be good

Why git matters far more with an agent

  • Volume. An agent can change dozens of files in minutes. The diff is the only way to see all of it
  • Undo. Every commit is a checkpoint. A bad afternoon costs one command to roll back
  • Provenance. The history records who wrote what. Our agent’s commits end with a Co-Authored-By: Claude line
  • Parallel work. Each task gets its own branch, so two agents never write over each other

Six git words, and what each does for you

Word What it is What it gives you with an agent
commit A saved snapshot with a message The checkpoint you roll back to
diff What changed, line by line How you review the agent’s work
branch A separate line of work One task per branch, no collisions
origin The shared copy on GitHub The copy of record, off your laptop
push, pull Send to origin, fetch from it How work reaches your coauthors
pull request A proposed change, up for review Where a person signs off

Origin: the copy of record

  • Your laptop holds a working copy, where you and the agent edit, commit and branch
  • origin is the shared copy on GitHub that everyone’s copy agrees with
  • Coauthors, the cluster and an agent running in the cloud each hold a copy too, kept in step by push and pull
  • git remote -v tells you where a repository’s origin is
  • If it is not pushed to origin, it exists on one laptop

Three things to do this week

  • Put one project in git and push it to origin. A private GitHub repository is enough
  • Write a CLAUDE.md for it: what the project is, how to run it, what not to touch
  • Give the agent one small, checkable task, then read the diff before you keep it
  • You are still the author. The agent’s work is yours to check