Skip to content
← Back to projects

SPAR 2026 Project: Differential Data for Automated AI Safety Research

September 2026

🤖AI Transparency Statement: This was fully written by me, and has not gone through any review with AI.

Introduction

Supervised Program for Alignment Research (SPAR) pairs up mentees and mentors for a ~3 month project, “that enables aspiring AI safety and policy researchers to work on impactful research projects with professionals in the field”. Luckily, they’re now expanding to generalist roles like myself too!

I was selected to work on Differential Data for Automated AI Safety Research, which explores how to automatically gather data that contains the context of what happens along the way during AI safety research to resurface later for meta-analysis to drive future research.

This page will track the project, my contributions, our outputs and any outcomes.

Project Stage

Last Updated: September 14th, 2026

Kickoff: We kickoff this week meeting as a team, opening up our problem and how we’re going to work on it.

I plan to update this page ~bi-weekly with what I feel comfortable publicly sharing until we reach our final outputs.

Overall Project Idea and Hypothesis

To be clear, the core project was defined by our mentor, Alec Harris in the SPAR Fall 2026 Projects. I’m joining in to evolve that and drive it towards its final outputs and outcomes.

Below is my own expansion of the project. Think about it this way:

When a research project concludes, a polished paper is published, potentially along with some code and final datasets. That’s great for someone that wants to quickly learn about the project and potentially how to apply it to whatever they’re interested in. But, it doesn’t reflect the messy reality of how the project evolved along the way. What doesn’t get shared is all the messy context that happened along the way. The messages, meetings and more where analysis was performed and decisions were made are discarded, but may still have valuable information that could help inform future research.

Further, this creates a bit of a survivor bias both on ideas, but especially on projects. A project could be abandoned for a myriad of reasons that don’t reflect the value of the research itself, and generally abandoned projects don’t get a publication, which means that we can’t learn from them. When working on online A/B experimentation we worked to change the culture where only wins were celebrated, because we posited that potentially the most important learnings are actually from failures.

The hypothesis is: If we can automatically collect the data and context from throughout a research project, whether it completed to the point of publication or not, it can then be recalled for meta-analysis. This meta-analysis can inform future research to skip or go deeper on, what techniques yield the best results (or not), and more. This can speed up the AI safety research learning loop, helping close the gaps to frontier developments.

Project Plan

This is currently just an estimate on my part based on the program timelines. Can update later as we learn more.

  • Weeks 1-4: Define the project further, assign team roles and ways of working, and begin engaging with users to answer core questions or begin reviewing proofs of concepts and early plans.
  • Weeks 4-6: Submit mid-term outline to SPAR, and receive feedback
  • Weeks 4-8 (overlaps with above): Move from discovery and requirement gathering into definition and early validation steps, gathering feedback and adjusting. Begin writing the final report.
  • Weeks 8-12: Finalize requirements, work with users to implement, gather feedback and iterate. Prepare the final report and presentation.
  • December 14-19th: Final report due then presentations.

My Contribution and What I Want to Learn

I expect I’ll end up in a bit of a project lead role (the mentor is not expected to organize and coordinate the work) based on my past experience. I’ll be happy to help mentor on everything from project management, prioritization, product fundamentals, user interviews, and more. I’d like to conduct some of the user interviews myself, and help write our question set for them. I’d be happy to assist in coding including Claude Code, but may find myself more in technical lead, TPM or systems analyst role coordinating with the users and their IT teams. I’d like to have a strong role in crafting the final report and presentation, but it’s not as important to me if I’m the one to present them.

I anticipate bringing a very pragmatic and user-focused lens to the work. There will likely be strong privacy and security requirements to work against, including coordinating with organization(s) IT teams. From a user-perspective, I expect that this will have to happen with as low effort as possible – I have lots of experience telling me that users are not going to want to constantly be moving data manually, or even tagging it or otherwise triggering events manually.

This project is a great example of the meta type solutions that I would like to work alongside AI safety organizations to build. This will give me deeper insights into how organizations operate internally and think about these problems. It will give me a much stronger version of their user personas and the types of requirements that this space requires. Ultimately, I hope it will be a good demonstration of how I can work alongside these organizations to build a project that helps them deliver more value.

Outputs

Once the project is completed in December 2026, I expect to have our final presentation + potentially GitHub or other code available for sharing.

Expected Outcomes

Given the part-time nature of the work, short timeframe and not being embedded with an organization, I think the outcome of this work will largely be a proof (or not) that gathering ‘differential data’ is feasible, how that might work and what the next steps might be.

If successful, this could be a stepping-stone to more robust solutions that could indeed speed up the learning loop of AI safety research, and help it catch up to frontier lab developments.