I am a postdoc at the NYU Center for Data Science working with Sam Bowman. I earned my PhD from the University of Washington where my advisor was Luke Zettlemoyer.
I work on a variety of things, mostly focused on AI alignment or formal semantics of natural language. In alignment, I focus on scalable oversight and agent alignment, from the lens of task formulation, data collection, and evaluation methodology. I’m especially interested in using debate as a training and evaluation paradigm; see our recent paper. In language, I work on ways to design, annotate, and model semantics in a scalable, data-driven way while taking advantage of our understanding of linguistic structure; see my thesis summary paper. I have worked on approaches to crowdsourcing annotation for syntactic parsing, semantic role labeling, and predicate-argument structure.
Research
I have been involved in the development of QA-SRL and related formalisms, where I build systems for annotating and modeling linguistic structure at large scale. I have released two corpora:
- The QAMR Corpus — over 5,000 sentences annotated with Question-Answer Meaning Representation.
- The QA-SRL Bank 2.0 — over 64,000 sentences annotated with Question-Answer Semantic Role Labels.
For QA-SRL, we have an online explorer for the dataset. Code for manipulating, crowdsourcing, or browsing QA-SRL data is available here. Recently, I have shown that QA-SRL can be used to recover traditional discrete semantic roles.
I am also interested broadly in the Science of AI and NLP, using empirical methods to improve our understanding of intelligent behavior and language use. Along these lines, I have worked on broad-coverage and fine-grained evaluation of models, unsupervised discovery of linguistic structure, and explicitly incorporating ambiguity into task design. See my publications for a full list.
Random Thoughts
- A 10-minute, accessible colloquium talk summarizing some of my early PhD work on crowdsourcing representations of language structure and meaning.
- Philosophical Foundations for AI Ethics, a lecture given for Yulia Tsvetkov’s 2022 class on Computational Ethics in NLP.
- An in-depth review of a mid-2021 version of the OpenPhil Biological Anchors report on transformative AI timelines.
- To Dissect an Octopus, a blog post taking a deep dive into the form/meaning debate around language models.
- A long comment thread on the Alignment Forum discussing limits on the extrapolations we can make about automation potential based on ML benchmarks.
- Fulfilling Imperatives, an essay investigating of the semantics of imperative sentences.
- Modern Cosmology: Explaining the Universe, an essay investigating whether inflation theory qualifies as science.