Samuel Dooley

LiveBench: A Challenging, Contamination-Free LLM Benchmark.
Colin White*, Samuel Dooley*, Manley Roberts*, Arka Pal*, Benjamin Feuer, Siddhartha Jain, Ravid Shwartz-Ziv, Neel Jain, Khalid Saifullah, Siddartha Naidu, Chinmay Hegde, Yann LeCun, Tom Goldstein, Willie Neiswanger, Micah Goldblum.
Preprint, 2024.
[Link] [HuggingFace] [Press]

Style over Substance: Failure Modes of LLM Judges in Alignment Benchmarking.
Benjamin Feuer, Micah Goldblum, Teresa Datta, Sanjana Nambiar, Raz Besaleli, Samuel Dooley*, Max Cembalest, John P Dickerson.
Preprint, 2024.
[Link]

Large Language Models Must Be Taught to Know What They Don’t Know.
Sanyam Kapoor, Nate Gruver, Manley Roberts, Katherine Collins, Arka Pal, Umang Bhatt, Adrian Weller, Samuel Dooley, Micah Goldblum, Andrew Gordon Wilson.
NeurIPS, 2024.
[Link]

Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive.
Arka Pal, Deep Karkhanis, Samuel Dooley, Manley Roberts, Siddartha Naidu, Colin White.
Preprint, 2024.
[Link] [HuggingFace] [Press]

Multi-objective Differentiable Neural Architecture Search.
Rhea Sanjay Sukthanker, Arber Zela, Benedikt Staffler, Samuel Dooley, Josif Grabocka, Frank Hutter.
Preprint, 2024.
[Link]

Calibration-Tuning: Teaching Large Language Models to Know What They Don't Know.
Sanyam Kapoor, Nate Gruver, Manley Roberts, Arka Pal, Samuel Dooley, Micah Goldblum, Andrew Gordon Wilson.
Workshop on Uncertainty-Aware NLP at EACL, 2024.
[Link]

To the Cutoff... and Beyond? A Longitudinal Perspective on LLM Data Contamination.
Manley Roberts, Himanshu Thakur, Christine Herlihy, Colin White, Samuel Dooley.
International Conference on Learning Representations (ICLR), 2024.
[Link]

Rethinking Bias Mitigation: Fairer Architectures Make for Fairer Face Recognition.
Samuel Dooley, Rhea Sanjay Sukthanker, John P Dickerson, Colin White, Frank Hutter, Micah Goldblum.
Neural Information Processing Systems (NeurIPS), 2023.
🎉Oral Presentation🎉
[Link]

ForecastPFN: Synthetically-Trained Zero-Shot Forecasting.
Samuel Dooley, Gurnoor Singh Khurana, Chirag Mohapatra, Siddartha Venkat Naidu, Colin White.
Neural Information Processing Systems (NeurIPS), 2023.
[Link]

Fairer and More Accurate Tabular Models Through NAS.
Richeek Das and Samuel Dooley.
Algorithmic Fairness through the Lens of Time - Workshop at NeurIPS, 2023.
[Link]

A Natural Experiment on LLM Data Contamination in Code Generation.
Manley Roberts, Himanshu Thakur, Christine Herlihy, Colin White, and Samuel Dooley.
I (Still) Can't Believe It's Not Better! Workshop at NeurIPS, 2023.
🎉Contributed Talk🎉
[Link]

Expanding Robustness in Responsible AI for Novel Bias Mitigation.
Samuel Dooley.
PhD Thesis 2023.
[Link]

A deep dive into dataset imbalance and bias in face identification.
Valeriia Cherepanova, Steven Reich, Samuel Dooley, Hossein Souri, Micah Goldblum, Tom Goldstein.
AAAI/ACM Conference on AI, Ethics, and Society (AIES), 2023.
[Link]

Giraffe: Adventures in expanding context lengths in LLMs.
Arka Pal, Deep Karkhanis, Manley Roberts, Samuel Dooley, Arvind Sundararajan, Siddartha Naidu.
Preprint, 2023.
[Link]

How library IT staff navigate privacy and security challenges and responsibilities.
Alan F. Luo, Noel Warford, Samuel Dooley, Rachel Greenstadt, Michelle Mazurek, and Nora McDonald.
USENIX Security Symposium (USENIX), 2023.
[Link]

Robustness Disparities in Face Detection.
Samuel Dooley, Geroge Z Wei, Tom Goldstein, and John P Dickerson.
Neural Information Processing Systems (NeurIPS), 2022.
[Link] [Press]

Ctrl-shift: How privacy sentiment changed from 2019 to 2021.
Angelica Goetzen, Samuel Dooley, and Elissa M Redmiles.
Privacy Enhancing Technologies Symposium (PoPETs), 2022.
[Link]

The dichotomous affiliate stable matching problem: Approval-based matching with applicant-employer relations.
Marina Knittel, Samuel Dooley, and John P Dickerson.
International Joint Conference on Artificial Intelligence (IJCAI), 2022.
[Link]

Field Evidence of the Effects of Pro-sociality and Transparency on COVID-19 App Attractiveness.
Samuel Dooley, Dana Turjeman, John P Dickerson, and Elissa M Redmiles.
ACM Conference on Human Factors in Computing Systems (CHI), 2022.
🎉Best Paper Award Honorable Mention🎉
[Link]

PreferenceNet: Encoding human preferences in auction design with deep learning.
Neehar Peri, Michael J Curry, Samuel Dooley, and John P Dickerson.
Neural Information Processing Systems (NeurIPS), 2021.
[Link.]

Fairness through robustness: Investigating robustness disparity in deep learning.
Vedant Nanda*, Samuel Dooley*, Sahil Singla, Soheil Feizi, and John P Dickerson.
ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT), 2021.
[Link.]

Comparing Human and Machine Bias in Face Recognition.
Samuel Dooley, Ryan Downing, George Wei, Nathan Shankar, Bradon Thymes, Gudrun Thorkelsdottir, Tiye Kurtz-Miott, Rachel Mattson, Olufemi Obiwumi, Valeriia Cherepanova, Micah Goldblum, John P Dickerson, Tom Goldstein.
Preprint, 2021.
[Link]

Sequential decision making in resource constrained global health settings.
Samuel Dooley, Candice Schumann, Han-Chin Shing, John P Dickerson, and Philip Resnik.
ML For Global Health at ICML, 2020.
[Link]

Can an algorithm be my healthcare proxy?
Duncan C McElfresh, Samuel Dooley, Yuan Cui, Kendra Griesman, Weiqin Wang, Tyler Will, Neil Sehgal, and John P Dickerson.
Workshop on Health Intelligence at AAAI, 2020.
[Link.]

Libraries' approaches to the security of public computers.
Samuel Dooley, Michael Rosenberg, Elliott Sloate, Sungbok Shin, and Michelle Mazurek.
Workshop on Inclusive Privacy and Security at SOUPS-20, 2020.
[Link.]

ProportionNet: Balancing fairness and revenue for auction design with deep learning.
Kevin Kuo, Anthony Ostuni, Elizabeth Horishny, Michael J Curry, Samuel Dooley, Ping-yeh Chiang, Tom Goldstein, and John P Dickerson.
Preprint, 2020.
[Link.]

xView: Objects in context in overhead imagery.
Darius Lam, Richard Kuzma, Kevin McGee, Samuel Dooley, Michael Laielli, Matthew Klaric, Yaroslav Bulatov, and Brendan McCord.
ML for the Developing World at NeurIPS, 2018.
[Link] [Press]

Overhead detection: Beyond 8-bits and RGB.
Eliza Mace, Keith Manville, Monica Barbu-McInnis, Michael Laielli, Matthew Klaric, and Samuel Dooley.
Naval Applications of Machine Learning (NAML), 2018.
[Link.]