Captchas in 2023 are cursed...
Remote
Quan Ze Chen
@cqz@hci.social
mastodon 4.7.30 Followers
0 Following
14 Posts
Joined May 12, 2022
Twitter:
Replying to
We observe that different sources of uncertainty are best resolved by applying _different_ interventions. E.g., adding context often disambiguates options for individuals, while deliberation can reduce group disagreement.
(2/n)
Replying to
Capturing and reducing uncertainty in human judgments: In Goldilocks, we introduce a novel annotation tool that uses ranges and anchors to capture ambiguity and disagreement separately in scalar rating tasks. arxiv.org/abs/2108.01799 (2/n)
Replying to
In the follow up, Judgment Sieve, we build upon these measurements to show that effective reduction of uncertainty should be targeted to the source.
arxiv.org/abs/2305.01615
(3/n)
Replying to
Defining Socially-Constructed Concepts for AI Alignment: In Case Law Grounding, we introduce a set of parallel human and AI (via prompting) workflows that uses cases as a medium to define preferences around nuanced socially-constructed concepts. arxiv.org/abs/2310.07019 (4/n)
Replying to
We follow this work up with Case Law for AI policy, where we present a democratic process for constructing a policy for AI based on case law, that aligns behaviors to community preferences.
https://social.cs.washington.edu/case-law-ai-policy/ (Website WIP)
(5/n)
Replying to
I'm currently looking for industry and academia positions. Please reach out if you're hiring! ❤️
https://homes.cs.washington.edu/~cqz/
(n/n)
Replying to
P.S.: I will be attending HCOMP '23 and later the MP2 Workshop at NeurIPS! If you're also going, I'd be happy to catch up in-person!
(n+1/n)
Replying to
In our paper, we present Judgment Sieve, a new workflow that uses uncertainty-aware human judgment collection to inform the application of interventions in a targeted way!
By measuring uncertainty on _each case_, we can pick what the best action to take is.
(3/n)
Replying to
We tested our workflow with crowd annotators on two scalar rating tasks---relatedness of word pairs and comment toxicity---and found that:
(1) different interventions indeed had affinity for different sources of uncertainty
(4/n)
Replying to
...and that:
(2) targeting the intervention is important, as uniformly applying the same intervention to all cases can _increase_ uncertainty on improperly targeted cases.
(5/n)
Replying to
We also discuss how Judgment Sieve can be incorporated as an iterative process, to address cases where uncertainty is a compound effect from multiple sources.
For details, check out our upcoming #CSCW2023 paper preprint (joint work w/
@axz)
http://arxiv.org/abs/2305.01615
(n/n)
