Data Collection and Generation: Using AI
For researchers, advancements in AI have created entirely new modes for collecting and generating data. This not only includes AI-powered tools for extracting information at scale1, but also the use of AI agents to manipulate2 and simulate human behavior3 (Figure 1). As a concrete example of such possibilities, one seminar participant reviewed a grant proposal they jokingly described as “social science for AI” (Figure 2, Panel A). More specifically, they argued that some interesting questions — like “how well can we predict changes in happiness around the world?” — can now be comprehensively investigated with little-to-no human involvement (Figure 2, Panel B). At least theoretically, AI tools can (1) compile the literature and derive hypotheses, (2) identify hypothesis-relevant data sources, (3) download and interoperate those data sources, and (4) deploy automated machine learning to establish initial estimates of the upper bound of predictability. The result, they further joked, is that it’s easier than ever to “publish fast, die unhappy anyways”. Whether that general sentiment is warranted and/or inevitable served as the focus of Week 6 of the Fall 2026 Collective [Artificial] Intelligence Seminar.
New to this week’s discussion were conversations around “silicon sampling”: using LLMs as substitutes for human participants. This technique is based on early evidence that LLMs can simulate output with many statistical similarities to real human responses4. Perhaps most impressively, a recent report indicated that GPT-4 could successfully simulate the results of 70 survey-based experiments conducted on nearly 120,000 participants3. When one considers the immense amount of resources that went into those original studies, silicon sampling presents massive cost savings and efficiency improvements. Such results are suggestive of a future where researchers use silicon samples to (1) pilot ideas before committing to costly experiments, (2) more rapidly test ideas, and (3) perhaps even replace human experimentation altogether (e.g., in contexts where we cannot ethically or feasibly run experiments with humans).
Areas of consensus — and dissensus
Like past weeks, there was an intentional attempt to identify both areas of agreement and disagreement5,6. Most seminar participants agreed that at least basic data extraction tasks could be completely handed off to AI tools. We further agreed that there are pressing methodological challenges in instances where constructs-of-interest are subjective. For example, if humans don’t agree on which verbal displays are appropriate in a given context7, there is not one but multiple ground truths to validate against. Furthermore, evidence that silicon samples can accurately replicate human responses in one context doesn’t mean that they can be expected to do so in the next8.
We most notably disagreed in conversations about how far things could and would go. For example, some seminar participants viewed silicon samples as a regular piloting technique in future research — others more so likened them to the popularity of existing simulation tools (e.g., computational modeling). We reflected upon the value of hard-earned scientific insights in a scenario where those patterns were predictable all along9. And we stared in the face of a real-world dual-use case: a public GitHub repository in which the same LLMs that may help us one day understand global happiness were subjected to an “AI torture chamber”10. Issues such as these set the backdrop for Week 7 of the Fall 2026 Collective [Artificial] Intelligence Seminar (see Data Collection and Generation: Critiquing AI).
