Voices from communities...
measuring the dark matter of AI
In another life I worked in survey design. It was a thankless job — if I am being completely honest. Clients never cared to understand the nuance of programmatic designs driving algorithms, how a respondents proclivity toward satisficing skewed results or how question design is critical for generating results for the specific questions you are asking.
Because I have both hemispheres in statistical modeling and social science qualitative assessment, I have learned to appreciate the limits of both probability and non-probability sampling.
A recent lecture by the London School of Economics, A Data Infrastructure for the Social Sciences, digs deeper into traditional survey design and a possible (plausible) evolution into something more meaningful and informative. In a guilty dalliance with over simplification, we can explore better survey design in the form of the American Voices Project.
The backbone includes the ability to probe into the dark matter of AI— social interactions.
If you have ever designed rigorous surveys you will understand why I wanted context to make relevance from the data collected. There are sophisticated methods to link questions in a manner that extends the utility of a survey. I wrote a book and a blog post that is still relevant:
Trends with Benefits: Improving Surveys
I remain intrigued by the ideas of generating more informative survey questions and continue to follow the thread introduced in the podcast regarding the American Voices Project:
*Generative Agent Simulations of 1,000 People
Naturally intrigued I clicked through to ChatGPT and completed a chunk of the interview although without an upgraded account I was summarily disconnected about 2/3rds of the way through.
My thoughts on the process so far? The agent seemed engaged although often forgot responses to prior questions but that could have been a way of validating responses. Ask questions in a variety of ways and check for congruence perhaps…
We need to add the layer of humanity to our scientific queries. I appreciate the push from the social sciences and encourage the integration of not only the spatial awareness of changes in the built infrastructure but the measurement of the wide variety of ways our communities feel the impact.