Digital Technologies and Sampling. Comparing Estimates From Different Recruitment – Approaches Within the GESIS Panel.dbd.

Using a dataset of the GESIS Panel.dbd from 2023, this article examines AI use and attitudes in Germany. Almost half of the respondents had tried ChatGPT while one in five had not heard of it. Both positive and negative attitudes toward AI were common. Most agreement was found on the statements that AI helps with work and can easily be misused by companies. In a second step, different recruitment approaches were compared using the absolute average relative bias. Participants recruited via Meta used ChatGPT more frequently than those recruited from the ALLBUS sample. However, the difference was smaller when looking at the attitudes toward AI. Overall, the findings suggest that most people had heard about ChatGPT, while frequent use was scarce, with mixed attitudes toward AI. Moreover, the results depend on the underlying sample.

Auf Grundlage der Daten des GESIS Panel.dbd von 2023 analysiert dieser Beitrag die Nutzung von und die Einstellungen gegenüber KI. Knapp die Hälfte der Befragten hatte ChatGPT mindestens einmal genutzt, während ein Fünftel noch nicht davon gehört hatte. Sowohl positive als auch negative Einstellungen gegenüber KI waren verbreitet. Am meisten Zustimmung fanden die Aussagen, dass KI bei der Arbeit helfe und von Unternehmen leicht missbraucht werden könne. In einem zweiten Schritt wurden verschiedene Ansätze der Rekrutierung miteinander verglichen. Teilnehmende, die über Meta rekrutiert wurden, nutzten ChatGPT häufiger als Teilnehmende aus der ALLBUS-Stichprobe. Unterschiede in Bezug auf Einstellungen gegenüber KI fielen dagegen geringer aus. Insgesamt deuten die Ergebnisse darauf hin, dass die meisten Befragten von ChatGPT gehört hatten, während eine regelmäßige Nutzung selten war. Die Einstellungen gegenüber KI waren gemischt. Zudem wurde deutlich, dass die Ergebnisse von der Stichprobe abhängen.

DOI: 10.34879/gesisblog.2025.112

---

The Spread of AI

About three years ago, on the 30^th of November 2022, the tech company OpenAI launched a generative artificial intelligence (gen AI) named ChatGPT (Introducing ChatGPT | OpenAI). In the months and years that followed, many further gen AI models were launched by other companies, resulting in a wide variety of such models becoming available to the public. We want to look back on the beginning of the AI boom in 2023: How many people have used ChatGPT? What attitudes did people have toward gen AI?

About the data: the GESIS Panel.dbd

To find out more, we used data from the first GESIS Panel.dbd Digital Behavioral Data Sample survey data release (https://doi.org/10.4232/1.14643). The GESIS Panel.dbd is a panel study that collects both survey as well as user centred digital behavioural data, such as web tracking data and covers a wide range of topics pertaining to digitalization. The panel is unique not only because it combines traditional survey data with digital behavioural data, but by virtue of its multifaceted recruitment approaches, which were both probabilistic and non-probabilistic in nature. The first data release encompasses two recruitment approaches leaving us with two distinct samples.  First, a non-probabilistic sample recruited through advertisement on Meta (n = 1,768) and second, a probabilistic sample recruited via piggybacking from the probability-based ALLBUS 2023 (n = 800). Consequently, the survey data we are looking at do not only give us insights into usage and attitudes at the beginning of the AI boom but also provides us with the opportunity to see how the recruitment approach and its resulting sample may change these insights. The Meta sample is a highly self-selective sample consisting only of people who use Meta platforms (Instagram and Facebook) and clicked on the advertisement there to sign up for the GESIS Panel.dbd. The ALLBUS 2023 on the other hand was recruited using probabilistic sampling strategies and as such should cover a more diverse pool of participants and be more representative of the German population than the Meta sample. Sample composition matters, when trying to generalize findings to a population of interest.¹ In the case of usage and attitudes towards gen AI it is likely that the findings differ because of the sample composition. Participants recruited through Meta are, by virtue of being registered with a Meta platform, already more attuned to the use of digital technologies. As such, they could show increased use of gen AI compared to a more representative sample, such as those recruited from the ALLBUS. This openness may come along with different attitudes toward AI as well. People using social media may generally have a more positive attitude towards new digital technologies. At the same time, they may be more informed about gen AI, which may lead to both greater interest and greater scepticisms. With this article we want to spread awareness on why sample composition matters and how it may influence one’s findings.

How widely was ChatGPT used?

Before looking at differences between samples, we present findings for the whole sample. In Figure 1, the total sample is represented by turquoise bars. The two samples ALLBUS and Meta are represented by purple and blue bars, respectively. Overall, less than 48 % had ever used ChatGPT at the end of 2023 and around 18 % did not know what ChatGPT was. As such, one year after the first release of ChatGPT, most people have heard of it, but its use was still moderate.

Figure 1

Responses to the question “Have you ever used ChatGPT?”, divided by groups.

What were people’s thoughts on AI?

Only a small group saw AI as a life improvement (see Figure 2). However, more than half of the participants agreed with the statement that AI helps with work. Negative attitudes were also common. Slightly less than 50 % feared that it can be easily misused by companies. Worries about AI making mistakes were less common, but still quite present. Thus, both positive and negative statements found agreement among participants.

Figure 2

Agreement to statements about AI, divided by groups.

Does Sampling Matter?

Turning back to Figure 1, we see differences between the ALLBUS (purple) and the Meta group (blue). Participants recruited via Meta stated that they have used ChatGPT more frequently, while the ALLBUS sample stated to have not used ChatGPT yet more frequently. To compare both groups analytically, we used the Average Absolute Relative Bias (AARB)², which calculates the average relative difference between a benchmark (here the ALLBUS sample) and a comparison sample (here the Meta sample) for every category under consideration. Looking at the use of ChatGPT, the AARB was 37 %, meaning that on average there was a 37 % difference in the relative frequencies of the responses between the ALLBUS and Meta sample. The biggest relative difference was observed in the response category “Regularly” where the percentage was twice as high for the Meta group as for the ALLBUS group.

Regarding attitudes towards AI, Figure 2 shows some differences between the groups especially for the items on making life better and making mistakes. However, looking at the AARB = 15 % over all four items, the difference was much smaller than for ChatGPT use.

What can be concluded?

One year after its launch, most people had heard about ChatGPT; however, frequent use was still scarce and the attitudes toward AI were mixed. Our sample comparison also highlights that the underlying samples matter and should be accounted for. Utilizing only the Meta sample would lead to the erroneous conclusion that ChatGPT was used more frequently at the time than it most likely was.

References

1. Cornesse, C., Blom, A. G., Dutwin, D., Krosnick, J. A., De Leeuw, E. D., Legleye, S., Pasek, J., Pennay, D., Phillips, B., Sakshaug, J., Struminskaya, B., & Wenz, A. (2020). A review of conceptual approaches and empirical evidence on probability and nonprobability sample survey research. Journal of Survey Statistics and Methodology, 8(1), 4-36.
2. Groves, R. M. (2006). Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly, 70(5), 646-675. https://doi.org/10.1093/poq/nfl033