Does pooling different models keep restoring diversity as models converge?

Research question

Across successive generations of language models from several providers, how much diversity of ideas does pooling models add compared with one model, and is that gain shrinking as models' outputs and errors become more alike?

Why it matters

The design position (P3) rests on remedies such as pooling ideas across vendors, varied personas and chain-of-thought, which raised diversity almost to human levels in one study. But larger, more accurate models share their errors across providers, and different models give similar creative answers. If the pooling gain shrinks with each model generation, the design remedy has an expiry date.

The debate lists this among what we still do not know: Does pooling models and prompts keep working as models' errors and outputs converge?

Evidence that motivates it

1. Ideas generated by language models for a new product had higher average purchase intent than human ideas and were 7 times more likely to rank in the top 10%, but were less diverse; pooling ideas across vendors, chain-of-thought prompting and varied personas or constraints raised their diversity “almost to the level of human idea generation”. (LD09-SL-C17 v1.0)
   Pooling across vendors, chain-of-thought and personas raised the diversity of product ideas almost to human levels.
   Cites: Terwiesch et al. (2026), “AI and Its Impact on Creativity…”
2. Across more than 350 language models, when two models both answer a leaderboard question wrongly they give the same wrong answer 60% of the time on one dataset, and larger, more accurate models have highly correlated errors even across different architectures and providers. (LD09-SL-C25 v1.0)
   Larger, more accurate models have highly correlated errors across architectures and providers.
   Cites: Kim et al. (2025), “Correlated Errors in Large Language…”
3. In a large dynamic experiment where each participant saw example ideas from ChatGPT or from earlier participants before brainstorming, high exposure to AI ideas did not make individual ideas more creative but increased the amount and rate of change of collective idea diversity: in the authors’ words, “AI made ideas different, not better”. (LD09-SL-C16 v1.0)
   AI ideas used as passive examples increased collective diversity.
   Cites: Ashkinaze et al. (2025), “How AI Ideas Affect the…”
4. Across 27 language models, 155 topics and 12 countries, the epistemic diversity of models’ answers rose substantially over three years, but every model was less diverse than a web-search baseline, larger models were less diverse than smaller ones, and for country-specific topics models’ built-in knowledge reflected English more than local-language knowledge. (LD09-SL-C32 v1.0)
   Larger models gave less diverse answers than smaller ones.
   Cites: Wright et al. (2026), “What and whose knowledge?…”

Claims are shown as they read on the day this pack was built; claim pages keep every earlier version.

A possible design

Fix a set of open-ended ideation and writing tasks with human baselines. For each provider, collect outputs from model versions released from 2023 to 2026 (open-weight and archived API versions), under single-model, pooled and diversity-prompted conditions, with equal numbers of ideas. Measure idea-level diversity with blind human coding of a sample and validated embeddings for the rest, and estimate how the pooled-minus-single gain changes with release date and capability, with preregistered hypotheses.

Data needed

Access to archived model versions; a task set with human baselines; rater time for idea coding.

Feasibility

One PhD chapter or paper. Low to medium; mostly inference costs and rater time. Retired closed models may be unavailable, which biases the sample toward open-weight models; record exact versions and dates.

Has this been done?

Partly done already: related work covers some of this question (first-pass screen of 15 October 2026; not yet checked in depth).

Studies show homogeneity across current models, a preliminary decline in output diversity across three years of releases, and gains from ensembles on benchmarks, but none was found that tracks the diversity gain from pooling across model generations.

Where the search could not reach. Screened by two web searches and arXiv record lookups on the date shown; HCI and creativity-research venues were not searched separately. Absence of a hit is not proof that the work does not exist.

Before building a project on it, run your own search in a scholarly database such as Scopus, Web of Science, Google Scholar or SSRN.

Nearest work found

• Wenger & Kenett, "We're Different, We're the Same: Creative Homogeneity Across LLMs," arXiv:2501.19361 (2025) Judged from an abstract, summary or search listing only
  Abstract read. Responses of many models to standard creativity tests are more similar to each other than humans' responses are; one point in time.
• Mathur, Boscoboinik, Tsai & Yager, "LLM Jaggedness Unlocks Scientific Creativity," arXiv:2605.10574 (2026) Judged from an abstract, summary or search listing only
  Abstract read. 19 base models from 8 providers on a scientific-idea benchmark; pooled meta-model ensembles outperform single models; counts valid ideas, not diversity over generations.
• Vallecillos-Ruiz, Hort & Moonen, "Wisdom and Delusion of LLM Ensembles for Code Generation and Repair," arXiv:2510.21513 (2025) Judged from an abstract, summary or search listing only
  Abstract read. Ensembles of ten models from five families; consensus selection falls into a "popularity trap" while diversity-based selection works; correctness, not creative diversity.
• Patel, Crossman, Aggarwal & Wenger, "Are LLMs becoming similarly creative? Evidence from three years of models," arXiv:2608.19437 (2026) Judged from an abstract, summary or search listing only
  Abstract read. A preliminary analysis finding a statistically significant decrease in model output diversity over three years of releases, on open-ended user queries and a creativity test; it tracks convergence but not whether pooling still adds diversity. It points the other way from the rise in epistemic diversity reported in LD09-SL-C32, on a different task.
• Haase, Hanel & Pokutta, "Has the Creativity of Large-Language Models peaked? An analysis of inter- and intra-LLM variability," arXiv:2504.12320 (2025) Judged from an abstract, summary or search listing only
  Abstract read. 14 models on two creativity tests; no rise in creative performance over 18–24 months and large variability within one model; scores, not pooled diversity.

References

Sources behind the motivating claims:

• Terwiesch, Meincke, Girotra, Mollick, Nave & Ulrich, “AI and Its Impact on Creativity and Diversity: An Empirical Study of LLM-Generated Product Ideas,” arXiv:2607.27553, 2026 (SRC-SL-TERWIESCH-ET-AL-ARXIV-2607-27553, original)
• Kim, Garg, Peng & Garg, “Correlated Errors in Large Language Models,” ICML 2025, arXiv:2506.07962 (SRC-SL-KIM-ET-AL-CORRELATED-ERRORS-ICML-2025, original)
• Ashkinaze, Mendelsohn, Li Qiwei, Budak & Gilbert, “How AI Ideas Affect the Creativity, Diversity, and Evolution of Human Ideas: Evidence From a Large, Dynamic Experiment,” ACM Collective Intelligence 2025, arXiv:2401.13481 (SRC-SL-ASHKINAZE-ET-AL-CI-2025, original)
• Wright, Masud, Moore et al., “What and whose knowledge? Measuring epistemic diversity in large language models,” arXiv:2510.04226, accepted to EMNLP 2026 (SRC-SL-WRIGHT-ET-AL-EPISTEMIC-DIVERSITY-ARXIV-2510-04226, original)

BibTeX

The first entry cites this opportunity; then its nearest work; then the sources above. Nearest work and sources are recorded as reference strings, so each goes in whole as the title; add authors and year from the original before you cite it.

@misc{sociolens_ro_ld09_04_v1,
  author = {{SocioLens project}},
  title = {Does pooling different models keep restoring diversity as models converge?},
  year = {2026},
  date = {2026-10-15},
  howpublished = {\url{https://sociolens-web.pages.dev/opportunities/ro-ld09-04/v1/}},
  note = {Research opportunity RO-LD09-04, version v1}
}

@misc{ro_ld09_04_nearest_1,
  title = {Wenger \& Kenett, "We're Different, We're the Same: Creative Homogeneity Across LLMs," arXiv:2501.19361 (2025)},
  url = {https://arxiv.org/abs/2501.19361},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated; Judged from an abstract, summary or search listing only}
}

@misc{ro_ld09_04_nearest_2,
  title = {Mathur, Boscoboinik, Tsai \& Yager, "LLM Jaggedness Unlocks Scientific Creativity," arXiv:2605.10574 (2026)},
  url = {https://arxiv.org/abs/2605.10574},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated; Judged from an abstract, summary or search listing only}
}

@misc{ro_ld09_04_nearest_3,
  title = {Vallecillos-Ruiz, Hort \& Moonen, "Wisdom and Delusion of LLM Ensembles for Code Generation and Repair," arXiv:2510.21513 (2025)},
  url = {https://arxiv.org/abs/2510.21513},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated; Judged from an abstract, summary or search listing only}
}

@misc{ro_ld09_04_nearest_4,
  title = {Patel, Crossman, Aggarwal \& Wenger, "Are LLMs becoming similarly creative? Evidence from three years of models," arXiv:2608.19437 (2026)},
  url = {https://arxiv.org/abs/2608.19437},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated; Judged from an abstract, summary or search listing only}
}

@misc{ro_ld09_04_nearest_5,
  title = {Haase, Hanel \& Pokutta, "Has the Creativity of Large-Language Models peaked? An analysis of inter- and intra-LLM variability," arXiv:2504.12320 (2025)},
  url = {https://arxiv.org/abs/2504.12320},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated; Judged from an abstract, summary or search listing only}
}

@misc{src_sl_terwiesch_et_al_arxiv_2607_27553,
  title = {Terwiesch, Meincke, Girotra, Mollick, Nave \& Ulrich, “AI and Its Impact on Creativity and Diversity: An Empirical Study of LLM-Generated Product Ideas,” arXiv:2607.27553, 2026},
  url = {https://arxiv.org/abs/2607.27553},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{src_sl_kim_et_al_correlated_errors_icml_2025,
  title = {Kim, Garg, Peng \& Garg, “Correlated Errors in Large Language Models,” ICML 2025, arXiv:2506.07962},
  url = {https://arxiv.org/abs/2506.07962},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{src_sl_ashkinaze_et_al_ci_2025,
  title = {Ashkinaze, Mendelsohn, Li Qiwei, Budak \& Gilbert, “How AI Ideas Affect the Creativity, Diversity, and Evolution of Human Ideas: Evidence From a Large, Dynamic Experiment,” ACM Collective Intelligence 2025, arXiv:2401.13481},
  url = {https://arxiv.org/abs/2401.13481},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{src_sl_wright_et_al_epistemic_diversity_arxiv_2510_04226,
  title = {Wright, Masud, Moore et al., “What and whose knowledge? Measuring epistemic diversity in large language models,” arXiv:2510.04226, accepted to EMNLP 2026},
  url = {https://arxiv.org/abs/2510.04226},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}