Award Abstract # 1620319
IBSS-L: Developing, Testing, and Designing from a Computational Theory of Online Communities

NSF Org: SMA
SBE Office of Multidisciplinary Activities
Recipient: CARNEGIE MELLON UNIVERSITY
Initial Amendment Date: August 9, 2016
Latest Amendment Date: August 9, 2016
Award Number: 1620319
Award Instrument: Standard Grant
Program Manager: Jeffrey Mantz
jmantz@nsf.gov
 (703)292-7783
SMA
 SBE Office of Multidisciplinary Activities
SBE
 Directorate for Social, Behavioral and Economic Sciences
Start Date: September 1, 2016
End Date: August 31, 2020 (Estimated)
Total Intended Award Amount: $919,979.00
Total Awarded Amount to Date: $919,979.00
Funds Obligated to Date: FY 2016 = $919,979.00
History of Investigator:
  • Robert Kraut (Principal Investigator)
    kraut@andrew.cmu.edu
Recipient Sponsored Research Office: Carnegie-Mellon University
5000 FORBES AVE
PITTSBURGH
PA  US  15213-3890
(412)268-8746
Sponsor Congressional District: 12
Primary Place of Performance: Carnegie Mellon University
5000 Forbes Avenue
Pittsburgh
PA  US  15213-3819
Primary Place of Performance
Congressional District:
Unique Entity Identifier (UEI): U3NKNFLNQ613
Parent UEI: U3NKNFLNQ613
NSF Program(s): Decision, Risk & Mgmt Sci,
Interdiscp Behav&SocSci IBSS
Primary Program Source: 01001617DB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s): 1321, 8213, 8605, 9179
Program Element Code(s): 132100, 821300
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.075

ABSTRACT

This interdisciplinary research project will build and test a computational theory describing the factors and processes that influence the success of online communities. The investigators will develop and test new theories during the course of this project in order to better predict community success at multiple levels of analysis, including members' support, community maintenance, production, and key stakeholder benefits. The computational theory produced in this research will provide new scientific insights to explain variations in success in existing online communities and new engineering insights that can be applied to improve design choices. The project will advance knowledge of how these design factors interact to affect community success and member experiences in online communities.

As the popularity of Wikipedia, Massively Open Online Courses (MOOCs), peer funding and lending sites, and online health support groups demonstrates, online communities have become commercially and societally important platforms for peer content production, information exchange, education, and social interaction. Although some communities succeed, the majority of newly created ones fail to survive or to achieve their goals to involve members or produce valuable artifacts. Even within a successful community like Wikipedia or the peer lending site Kiva, some subgroups are more successful than others. One reason for failure is the lack of evidence-based guidance for building and managing online communities as well as the paucity of techniques to predict the effects of design and management decisions before implementation. Prior research on online community success consists of empirical studies and specialized theories to explain single facets of community success, such as membership commitment or contribution. Commercial firms routinely use A/B testing to make specific design choices in their communities. There have been few attempts, however, to build a comprehensive, evidence-based theory to explain how online communities' attributes and processes interplay to determine their success. This investigators will use agent-based modeling, a computer simulation technique that models the decisions individual community members make in joining, contributing to, or leaving the community. They will draw on and integrate component theories from social psychology, economics, organizational behavior, and communications to model these decisions. They will use empirical data from three distinct communities -- crowd-lending teams, health support groups, and peer support forums within STEM classes -- to ground and test the model. Once empirical research has verified that the model can account for behavior in the communities as they currently exist, the model will be used to explore how to improve communities by instituting changes with respect to the size and diversity of subgroups, content and connections recommendations, and leaderboard design. The investigators will use virtual and field experiments to test predictions from the model. This project is supported through the NSF Interdisciplinary Behavioral and Social Sciences Research (IBSS) competition.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

(Showing: 1 - 10 of 29)
Carton, Samuel and Mei, Qiaozhu and Resnick, Paul "Feature-Based Explanations Don't Help People Detect Misclassifications of Online Toxicity" Proceedings of the International AAAI Conference on Web and Social Media , v.14 , 2020 , p.95
Chen, Yan and Cramton, Peter and List, John A and Ockenfels, Axel "Market Design, Human Behavior, and Management" Management Scienc2 , 2020 https://doi.org/10.1287/mnsc.2020.3659
Chen, Yan and Jiang, Ming and Kesten, Onur and Robin, St{\'e}phane and Zhu, Min "Matching in the large: An experimental study" Games and Economic Behavior , v.110 , 2018 , p.295--317 10.1016/j.geb.2018.04.004
Chen, Yan and Jiang, Ming and Kesten, Onur and Robin, St{\'e}phane and Zhu, Min "Matching in the large: An experimental study" Games and Economic Behavior , v.110 , 2018 , p.295--317
Chen, Yan and Lu, Fangwen and Zhang, Jinan "Social comparisons, status and driving behavior" Journal of Public Economics , v.155 , 2017 , p.11--20 10.1016/j.jpubeco.2017.08.005
Chen, Yan and Lu, Fangwen and Zhang, Jinan "Social comparisons, status and driving behavior" Journal of Public Economics , v.155 , 2017 , p.11--20
Chen, Yan and Lu, Fangwen and Zhang, Jinan "Social comparisons, status and driving behavior" Journal of Public Economics , v.155 , 2017 , p.11--20
Chen, Yan and YeckehZaare, Iman and Zhang, Ark Fangzhou "Real or bogus: Predicting susceptibility to phishing with economic experiments" PloS one , v.13 , 2018 , p.e0198213 10.1371/journal.pone.0198213
Chen, Yan and YeckehZaare, Iman and Zhang, Ark Fangzhou "Real or bogus: Predicting susceptibility to phishing with economic experiments" PloS one , v.13 , 2018 , p.e0198213
Chen, Yan and YeckehZaare, Iman and Zhang, Ark Fangzhou "Real or bogus: Predicting susceptibility to phishing with economic experiments" PloS one , v.13 , 2018 , p.e0198213
Chen, Yan, Ming Jiang and Erin Krupka. "Hunger and the gender gap." Experimental Economics , v.22 , 2019 , p.885 10.1007/s10683-018-9589-9
(Showing: 1 - 10 of 29)

PROJECT OUTCOMES REPORT

Disclaimer

This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.

This project was design build and test computational theories describing the factors and processes that influence the success of online communities and test these theories again empirical data.   Although some communities succeed, the majority of newly created ones fail to survive or to achieve their goals to involve member or produce valuable artifacts. Even within a successful community like Wikipedia or the peer-to-peer lending site Kiva, some subgroups are more successful than others. One reason for failure is the lack of evidence-based guidance for building and managing online communities, as well as a lack of techniques to predict the effects of design and management decisions before implementation.  The theories developed and tested in this project predict community success at multiple levels of analysis ?members? support, community maintenance, production, and key stakeholder benefits.

This project used agent-based modeling, a computer simulation technique that models the decisions individual community members make in joining, contributing to or leaving the community. It will be based on and integrate component theories from social psychology, economics, organizational behavior and communications to model these decisions. It also built game-theoretic models to predict and test how team size and complementarity of members? skill predict the extent to which the community produces public benefits. The bullet points below describe some of the results from this project:

  • Success in Wikipedia. We designed and built an agent-based model of to predict success of Wikipedia at multiple -levels, including the likelihood that new editors will continue participate, the number of articles produced and the quality of articles. This model was based on our literature review of success factors in Wikipedia and empirical studies showing when it its lifecycle article quality grows most quickly and how automated deletion of articles drives valuable editors away. We also conducted experiments to show ways to recruit experts to contribute to Wikipedia and to reduce the under-representation of women among Wikipedia editors. Experts are most likely to contribute when they are asked to contribute to topics that most closely match their expertise, and women are more likely to contribute when they are asked to write about a gender-neutral topic.
  • Complementarity and team size. Do differences among team members? skills and other production-relevant attributes and the size of a team influence how much members contribute to a group outcome? For example, people who contribute money to a accomplish a group goals are relatively interchangeable, while those who contribute unique skills are complementary. We developed a theoretical model that predicts that for low complementarity teams (i.e., members substitute for each other), members in large groups contribute more whereas for high complementarity groups, members in small groups contribute more. This prediction is confirmed in our lab experiment. Moreover, our empirical research among Kiva lending teams shows that Kiva teams that in which members occupy a diversity of social roles contribute more than to teams where the members occupy fewer distinct social roles.
  • Contributing a part of a group or individually. People contribute to some goal when they work with a group of other people than when they work alone. For example, joining a Wikiproject, which is a group of editors who curate Wikipedia articles on a defined topic, write more on articles that belong to the project but not to articles not connected to the project. Similarly, people who are experimentally encouraged to join Kiva peer lending teams contribute more money than those who do not get an invitation.
  • Designing leaderboards: We built theory and conducted experiments that show that leaderboards, which show members of a group how well people are doing compared to each other, increase the amount that people work and that a fine-grained leader board with complete ranking outperforms a coarser leaderboard with just two categories.
  • Identifying the roles members play in online communities. Using natural language processing and statistical models, we identified conversational acts that members perform when participating in online cancer support groups such as seeking informational support, providing empathy and self-disclosing information about themselves. We also identified 11 coherent social roles that members occupy in these groups, such as support provider, newcomer welcomer, and story sharer. Occupying certain roles such as newcomer-welcomer and encourager was positively correlated with members? sustained participation. In Kiva lending teams, we identified the conversational acts members perform, such as welcoming newcomers, encouraging others to give, reminding teammates of deadlines, or competing against other teams. Kiva teams that in which members occupy a diversity of social roles contribute more than to teams where the members occupy fewer distinct social roles.

 


Last Modified: 06/02/2021
Modified by: Robert Kraut

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page