| NSF Org: |
SES Division of Social and Economic Sciences |
| Recipient: |
|
| Initial Amendment Date: | June 20, 2017 |
| Latest Amendment Date: | June 24, 2020 |
| Award Number: | 1743044 |
| Award Instrument: | Standard Grant |
| Program Manager: |
Tara Behrend
SES Division of Social and Economic Sciences SBE Directorate for Social, Behavioral and Economic Sciences |
| Start Date: | September 1, 2017 |
| End Date: | August 31, 2021 (Estimated) |
| Total Intended Award Amount: | $49,960.00 |
| Total Awarded Amount to Date: | $49,960.00 |
| Funds Obligated to Date: |
|
| History of Investigator: |
|
| Recipient Sponsored Research Office: |
1523 UNION RD RM 207 GAINESVILLE FL US 32611-1941 (352)392-3516 |
| Sponsor Congressional District: |
|
| Primary Place of Performance: |
FL US 32611-7150 |
| Primary Place of
Performance Congressional District: |
|
| Unique Entity Identifier (UEI): |
|
| Parent UEI: |
|
| NSF Program(s): | SoO-Science Of Organizations |
| Primary Program Source: |
|
| Program Reference Code(s): |
|
| Program Element Code(s): |
|
| Award Agency Code: | 4900 |
| Fund Agency Code: | 4900 |
| Assistance Listing Number(s): | 47.075 |
ABSTRACT
The research community of Science of Organizations and other social, behavioral, and economic sciences face a number of challenges related to promoting robustness and reliability in our research. The workshop aims to address these challenges and offer an opportunity to discuss best practices regarding how to improve (1) reproducibility; (2) replicability; (3) generalizability; and (4) the rigor of peer review with an explicit focus on robustness and reliability. Each of the four aims for improvement has intellectual merits in theory development, methodology, as well as publishing and reviewing. The workshop will promote the awareness about and the standards of robust and reliable science. It serves educational purposes globally by allowing researchers at all locations to access the online video library that will be generated by this workshop. Participants will gain a deeper understanding about the best research practice and transmit the learning to their network. Results from this workshop can help develop insights into and awareness about issues related to robust and reliable research, and improve researcher abilities to conduct and publish robust and reliable science.
The objective of the workshop is to promote and develop robust and reliable research practice in Science of Organizations and other social, behavioral, and economic sciences by focusing on three aspects of research activities. First, in theory development, the discussion aims to focus on good practices for achieving rigor, transparency and exactness; adding replication to the research trajectory; good practices for testing existing theories in new contexts; as well as examining the cause, practice, and consequence of irresponsible research practices. Second, in methodology, the discussion aims to address transparency concerning the data gathered, the design, the analysis methods, and the results; the collection of new data for a fair test of the original study; useful methodologies to uncover the conditions under which a theory may not be applicable; as well as registered trials. Finally, in publishing and reviewing, the discussion aims to address the debates about requiring data availability as a part of the peer review process; debunking studies and theories that failed to survive attempted refutations; as well as evaluating studies that demonstrate a lack of statistical support. The workshop will include participants sampled from thought leaders and major journal editors from a variety of disciplines, and scholars at different career stages to participate in the discussion. Results of the workshop will be archived and disseminated to reach a wide audience.
PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH
Note:
When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external
site maintained by the publisher. Some full text articles may not yet be available without a
charge during the embargo (administrative interval).
Some links on this page may take you to non-federal websites. Their policies may differ from
this site.
PROJECT OUTCOMES REPORT
Disclaimer
This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.
The research community of Science of Organizations and other social, behavioral, and economic (SBE) sciences face a number of challenges related to promoting robustness and reliability in our research. As defined in the SBE Advisory Committee 2015 report, Social, Behavioral, and Economic Sciences Perspectives on Robust and Reliable Science, "robust and reliable" science refers to research that is reproducible, replicable, and generalizable. This project outcomes report summarizes the educational materials and the professional development opportunities that the project has created for the purpose of promoting Robust and Reliable Research Practices.
The project outcomes cover three aspects of research activities--theory development, methodology, and the process of quality control through rigorous peer review. In theory development, the focus is on good practices for achieving rigor, transparency and exactness; adding replication to the research trajectory; good practices for testing existing theories in new contexts; as well as examining the cause, practice, and consequence of irresponsible research practices. In methodology, the focus is on transparency concerning the data gathered, the design, the analysis methods, and the results; the collection of new data for a fair test of the original study; useful methodologies to uncover the conditions under which a theory may not be applicable; as well as registered trials. In the process of quality control through rigorous peer review, the focus is on the debates about requiring data availability as a part of the peer review process; as well as retracting studies that fail to survive attempted refutations.
The project's broader impact is on researchers' practices regarding how to improve (1) reproducibility by enhancing researchers' ability to generate the same results from the data of the initial study under the conditions initially specified; (2) replicability by enhancing researchers' ability to repeat the initial study but with new data and other potential departures from the initial study (e.g., new sample, new measures); (3) generalizability by enhancing researchers' ability to verify whether the finding of a subsequent study applies in other contexts or populations that differ from the initial study; and (4) falsifiability by enhancing researchers' ability to detect the extent to which a hypothesis has survived attempts of refutation. During 2017-2021, a collection of best practices has been developed for researchers through this project regarding possible solutions to improve (1) reproducibility; (2) replicability; (3) generalizability; and (4) the rigor of peer review with an explicit focus on robustness and reliability. Each of the four aims for improvement has intellectual merits in theory development, methodology, as well as the process of quality control through rigorous peer review.
Providing educational materials and professional development opportunities equitably and inclusively has been the principal investigator's life-long commitment to the training and upgrading of human resources by improving authors' and editors' abilities to generate and publish robust and reliable knowledge. The tangible impact on the development of human resources is: (1) Improving methodological approaches through rigorous training in statistics and research practices; (2) Training our next-generation scholars and publication gatekeepers on fraud detection.
The educational materials, which combine the video recordings of speaker presentations and the speakers' recommended digital readers, are available in the public domain for equal and easy access. All the materials are archived and disseminated to reach a diverse audience via online platforms. The professional development opportunities are the conferences, symposia, and workshops that the principal investigator has provided and continues to provide after the project is complete.
A website developed to host the educational materials is located at the University of Florida (UF) College of Business and connected to YouTube video:
https://warrington.ufl.edu/reliable-research-in-business/
https://warrington.ufl.edu/reliable-research-in-business/best-practices/
A set of commentaries has been curated from some of these speakers and published in a Special Issue in the Journal of Management (Volume 46 Issue 7, September 2020). The theme of the special issue co-edited by the principal investigator is Embracing Robustness and Reliability in the Science of Organizations.
In addition to creating and producing the educational content, the principal investigator collaborates with many professional associations in promoting and disseminating the content for training and professional development. For example, a collaboration with the Strategic Management Society created a conference theme of Fostering Robust and Reliable Research in Strategic Management in 2019. As another example, a collaboration with the Academy of Management in 2021 created a virtual symposium on formal models in strategic management research. A video recording of the symposium is provided via a YouTube channel for sustaining the continuity of this NSF project's mission. As a further example, a collaboration with the Academy of International Business has created a research methods workshop in 2022 on the topic of "Surprises, mixed findings, or questionable research practices? A methodology for evaluating cumulative empirical analyses and replication studies." These collaborations add to the principal investor's numerous presentations at research seminars, junior faculty consortia, doctoral student workshops, and plenary sessions on robust and reliable science.
Last Modified: 12/21/2021
Modified by: Gwendolyn K Lee
Please report errors in award information by writing to: awardsearch@nsf.gov.
