About the ICPSR Bibliography of Data-related Literature

Purpose, Goal, Outcomes

Note: This report was written by Jeri Schneider in the early 2000s, in order to document how and why the ICPSR Bibliography of Data-related Literature was created.

The purpose of this project is to facilitate the use of the ICPSR holdings by providing a means to investigate previous research based on ICPSR data. Instructors often direct students to begin data-related research projects by reading some of the major works based on the data. Advanced researchers also find reading the existing literature to be an effective way to begin using a dataset. Reporters and policymakers looking for processed statistics look for reports explaining particular studies. Principal investigators and funding agencies often want to track how data are used after they are deposited.

The Bibliography makes it possible to:

Identify much of the research that has already been undertaken using a given ICPSR dataset
Replicate analyses in order to understand, evaluate, and build upon others' findings
Determine the usage patterns of data resources
Investigate the life cycle of data and the types of analyses undertaken
Learn more about methodological issues, some of which are covered solely in the published literature
Understand the limitations as well as the research potential of the data, by seeing the data in use and reading the observations and findings of other researchers
Avoid accidentally duplicating, in whole or in part, an analysis that has already been done
Identify cross-disciplinary implications and uses of the data

Our goal is to collect accurate citations of publications making reference to data archived and distributed by ICPSR. The outcomes of the project, when it is complete, will include:

Enhanced study descriptions with the addition of a comprehensive list of publications related to each dataset
An online database and bibliographic list of works that reference ICPSR data
Mechanisms for collecting, organizing, and displaying citations
A systematic plan to gather citations on an ongoing basis

While items 1 and 2 have already been accomplished, we continue to add to the citations in the database. The vehicles for collecting, organizing, and displaying citations are functioning, and we are in the process of developing more effective database mechanisms for managing the Bibliography.

CLOSE

Project Methodology

When this project began in July 2000, we identified its goals and developed a plan to systematically find and gather citations. This task involved developing a collection policy, selecting databases and print publications to search and/or browse, selecting a method for collecting and organizing citations, determining search strategies, and developing mechanisms for providing online access to the citations.

Collection Policy

In developing the collection policy, it was helpful to think about what the uses of the collection would be. Primarily, this bibliographic collection is intended to allow researchers to trace uses of data for a variety of purposes. It is also meant to enable researchers to replicate others' findings and to demonstrate to funding agencies the varied uses of data. With these considerations in mind, we decided to capture citations to publications that met at least one of the following criteria:

The resource used data in the ICPSR holdings as the primary data source.
The resource used ICPSR data in a comparison with the primary dataset investigated.
The resource was "about" an ICPSR dataset or study series. That is, the publication described the data, data collection procedures, or changes to a study series; compared it to other studies; or critiqued the data, survey questions, or collection process.

Since some datasets are available from multiple archives or data providers and we could not always verify that the source of the data used was ICPSR, we made the determination that ICPSR need not be the actual source.

References to all types of literature and media are being collected. Thus far, the collection includes journal articles, books, book chapters, government and agency reports, working papers, dissertations, conference papers, meeting presentations, unpublished manuscripts, audiovisual materials, and electronic resources.

Selecting Search Resources

Our primary resource for searching has been the University of Michigan (UM) Libraries Networked Electronic Resources, now known as Search Tools. This online collection is available to UM affiliates and includes a comprehensive selection of full-text databases and index and abstract services in the social sciences and other disciplines. We have made use of the UM Libraries' excellent collection of print publications and its inter-library loan service to browse materials not available online. We have also conducted Internet searches for various studies and publications.

Collecting and Organizing Citations

In order to collect and organize citations, we decided to purchase database software that was specifically designed for collecting bibliographic citations. Our primary selection criteria included:

Multiple, simultaneous user capability, either via the Internet or an internal network
Easy installation and start-up
Effective sorting and searching mechanisms
Ability to display citations in a Web-based format
User-friendly interface
Affordable price

After comparing several options, we selected Biblioscape, which was the only software that allowed multiple simultaneous users and met the other requisites.

Search Strategies

ICPSR was established in 1962 and presently archives thousands of studies in the social and behavioral sciences, covering a broad array of topics including: elections, education, mental health, criminal justice, aging, gender, race, religion, medical care, war, family, and geography. Given the decades-long time period and the large number of datasets and topics involved, the task of locating related publications was monumental, and we realized at the outset that we could never locate every single publication based on ICPSR data. Yet we wanted to cast a wide enough net to capture as many relevant citations as possible during the allotted three-year grant period. This project involved developing effective search strategies to strike a balance between comprehensiveness and precision. It also meant setting priorities in selecting search resources and formulating search queries.

We decided to use a two-pronged approach, whereby we would search online resources for specific study titles and also identify a list of core journals to browse, either in print or electronic format, for references to any ICPSR dataset. Using the resources available through UM Libraries, we began searching online databases and putting together a list of core journals in subject areas relevant to ICPSR data. Below is a summary of the searches that have been conducted thus far and lists of the electronic and print resources searched and browsed.

Summary of Searches Conducted

Initial searches:
In order to familiarize ourselves with the content and search capabilities of the available online citation, abstract, and full-text resources, we began searching for the following terms/phrases:

icpsr
icpr
interuniversity consortium for political
inter-university consortium for political

The following databases were searched (the number of hits is in parentheses):

Alt-Health Watch (3)
Cambridge University Press Online Journals (26)
Columbia International Affairs Online (11)
Congressional Universe (13)
Contemporary Women's Issues (3)
Ethnic Newswatch (1)
Full-Text (13)
Gender Watch (7)
Infotrac Gold: Health Reference Center (27)
Infotrac Gold: General Reference Center (211)
Highwire Press Journals (15)
JSTOR (1,456)
Legaltrac (401)
New York Times (0)
Oxford University Press Online Journals (3)
Proquest Research Library (1,044)
Proquest ABI Inform (214)
Proquest Newspapers (4)
Science Direct (32)
Springer-LINK (5)
Web SPIRS (all available databases) (123)

Most of these searches yielded relevant hits, and we thus began the process of building the collection. Using the results of these searches, as well as recommendations from ICPSR staff, we began to identify the core journals that publish articles based on ICPSR data.

Study Titles:
We searched for the titles, partial titles, and variant titles of studies in the ICPSR archive. The databases searched are described below. We are using spreadsheets to record the details of each title search, including study titles, specific databases searched, dates searched, search queries, and number of hits. To date, we have searched for thousands of titles.

P.I. names:
Authors do not always include the titles of datasets used in published works, or they may use a variation of the ICPSR title. We made every attempt to formulate queries that would account for variant versions of study titles. However, we were not always successful due to limitations of database search mechanisms, spelling errors, and the mere absence of a reference to the name of the dataset used in a published work. Therefore, in cases where no hits were generated when searching for variations of a study title, the P.I. names were searched for in the relevant databases. This occasionally yielded positive results.

Online Databases Searched

The following online databases are being searched for a variety of study titles. Databases were selected based on the relevancy of their contents to the subject and time period of the study being searched. Therefore, every database is not searched for each study. Databases are listed by order of the number of citations found.

Journals Browsed

We identified a list of core journals in the fields of political and social science, health care, and criminal justice, based on their known relevancy. We also analyzed the references we found through our online searches in order to identify those that contained the most articles based on ICPSR data. We began browsing through articles in the print and online versions of these core journals, looking for references to ICPSR data. From 1974-1984, ICPSR staff had browsed through a number of core journals in the political and social sciences to capture citations to ICPSR data-related articles. For our search, we are omitting those journals in that time period since those citations have already been included in the Bibliography. We are making extensive use of full-text databases for browsing electronic versions of these journals. In particular, we've used JSTOR and Proquest, also ScienceDirect and Periodicals Contents Index (PCI) Full Text. Otherwise, we are browsing the print versions of these journals.

Other Citation Sources

Citations were gathered from other sources as well. When a dataset is deposited with ICPSR, the Principal Investigator is asked to submit citations of publications based on the data. Since 1962, several thousand citations have been submitted in this way. Similarly, researchers requesting ICPSR data are asked to submit copies of work they publish based on that data. Government agencies usually submit a list of reports and other publications related to the data they deposit. These references were previously included in the "related publications" field of study descriptions and have been added to the Bibliography database.

In addition, staff of many of the large ongoing data collections in the social sciences maintain their own bibliographies of works related to their data. We conducted Web searches for major study titles using Google and found a number of these bibliographies, most of which have been included in the Bibliography. The Panel Study of Income Dynamics, Asset and Health Dynamics Among the Oldest-Old (AHEAD), and the Health and Retirement Study (HRS) contributed their citations in a tagged format for inclusion in the ICPSR Bibliography. Smaller online bibliographies were added manually, such as the World Values Survey bibliography.

We also stumbled upon many citations serendipitously. While browsing articles for references to ICPSR data, we often came upon references to other publications that used ICPSR data. In the process of conducting searches or browsing journals, we occasionally discovered bibliographies in books, journals, codebooks, and other publications. Some bibliographies were brought to our attention by researchers and librarians. All of these citations have been included in the database.

Online Access

We originally collected citations in Biblioscape, a bibliographic database application. In order to link them to the ICPSR website, the references were exported from Biblioscape in a delimited format, then reorganized using perl scripts and imported into Oracle tables, from which they are displayed on the ICPSR site. We now enter and manage all the citations in Oracle, using a web-based data entry, management, and reporting tool developed at ICPSR.

We created search interfaces and displays to enable users to locate citations by linking through dataset descriptions or by conducting searches for specific publications. Enhanced search, display, and navigation tools are currently being developed.

Full-text Linking

ICPSR recently changed the way in which we provide access to full-text articles via Open URL. We no longer ask members to submit to us a link to their institution's full-text linking service page. We now use a linking system available through OCLC's WorldCat. We have made this change because most member institutions have established referencing links with WorldCat, so they will not have to take the extra step of updating their Open URL information with ICPSR anymore.

Beneath each citation in a given search results list in the Bibliography, users see "Full Text Options" followed by one or more linked words. For journal articles, there is a Worldcat link, as well as a Google Scholar link. The DOI (Digital Object Identifier) will also appear when it is available.

When users click on the DOI, they will be taken directly to the full-text, without going through OpenURL linking, so a user's member institution's branding will not necessarily appear. When users click on the WorldCat link, they should see the same institutional branding and logos as provided through their institution's links with WorldCat. If the logo for full-text linking is not appearing, ORs should check with their library systems folks to ensure that OCLC has their institution's updated referencing information to their Open URL software and icon.

As in the past, for freely available reports, presentations, and other resources, the URL or PDF URL will appear if it was known to ICPSR when the citation was entered in the Bibliography.

CLOSE

Exporting Citations

Exporting individual citations is now possible. Underneath each ICPSR citation, there is an "Export citation" link. Clicking the link lets a user export directly into reference management software like EndNote or online tools like Zotero (a free extension to the Internet browser, Firefox).

The Bibliography's citations are exported in RIS file format, a tagged format for expressing bibliographic citations. It is compatible with most reference management software. For advice about exporting ICPSR's RIS-formatted citations, users can click the "information" button next to the citations export link. Users should check their reference software for specific instructions on importing citations in RIS format.

CLOSE

Conclusion

We have only begun to understand the uses of this collection and anticipate that researchers will discover further applications as it continues to grow. We look to developing standards for citing data as a means to facilitate the process of identifying works based on social science data. Our hope is that the existence of the Bibliography itself will inspire researchers to notify us when they use data acquired from ICPSR. These factors, combined with ongoing searches of online and print resources, will continue to expand this valuable social science research tool.

CLOSE

Appendix: Project Contributors

Creating this bibliography and making it accessible is a collaborative task involving many people who have contributed in a variety of ways. The following people in particular have assisted in this task:

Elizabeth Moss, Librarian, Project Manager
Jeri Schneider, Librarian, Originating Project Manager
Mary Vardigan, former Director, ICPSR Collection Delivery Department
Jared Lyle, Director, Metadata & Preservation Unit
ICPSR Computer Network Support staff, programmers, and webmasters: Nathan Adams, Bryan Beecher, Jon Brode, Wendi Fornoff, Laurie Howland, Peter Joftis, Vicki Neff, Matthew Richardson
Other current and former ICPSR staff: Erik Austin, Elizabeth Bonner, Linda Brown, Pamela Brown, Christin Cave, Corey Colyer, Joyce French, Amy Fuhrman, Alison Goldberg, John Gray, Chris Greene, Bree Gunter, Sarah Harrison, Hank Heitowit, Sue Hodge, Sanda Ionescu, Leah Kasper, Shova KC, Sally Kiser, Kaye Marz, Emily Merchant, Jim McNally, Bonnie Murray, Shelly Petrinko, Ruth Shamraj, Shannon Stagman, Janet Stamatel, Karen Sullivan, Ken Sylvester, Adam Williams
University of Michigan Graduate Library staff: Scott Dennis, Joanne Dionne, Rebecca Dunkle, Darlene Nichols
University of Michigan Population Studies Center Library staff
National Opinion Research Center (NORC): Tom Smith, Jennifer Berktold
Camille Ward, University of Michigan Institute for Social Research, Panel Study of Income Dynamics
Heather Hewett, University of Michigan Institute for Social Research, Health and Retirement Study/Asset and Health Dynamics Among the Oldest Old
Dan Tsang, Librarian, University of California

CLOSE

ICPSR Bibliography of Data-related Literature: Originating Methodology

Collection Policy

Selecting Search Resources

Collecting and Organizing Citations

Search Strategies

Summary of Searches Conducted

Online Databases Searched

Journals Browsed

Other Citation Sources

Online Access

Full-text Linking