Survey Design
Survey Quality Measures
Trend Data
Availability of Data
The National Survey of Recent College Graduates (NSRCG) provides information about individuals who recently obtained bachelor's or master's degrees in a science, engineering, or health (SEH) field. This group is of special interest to many decision makers, because it represents individuals who have recently made the transition from postsecondary school to the workplace. It also provides information about individuals attending graduate school. The results of this survey are vital for educational planners in the federal government and in academia. Employers in all sectors (education, industry, and government) use these survey results to understand and predict trends in employment opportunities and salaries in SEH fields for recent graduates and to evaluate the effectiveness of equal opportunity efforts. This survey is a component of the Scientists and Engineers Statistical Data System (SESTAT), which provides data on the total number and characteristics of individuals with education or employment in SEH fields in the United States.
Respondents are individuals who recently received bachelor's or master's degrees in an SEH field from a U.S. institution, were not institutionalized, and were living in the United States during the survey reference week of 1 October 2010, and were under age 76.
The target population of the 2010 survey consisted of all individuals with the following characteristics:
The NSRCG sample is a two-stage sample: at the first stage a sample of institutions is selected and at the second stage a sample of graduates is selected from lists provided by the sampled institutions. The sample frame of schools for inclusion in the first stage is obtained from the Integrated Postsecondary Education Data System (IPEDS) database maintained by the National Center for Education Statistics.
For the 2010 NSRCG, the first stage institution sample frame consisted of 2,168 U.S. postsecondary institutions that conferred at least one bachelor's or master's degree in an SEH field during the 2008 and 2009 academic years (AYs). For the selection of graduates in the second stage, the sample frame was constructed from lists of graduates obtained from representatives of the institutions selected at the first stage. The second-stage sample frame of graduates consisted of 2,080,732 records. Of these records, only 866,012 graduates were eligible for the second-stage selection of graduates.
The first stage of the sample was selected with probability proportional to size (PPS). A composite size measure was related to the number of eligible graduates, controlling for sample size domains defined by degree level, field of major, race, ethnicity, and sex. Institutions that produce relatively large numbers of bachelor's or master's degrees were selected with certainty. Institutions selected proportionately to a measure of size reflected the maximum percentage of graduates in each of the degree fields within level-of-degree categories. The measure of size was adjusted to increase the probability of selection of institutions with relatively high percentages of graduates in targeted minority groups.
To maintain the efficiency of the institution sample, 301 institutions selected for the NSRCG in 2008 were retained for the 2010 sample. To reflect the population of schools newly eligible for the NSRCG, a supplemental sample of one institution was added to the existing sample. Consequently, the 2010 NSRCG school sample consisted of 302 institutions.
The second-stage sampling consisted of selecting 18,000 bachelor's or master's degree recipients (9,000 for each AY) who received science, engineering, or health degrees from the institutions selected in the first stage. Composite size measures were used to incorporate differential sampling rates for domains subject to over- or under-sampling [1] to satisfy various analytical interests, including minority representation in SEH fields. To formulate the composite size measure for institutions, 222 domains were identified and used, which consisted of combinations from the following four variables:
Institution-level sample sizes of the graduates were calculated separately for certainty and non-certainty institutions to achieve equal weights within key NSRCG domains across institutions. A proportional allocation of the total sample to 85 certainty institutions resulted in institution-level sample sizes from 40 to 240 for those institutions. The noncertainty institutions were implicitly stratified by sorting the list by type of control (public, private), geographic region (northeast, northwest, southeast, southwest), and the percentage of degrees awarded in SEH fields of study. An equal sample allocation was constructed for each of 205 responding non-certainty institutions. However, not all non-certainty institutions had enough numbers of students, thus the sample allocation resulted in institution-level sample sizes from 7 to 153 for those institutions. The 290 participating institutions provided the total sample of 18,000 graduates (9,874 bachelor's and 8,126 master's recipients).
In 2010, Mathematica Policy Research, Inc., under contract to the National Center for Science and Engineering Statistics, conducted the first-stage data with the sampled institutions and the second-stage survey data collection with the sampled individuals. The first-stage data collection began with contacting the 302 sampled institutions to obtain their lists of graduates for AY 2008 and AY 2009. Of the 302 sampled institutions, 290 provided a list of graduates.
The second-stage survey data collection for sampled individuals used three data collection modes — paper, Web, and computer-assisted telephone interviewing (CATI). Paper and Web were the primary modes in the initial stage of data collection, followed by CATI. The 2010 CATI and Web survey instruments were modeled after, and were very similar to the 2008 paper questionnaire used in the mail phase. The NSRCG questionnaire instruments were designed to be as similar as possible to the survey instruments used in the 2010 National Survey of College Graduates (NSCG) and the 2010 Survey of Doctorate Recipients (SDR) to facilitate combining results into estimates of the total SEH population. A few questions in the NSRCG, however, obtain information of special interest for the population of recent graduates. For example, the NSRCG had more information related to education history than did the NSCG or SDR.
Information in the 2010 survey was collected for the reference week of 1 October 2010. Data collection took place between February 2012 and September 2012.
Weights are attached to each responding graduate record to estimate characteristics of the population of graduates. The weights were created in the following stages:
In addition to creating estimation weights for each graduate, a hot deck imputation procedure was used to estimate missing item values, using responses from other graduates who had similar characteristics (age, major, sex, etc.).
The sample size is sufficiently large that estimates based on the total sample should be subject to minimal sampling error. However, sampling error can be quite substantial in estimating the characteristics of small subgroups of the population. Estimates of the sampling errors associated with various measures are included in the methodology report for the survey and in the basic publications.
A major source of coverage error is when institutions don't identify a graduate as having received a degree of interest. This failure can arise when institutional records are incorrect (e.g., when incorrect dates for degree receipts are recorded or incorrect degree fields are recorded). It also can arise because of the difficulty in correctly classifying the degree fields according to the taxonomy that NSF uses to identify whether the degree field is in-scope. To minimize the impact of this latter problem, graduates with ambiguous degree fields are included in the sample and eliminated if their survey responses indicate they are out-of-scope.
(1) Unit nonresponse. The unweighted response rate for the first stage (institution-level response rate) was 96.0%; the weighted response rate was 95.7%. The unweighted response rate for the second stage (graduates response rate) was 73.1%; the weighted response rate was 72.6%.
(2) Item nonresponse. In 2010, the item nonresponse rate for key items (employment status, type of employment, occupation, and primary work activity) ranged from 0.0 to 1.61%. Other variables, especially those involving sensitive information, had higher nonresponse rates. For example, the reasons for working outside the field of the sampled graduate's highest degree had item nonresponse rates of approximately 9.21% to 13.24%. A hot deck imputation procedure was used to impute all missing data items, except for the critical complete items and verbatim text items. Any cases missing the critical complete items were considered as survey nonresponse. The critical complete items in the 2010 NSRCG are: living in the United States on 1 October 2010, working for pay or profit, looking for work, last job code, sector of employment, employer educational institution, type of educational institution, principal job code, and state date of principal job.
To examine the potential nonresponse bias in the 2003 NSRCG data, a nonresponse analysis study was conducted and the results showed that any detectable differences were properly addressed by the nonresponse weighting adjustments of the survey data [2].
Several of the variables in this survey are difficult to measure and thus are prone to measurement error.
As is true for any multi-modal survey, it is likely that the measurement errors associated with the different modalities are somewhat different. This possible source of measurement error is especially troublesome, because the proclivity to respond by one mode or the other is likely to be associated with variables of interest in the survey. To the extent that certain types of individuals may be relatively likely to respond by one mode compared with another, the multi-modal approach may have introduced some systematic biases into the data. In an effort to reduce measurement error, the instrument was pretested, using cognitive interviewing.
There have been a number of changes in the definition of the population surveyed over time. For example, the surveys conducted in the 1980s included individuals receiving bachelor's degrees in the field of engineering technology; however since 1993, individuals in this field have been excluded. Beginning 2003, the NSRCG sample was expanded to cover the population of bachelor's and master's degree graduates in health fields and that population has been kept as it since then. Given these changes and other survey improvements, caution should be used when examining trend analyses involving the 1993 NSRCG and NSRCG data prior to 1993; 2003 NSRCG and NSRCG data prior to 2003; and 2010 NSRCG compared to 2008 NSRCG given the 2010 raking methodology change.
The data from this survey are published biennially in Detailed Statistical Tables in the series Characteristics of Recent Science and Engineering Graduates, as well as in InfoBriefs and Special Reports.
Information from this survey is also included in Science and Engineering Indicators and Women, Minorities, and Persons With Disabilities in Science and Engineering.
Data from this survey are available on the NCSES website and on the SESTAT website. Selected aggregate data are available in public use data files upon request. Access to restricted data for researchers interested in analyzing microdata can be arranged through a licensing agreement.
Additional information about this survey can be obtained by contacting:
Flora Lan
Project Officer
Human Resources Statistics Program
National Center for Science and Engineering Statistics
National Science Foundation
2415 Eisenhower Avenue, Suite W14200
Alexandria, VA 22314
Phone: (703) 292-4758
E-mail: flan@nsf.gov
Footnotes
[1] See Folsom RE, Potter FJ, Williams SR. 1987. Notes on a Composite Measure for Self-Weighting Samples in Multiple Domains. Proceedings of the Section on Survey Research Methods, American Statistical Association: 792–96.
[2] See Dajani A, Maples J. 2005. The 2003 NSF/RCG Nonresponse Bias Analysis. Washington, DC: U.S. Department of Commerce, Bureau of the Census.