When Every Dollar Counts: Comparing Reported Earnings of Social Security Disability Program Beneficiaries in Survey and Administrative Records

by
Social Security Bulletin, Vol. 78 No. 4, 2018

This article examines differences between survey- and administrative data–based estimates of employment and earnings for a sample of Social Security Disability Insurance and Supplemental Security Income beneficiaries. We use linked records from the Social Security Administration's National Beneficiary Survey and administrative earnings records from the agency's Master Earnings File. We find that estimated employment rates and earnings levels are consistently higher in administrative data than in survey data. The differences between survey- and administrative data–based estimates of employment rates and earnings are larger in absolute and proportional terms for beneficiary sociodemographic subgroups whose survey-reported employment rates are lower than those of beneficiaries overall. Nonetheless, we estimate beneficiary employment rates of less than 20 percent from both survey and administrative data, suggesting that both sources provide plausible estimates for the overall beneficiary population.


David Wittenburg is a senior researcher and director of Health Research at Mathematica Policy Research. Jeffrey Hemmeter is director of the Office of Research and Demonstration, Office of Research, Demonstration, and Employment Support, Office of Retirement and Disability Policy, Social Security Administration. Holly Matulewicz is a senior survey researcher, Lindsay Glassman is a survey specialist, and Lisa Schwartz is a vice president at Mathematica.

Acknowledgments: The authors thank Gina Livermore, Elaine Gilby, Jim Sears, and Emily Roessel for their very helpful comments on earlier drafts.

Note: This study was made possible by the Employment Policy and Measurement Rehabilitation Research and Training Center, which is funded by the Department of Education's National Institute on Disability Independent Living and Rehabilitation Research, under a cooperative agreement (no. H133B100030) with the University of New Hampshire. The findings and conclusions presented in the Bulletin are those of the authors and do not necessarily represent the views of the Social Security Administration.

Introduction

Selected Abbreviations
DI Disability Insurance
MEF Master Earnings File
NBS National Beneficiary Survey
SSA Social Security Administration
SSI Supplemental Security Income

The Social Security Administration (SSA) aims to make the best use of administrative and survey data for research and program operations, particularly in measuring employment and earnings. Both sources offer advantages in monitoring program operations, capturing beneficiary characteristics, and measuring the effects of demonstration projects. However, information on how the sources might produce differing estimates is limited. A better understanding of these differences might be critical to identifying important evaluation outcome measures and/or designing interventions to customize supports for SSA disability program beneficiaries.

This article compares employment and earnings outcomes for disability program beneficiaries based on linked data from SSA's National Beneficiary Survey (NBS) and administrative records from the agency's Master Earnings File (MEF). NBS respondents are a nationally representative sample of people who received benefits from Social Security Disability Insurance (DI), Supplemental Security Income (SSI), or both. We use NBS data to construct annual measures of employment and earnings for 2003 through 2005 that we then compare with linked annual employment and earnings reports in the MEF (which are not accessible to the general public). We present detailed comparisons of survey and administrative data on employment and earnings for DI beneficiaries, comprising those who receive DI benefits only and those who receive DI and SSI benefits concurrently. We also summarize similar comparisons for individuals who receive SSI payments but no DI benefits, and present detailed tables for that population in Appendix A.

We find that employment rates and earnings levels are higher in administrative records than in survey reports for SSA disability program beneficiaries overall and for all major sociodemographic subgroups. The proportional differences between the administrative and survey records can be substantial because the employment rates of DI and SSI beneficiaries are low. For example, employment rates for DI beneficiaries in our sample were about 40 percent higher in administrative records than in survey data, although the absolute difference is only 5.6 percentage points (19.2 percent versus 13.6 percent). Among subgroups, we find that the largest relative differences between administrative and survey data are for beneficiaries with survey-reported employment rates and earnings levels that are lower than the average survey-reported employment rate and earnings level of DI beneficiaries overall. For example, the survey and administrative data differ significantly for beneficiaries with a musculoskeletal primary disabling condition,1 and that subgroup's employment rate and earnings are considerably lower than those of DI beneficiaries overall. Absolute differences between two relatively low employment rates or earnings levels would thus be proportionally greater than similar absolute differences among subgroups with higher employment rates or earnings.

Background

Measuring employment outcomes poses challenges for both survey and administrative data collection. Several studies suggest using both survey and administrative data to identify potential underreporting associated with one source or the other (Abowd and Stinson 2011; Barnow and Greenberg 2014; Ford and others 2014). Using both sources can be especially advantageous for data on subpopulations that may be underrepresented or prone to reporting error in one of the sources. As Davies and Fisher (2009) noted, researchers have used matched survey and administrative data to assess the accuracy of the survey data and used the resulting information to adjust for error in the survey-based estimates.

Several studies have shown that administrative data produce higher estimated employment rates and earnings levels than survey data (Coder and Scoon-Rogers 1996; Pedace and Bates 2000; Gottschalk and Huynh 2005). Most authors have speculated that administrative earnings records are higher because survey respondents tend to underreport earnings from seasonal or temporary jobs (Moore, Marquis, and Bogen 1996; Kornfeld and Bloom 1999). For example, survey respondents might be prone not to recall earnings from each of multiple part-time or seasonal jobs, whereas those earnings would be recorded in the administrative data whenever they were provided by employers. Incomplete recall by respondents might also explain why administrative estimates of earnings tend to be higher than survey-based estimates for short-term, marginal, or overlapping jobs (Bridges, Del Bene, and Leonesio 2003; Hurd, Juster, and Smith 2004; Abraham and others 2009) as well as for highly variable or unpredictable sources of income. Alwin, Zeiser, and Gensimore (2013) found that administrative data–based estimates for irregular earnings, such as those from short-term employment, were higher than estimates from surveys. Bound and others (1994) also speculated that administrative data–based earnings estimates are higher than survey estimates because respondents might have trouble accurately recalling earnings for hourly work, which are automatically recorded in the administrative data.

In general, proportional differences between administrative and survey data on earnings tend to be larger among subgroups with lower earnings—again, with estimated earnings based on administrative data tending to be higher than corresponding estimates from the survey data. The lower estimates in survey data could reflect recall error for work at sporadic or multiple jobs (Rodgers, Brown, and Duncan 1993; Pischke 1995; Gottschalk and Huynh 2005). Recall error aside, people who work more sporadically tend to have lower earnings than other workers, and a $50 difference between two earnings estimates for a low earner is proportionally greater than a $50 difference between two earnings estimates for a higher earner.

Information on differences in administrative and survey records by demographic or disability characteristics is scarce. A key challenge in measuring these differences is that the sample sizes of many of the subgroups in national surveys that are linked to administrative data are small (Kornfeld and Bloom 1999; Monti and Gathright 2013). For example, many of the studies mentioned above draw on data from national surveys such as the Current Population Survey and the Survey of Income and Program Participation that have been linked to administrative records. Although those surveys have large samples overall, the samples of specific subgroups—such as DI and SSI beneficiaries—are limited.

Program Descriptions, Data Sources, and Methodology

To qualify for DI or SSI benefits, an applicant must demonstrate an inability to engage in substantial gainful activity (SGA) because of a medically determinable impairment that is expected to last at least 12 months or to result in death. SGA is determined by a monthly earnings level; each year, SSA adjusts the SGA definition, if needed, based on changes in the national average wage. In 2018, SSA defines SGA as monthly earnings of $1,180 or more for nonblind applicants and $1,970 or more for blind applicants.

DI and SSI eligibility rules differ in ways that might influence employment outcomes. DI eligibility is contingent on having sufficient levels of recent and lifetime Social Security–covered employment. By contrast, SSI is a means-tested program, with eligibility subject to strict income and asset limits. Individuals may qualify for both programs if their income (including DI benefits) and assets are low enough to meet the SSI limits. Perhaps not surprisingly, DI beneficiaries tend to be older and have more extensive work histories than SSI recipients (Mamun and others 2010). Finally, there are important differences in DI and SSI benefit reductions resulting from work earnings. DI beneficiaries face a “cash cliff” (that is, benefit payments stop altogether) for earnings above a certain threshold, whereas SSI payments decline incrementally, generally being reduced by $1 for every additional $2 of earnings.

Previous studies have examined the employment and earnings outcomes of DI beneficiaries and SSI recipients using survey and administrative data separately. Livermore (2009) documented the work activities of all DI beneficiaries and SSI recipients using 2004 survey data and found that 13 percent reported working during the previous year and 9 percent were working as of the date of the interview. A much higher proportion had aspirations of work: 40 percent of working-age disability program beneficiaries reported having work goals or expectations. Mamun and others (2010) used administrative earnings records to find that 12 percent of disability program beneficiaries had at least $1,000 in earnings in 2007.

Our analysis extends the literature by comparing the survey data on employment and earnings from the NBS to similarly constructed measures in the administrative data. The NBS, which is sponsored by SSA and was developed and implemented as part of the agency's Ticket to Work (TTW) program evaluation, collects cross-sectional data from a national sample of DI beneficiaries and SSI recipients and a sample of TTW program participants. Its primary purpose is to provide information on work-related activities of SSI and DI beneficiaries, particularly as they relate to TTW implementation. The survey collects information on respondent employment status, employment services used in the past year, disability status, experience with SSA programs, health and functional status, health insurance, earnings, income support, and sociodemographic characteristics (Livermore and others 2011). Proxy respondents are permitted for individuals whose disabilities make the interview prohibitively challenging and for people who cannot be contacted and interviewed directly.

We pooled data from three rounds of the NBS—fielded in 2004, 2005, and 2006—to obtain a larger sample of beneficiaries with earnings than a single round would provide. Our total pooled sample includes 7,987 observations for individuals aged 21–64 at the time of the NBS interview. We generated statistics using survey weights (adjusted for pooling); all standard errors used to compute tests of statistical significance account appropriately for the complex NBS sampling design.

We used the MEF to generate administrative estimates of employment and earnings. The MEF contains annual earnings data from Internal Revenue Service Form W-2, quarterly earnings records, and annual income tax forms (Olsen and Hudson 2009). Annual earnings are defined as the maximum of (1) Social Security–taxable wages and self-employment earnings (capped at $127,000 in 2017), (2) Medicare-taxable wages and self-employment earnings, or (3) total compensation (which captures earnings taxable by neither Social Security nor Medicare). Earnings not reported on a W-2 are not included in the underlying data and are thus not included in the analysis. Additionally, earnings from known “third-party payers”—that is, insurance, pensions, and similar nonwork earnings—are subtracted from our earnings measure.

We linked MEF data to survey results for each NBS respondent. Survey questions covered the year prior to the interview; for example, the 2004 NBS covered employment and earnings for 2003. Therefore, we merged 2003 MEF earnings data with the 2004 NBS results, and so forth, to produce comparable survey- and administrative data–based estimates.

We developed an annual frame for definitions of employment and earnings to facilitate comparisons between the survey and administrative data. The choice of an annual frame was necessary to allow the survey data to be consistent with the annual reporting in the administrative data described above. The NBS includes questions about earnings from each respondent's work that lasted for at least 30 days in the prior year. Hence, we defined annual employment as the presence of any earnings from a job the NBS respondent reported working that year. The sum of earnings for each reported job establishes the annual earnings measure.2 The earnings values in both the survey and administrative data are reported in nominal dollars (that is, they are not adjusted for inflation).

The earnings estimate we derive from the administrative data is likely to be higher than that from the survey data for two reasons. First, the survey questions potentially impose a burden on respondents to recall detailed information about all jobs in which they received earnings during the past year, which might be more difficult for those who held multiple jobs. Respondents who were frequently paid on a piece-rate basis rather than by the hour or with an annual salary might face similar recall challenges. Second, given the potential benefit reductions for SSI and DI beneficiaries with work earnings, NBS respondents may choose to underreport job and earnings information out of concern that full disclosure in a survey sponsored by SSA could jeopardize their disability program benefits.

Table 1 provides descriptive statistics for our sample of DI beneficiaries. These statistics provide context for the types of jobs held, the resultant earnings, and the nature of employment overall for this population. The majority of individuals in the sample (74.8 percent) received only DI benefits; the other 25.2 percent received SSI payments as well. Of the entire sample, 20.7 percent had a musculoskeletal primary condition, 18.3 percent had a psychiatric condition, 5.7 percent had an intellectual condition, 3.4 percent had a sensory condition, and 47.5 percent had conditions of other types. Most beneficiaries were middle-aged or older: 84.7 percent were older than 39 and 59.5 percent were older than 49. The division by sex was almost equal, with slightly more women than men. Nearly three-fourths of the sample was white only, 18.2 percent was black or African American only, and the remainder represented other racial groups. The majority of survey respondents self-reported data; less than 20 percent of the interviewees were proxies.

Table 1. Descriptive statistics for the DI beneficiary study sample, 2003–2005
Characteristic Number (weighted) a Percentage distribution Standard error b
Total 6,233,868 100.0 . . .
Benefit type
DI only 4,661,112 74.8 0.9
Concurrent DI/SSI 1,572,756 25.2 0.9
Primary disabling condition
Psychiatric 1,140,178 18.3 0.7
Intellectual 355,518 5.7 0.4
Musculoskeletal 1,290,348 20.7 0.7
Sensory 209,490 3.4 0.3
Other 2,959,033 47.5 0.8
Missing c 279,301 4.5 0.4
Sex
Men 2,968,716 47.6 0.9
Women 3,265,152 52.4 0.9
Age
21–29 261,801 4.2 0.2
30–39 692,642 11.1 0.3
40–49 1,571,112 25.2 0.5
50–59 2,241,472 36.0 1.0
60–64 1,466,841 23.5 1.0
Race
White only 4,654,797 74.7 2.0
Black or African American only 1,135,446 18.2 1.9
Other 443,625 7.1 0.8
Ethnicity
Non-Hispanic 5,697,871 91.4 1.5
Hispanic 535,997 8.6 1.5
Type of survey response
Self-report 5,084,438 81.6 0.7
Proxy report 1,149,430 18.4 0.7
SOURCE: Authors' calculations using linked MEF and NBS data.
NOTE: . . . = not applicable.
a. The weights for the survey data have been adjusted to reflect the three NBS rounds (2004, 2005, and 2006) combined into a single file. The unweighted total sample size is 7,987.
b. Estimated using the complex survey weights provided in the data, which control for the clustering and stratification of the survey.
c. Not reported in the matched MEF record.

In Tables 2–5, we examine differences in employment and earnings statistics between administrative and survey data. In each table, we find that administrative data produce higher estimates than survey data. We measure absolute differences in percentage-point or dollar terms by subtracting the survey estimate from the administrative estimate. We measure proportional differences in percentage terms by dividing the absolute difference by the survey estimate. The absolute differences are relatively uniform across beneficiary subgroups while the proportional differences offer the context of relative magnitude, which is relevant to understanding the extent of the disparity between the survey estimate and the administrative data. The proportional difference is therefore particularly useful for researchers interested in understanding the potential undercount of earnings if survey records are the only available source of earnings information.

Findings

Table 2 presents employment-rate estimates for DI beneficiaries based on administrative and survey data, overall and by subgroup. For the full sample, we find a higher employment rate in the administrative data (19.2 percent) than in the survey data (13.6 percent). Given the relatively low employment rate of this population, the absolute difference of 5.6 percentage points means that the administrative estimates are 41.3 percent higher than the survey estimates.

Table 2. Estimated mean annual employment rates of DI beneficiaries: Differences between administrative (MEF) and survey (NBS) data, 2003–2005
Characteristic Number (weighted) a Estimate based on— Absolute difference (MEF minus NBS) Proportional difference b (%)
MEF NBS
Employment rate (%) Standard error Employment rate (%) Standard error Percentage points Standard error p-value
Total 6,233,868 19.2 0.9 13.6 0.7 5.6 0.5 0.0 41.3
Benefit type
DI only 4,661,112 19.1 1.0 12.8 0.8 6.2 0.6 0.0 48.4
Concurrent DI/SSI 1,572,756 19.5 1.2 15.7 1.0 3.8 0.8 0.0 24.0
Primary disabling condition
Psychiatric 1,140,178 23.3 1.6 19.7 1.5 3.6 1.1 0.0 18.2
Intellectual 355,518 35.0 3.0 29.3 2.8 5.7 2.6 0.0 19.5
Musculoskeletal 1,290,348 14.4 1.4 8.0 1.0 6.4 1.1 0.0 79.8
Sensory 209,490 28.0 3.7 22.7 3.4 5.3 4.3 0.2 23.3
Other 2,959,033 16.0 1.0 9.6 0.8 6.4 0.7 0.0 66.4
Missing c 279,301 31.2 2.9 29.3 2.9 2.0 1.7 0.3 6.7
Sex
Men 2,968,716 19.4 1.0 12.3 0.8 7.1 0.7 0.0 57.6
Women 3,265,152 18.9 1.1 14.7 0.9 4.2 0.6 0.0 28.9
Age
21–29 261,801 39.3 1.9 34.5 2.2 4.8 1.3 0.0 13.8
30–39 692,642 30.5 1.3 25.2 1.2 5.3 0.8 0.0 21.1
40–49 1,571,112 21.5 1.1 16.9 1.0 4.7 0.7 0.0 27.6
50–59 2,241,472 14.9 1.3 8.8 1.0 6.2 1.0 0.0 70.5
60–64 1,466,841 14.2 1.5 8.2 1.2 6.0 1.1 0.0 73.8
Race
White only 4,654,797 19.7 1.0 13.9 0.9 5.8 0.5 0.0 41.3
Black or African American only 1,135,446 19.3 1.5 12.6 1.2 6.7 1.2 0.0 53.7
Other 443,625 13.3 1.5 12.2 1.2 1.1 1.4 0.4 8.8
Ethnicity
Non-Hispanic 5,697,871 19.5 1.0 13.9 0.8 5.6 0.5 0.0 40.1
Hispanic 535,997 15.4 1.3 9.6 1.2 5.8 1.2 0.0 60.0
Type of survey response
Self-report 5,084,438 18.4 0.9 12.6 0.7 5.8 0.5 0.0 46.1
Proxy report 1,149,430 22.4 1.8 17.7 1.5 4.7 1.1 0.0 26.2
SOURCE: Authors' calculations using linked MEF and NBS data.
NOTE: MEF estimates account for individuals with any earnings reported in the year. NBS estimates account for respondents who reported working at least one job held for 30 days or more in the year.
a. The weights for the survey data have been adjusted to reflect the three NBS rounds (2004, 2005, and 2006) combined into a single file. The unweighted total sample size is 7,987.
b. Calculated using unrounded employment-rate estimates.
c. Not reported in the matched MEF record.

The absolute differences between administrative data– and survey-based employment rates are relatively consistent among subgroups. With two minor exceptions (the “missing data” disabling condition and “other” race), administrative estimates of employment are from 3.6 to 7.1 percentage points higher across all subgroups. However, the proportional differences vary substantially because the employment rates themselves can vary by subgroup. The largest proportional differences (70 percent or more) are seen for subgroups that have some of the lowest employment rates (beneficiaries with musculoskeletal conditions and those aged 50 or older). Notably, this pattern would hold if we defined “proportional difference” as the absolute difference divided by the administrative estimate. In any event, although the absolute differences are relatively small, the proportional differences for many of the subgroups are quite large. In summary, we find that the subgroups of beneficiaries with the lowest estimated employment rates (regardless of the data source) tend to exhibit higher proportional differences between the survey- and administrative data–based estimates of their employment rates. This is consistent with prior literature citing potential issues related to recall error or underreporting of earnings in surveys.

Table 3 compares administrative data– and survey-based mean annual earnings estimates for DI beneficiaries overall and by subgroup. The data cover all DI beneficiaries regardless of employment status and hence include many individuals who have no earnings. The average earnings of the overall sample are $1,125 based on the administrative data and $514 based on survey data. Consistent with the employment-rate results, we find that estimated earnings based on administrative data are higher than those based on survey data for all subgroups. The consistency in patterns between Tables 2 and 3 is not surprising given that the definitions for the employment and earnings measures are directly related. In absolute dollar amounts, the differences in the earnings estimates by subgroup range from $262 (individuals receiving concurrent DI/SSI benefits) to $778 (beneficiaries with a sensory condition).

Table 3. Estimated mean annual earnings of DI beneficiaries: Differences between administrative (MEF) and survey (NBS) data, 2003–2005
Characteristic Number (weighted) a Estimate based on— Absolute difference (MEF minus NBS) Proportional difference (%)
MEF NBS
Earnings (nominal $) Standard error Earnings (nominal $) Standard error In dollars Standard error p-value
Total 6,233,868 1,125.28 73.95 513.79 39.70 611.49 66.60 0.0 119.0
Benefit type
DI only 4,661,112 1,257.83 93.70 528.49 49.56 729.33 87.49 0.0 138.0
Concurrent DI/SSI 1,572,756 732.41 60.00 470.23 43.33 262.18 57.30 0.0 55.8
Primary disabling condition
Psychiatric 1,140,178 1,143.80 107.42 639.99 75.90 503.81 75.17 0.0 78.7
Intellectual 355,518 856.35 104.14 586.26 83.35 270.09 74.36 0.0 46.1
Musculoskeletal 1,290,348 1,028.98 171.91 395.23 95.74 633.75 150.40 0.0 160.3
Sensory 209,490 2,128.24 393.97 1,350.64 277.91 777.60 251.07 0.0 57.6
Other 2,959,033 1,054.07 127.09 354.36 41.06 699.70 121.72 0.0 197.5
Missing b 279,301 1,839.80 319.68 1,516.44 269.73 323.36 192.50 0.1 21.3
Sex
Men 2,968,716 1,033.99 84.92 433.58 47.31 600.41 78.16 0.0 138.5
Women 3,265,152 1,208.29 106.77 586.73 52.02 621.56 93.46 0.0 105.9
Age
21–29 261,801 1,767.22 141.22 1,234.88 131.61 532.35 92.79 0.0 43.1
30–39 692,642 1,653.52 120.07 947.02 91.71 706.51 107.28 0.0 74.6
40–49 1,571,112 1,369.21 131.85 700.72 57.98 668.49 118.28 0.0 95.4
50–59 2,241,472 866.12 121.68 288.89 56.45 577.23 119.80 0.0 199.8
60–64 1,466,841 896.07 172.50 324.02 86.33 572.05 143.44 0.0 176.6
Race
White only 4,654,797 1,136.14 91.17 524.00 48.06 612.14 81.47 0.0 116.8
Black or African American only 1,135,446 1,142.81 107.70 487.36 73.00 655.45 92.49 0.0 134.5
Other 443,625 966.54 340.17 474.34 85.20 492.20 333.89 0.1 103.8
Ethnicity
Non-Hispanic 5,697,871 1,137.65 77.77 527.17 42.12 610.49 68.87 0.0 115.8
Hispanic 535,997 993.77 105.80 371.62 58.89 622.16 102.84 0.0 167.4
Type of survey response
Self-report 5,084,438 1,200.18 78.11 550.15 45.62 650.04 68.26 0.0 118.2
Proxy report 1,149,430 793.89 147.03 352.94 46.35 440.94 143.83 0.0 124.9
SOURCE: Authors' calculations using linked MEF and NBS data.
NOTES: Earnings estimates represent the mean amounts for all beneficiaries, regardless of whether they had earnings during the year.
Of beneficiaries with earnings, MEF estimates account for all individuals with nonzero earnings reported in the year, and NBS estimates account for respondents who reported working at least one job held for 30 days or more in the year; the NBS estimates reflect the sum of earnings from all such jobs.
a. The weights for the survey data have been adjusted to reflect the three NBS rounds (2004, 2005, and 2006) combined into a single file. The unweighted total sample size is 7,987.
b. Not reported in the matched MEF record.

Repeating the patterns seen for employment rates, subgroups with the highest average earnings have most of the lowest proportional differences between administrative data– and survey-based estimates. For example, higher-earning subgroups include those with sensory conditions, for whom we find a 57.6 percent relative difference between administrative and survey data; and beneficiaries aged 21–29, for whom we find a 43.1 percent relative difference. Conversely, subgroups with lower average earnings tend to have higher proportional differences, including beneficiaries with musculoskeletal or “other” impairment conditions, those aged 50–64, and those who are Hispanic (all with relative differences exceeding 160 percent).

In Table 4, we restrict our analysis to beneficiaries who have nonzero annual earnings reported in both administrative and survey data. This table provides a useful contrast to Table 3 because it includes only those survey respondents who recall having some earnings. Hence, all else being equal, we would expect survey-estimated earnings to have closer concordance with administrative data for these beneficiaries than we would see for a sample that might include individuals who had earnings they did not recall.

Table 4. Estimated mean annual earnings of employed DI beneficiaries: Differences between administrative (MEF) and survey (NBS) data, 2003–2005
Characteristic Number (weighted) a Estimate based on— Absolute difference (MEF minus NBS) Proportional difference (%)
MEF NBS
Earnings (nominal $) Standard error Earnings (nominal $) Standard error In dollars Standard error p-value
Total 714,704 6,402.06 361.49 4,181.25 264.61 2,220.81 363.07 0.0 53.1
Benefit type
DI only 505,261 7,263.49 505.85 4,586.36 357.51 2,677.13 500.18 0.0 58.4
Concurrent DI/SSI 209,443 4,323.96 288.05 3,203.95 228.62 1,120.00 310.28 0.0 35.0
Primary disabling condition
Psychiatric 185,574 5,472.39 405.94 3,746.68 344.73 1,725.71 326.92 0.0 46.1
Intellectual 91,959 3,010.69 262.94 2,173.88 257.73 836.81 230.20 0.0 38.5
Musculoskeletal 87,229 9,466.25 1,902.86 5,272.36 1,292.31 4,193.89 1,776.80 0.0 79.5
Sensory 35,565 9,388.18 1,505.71 7,158.96 1,365.14 2,229.22 733.27 0.0 31.1
Other 240,622 6,791.85 660.89 4,102.68 371.62 2,689.17 730.98 0.0 65.5
Missing b 73,754 6,634.06 942.26 5,307.51 850.02 1,326.54 665.61 0.0 25.0
Sex
Men 323,305 5,483.63 337.96 3,791.32 325.28 1,692.31 269.26 0.0 44.6
Women 391,399 7,160.70 659.27 4,503.34 382.46 2,657.36 637.44 0.0 59.0
Age
21–29 79,633 5,176.43 326.56 3,792.14 300.02 1,384.30 266.41 0.0 36.5
30–39 154,251 5,852.72 403.87 4,026.28 346.32 1,826.44 345.11 0.0 45.4
40–49 230,996 7,046.49 738.21 4,522.37 369.75 2,524.12 689.49 0.0 55.8
50–59 156,778 6,404.15 1,157.05 3,773.42 681.89 2,630.73 1,223.04 0.0 69.7
60–64 93,046 6,758.35 1,718.79 4,611.48 1,051.78 2,146.87 983.44 0.0 46.6
Race
White only 554,619 6,055.85 425.89 4,190.54 308.83 1,865.31 374.07 0.0 44.5
Black or African American only 123,076 7,234.57 712.67 4,085.28 507.96 3,149.29 716.47 0.0 77.1
Other 37,009 8,821.90 3,537.33 4,361.21 816.93 4,460.70 3,630.02 0.2 102.3
Ethnicity
Non-Hispanic 672,729 6,349.97 384.67 4,167.87 276.36 2,182.11 377.81 0.0 52.4
Hispanic 41,975 7,236.86 757.96 4,395.75 596.00 2,841.11 785.19 0.0 64.6
Type of survey response
Self-report 540,221 7,344.27 481.88 4,826.08 315.54 2,518.19 474.98 0.0 52.2
Proxy report 174,482 3,484.85 222.72 2,184.77 218.60 1,300.08 211.08 0.0 59.5
SOURCE: Authors' calculations using linked MEF and NBS data.
NOTES: MEF estimates account for individuals with nonzero earnings reported in the year. NBS estimates account for respondents who reported working at least one job held for 30 days or more in the year and reflect the sum of earnings from all such jobs.
. . . = not applicable.
a. The weights for the survey data have been adjusted to reflect the three NBS rounds (2004, 2005, and 2006) combined into a single file. The unweighted total sample size is 1,357.
b. Not reported in the matched MEF record.

As in Table 3, administrative records produce higher average-earnings estimates than survey records do ($6,402 versus $4,181, respectively, for employed beneficiaries overall); but not surprisingly, the proportional difference (53.1 percent) is much lower than that in Table 3 (119.0 percent). In a change from previous tables, we find that the relationship between average earnings levels and proportional differences between administrative data– and survey-based estimates varies across subgroups in Table 4. For example, employed beneficiaries with musculoskeletal disabilities had both the highest MEF-based mean earnings and a very high proportional difference between the data sources, with administrative data–based earnings that were 79.5 percent higher than the survey-based estimates.3 Other higher-earning subgroups, however, exhibit the opposite relationship: For beneficiaries with sensory disabilities, administrative earnings estimates were only 31.1 percent higher than survey estimates. Among the subgroups with the lowest earnings, beneficiaries with intellectual disabilities had the smallest difference between data sources in absolute terms but the proportional difference of 38.5 percent is in the middle range of proportional differences among the subgroups.

We also examined median earnings estimates to assess whether the differences between survey and administrative data were consistent with the patterns seen for mean earnings (not shown). Overall and by subgroup, the median absolute differences were similar in magnitude to the mean differences. For some subgroups, we observed some proportional differences in the median earnings estimates that departed substantially from the mean proportional differences. However, those departures arose because the survey medians were substantially lower than the survey means, given that many beneficiaries had low earnings.

Finally, we examined whether the differences shown in Table 4 persisted if we measured earnings reported in either administrative or survey data, but not necessarily in both (not shown). Specifically, we ran the same estimates shown in Table 4 for (1) any earnings reported in the survey and (2) any earnings reported in the administrative records. As with our findings described above, we found that mean earnings estimated with administrative data were consistently higher than those estimated with survey data.

Results for SSI-Only Recipients

Table 5 provides a summary comparison of survey- and administrative data–based earnings differences for the total DI beneficiary sample and the SSI-only recipient sample. (Appendix Tables A-1 through A-4 repeat Tables 1 through 4 for the SSI-only sample.) SSI-only recipients have lower employment rates than do DI beneficiaries—13.6 percent versus 19.2 percent, respectively, based on administrative data. The difference based on survey data is smaller (11.6 percent versus 14.2 percent).

Table 5. Estimated mean annual employment rates and earnings: Differences between administrative (MEF) and survey (NBS) data, DI beneficiaries and SSI-only recipients, 2003–2005
Outcome Estimate based on— Difference in estimates
MEF NBS Absolute (MEF minus NBS) Proportional (%)
DI SSI only DI SSI only DI SSI only DI SSI only
Employment rate (%) 19.2 14.2 13.6 11.6 5.6 2.6 41.3 22.0
Mean annual earnings (nominal $)
All beneficiaries a 1,125.28 785.79 513.79 603.07 611.49 182.72 119.0 30.3
Employed beneficiaries 6,402.06 7,147.56 4,181.25 6,338.34 2,220.81 809.22 53.1 12.8
SOURCE: Authors' calculations using linked MEF and NBS data.
NOTE: MEF estimates account for individuals with any earnings reported in the year. NBS estimates account for respondents who reported working at least one job held for 30 days or more in the year and reflect the sum of earnings from all such jobs.
Earnings estimates represent the mean amounts for all beneficiaries, regardless of whether they had earnings during the year.
a. The weights for the survey data have been adjusted to reflect the three NBS rounds (2004, 2005, and 2006) combined into a single file.

As with the patterns for DI beneficiaries discussed above, the administrative estimates of employment rates and earnings for SSI-only recipients are higher than the survey estimates. Specifically, administrative records for SSI-only recipients consistently show a higher employment rate (14.2 percent) and average earnings ($786) than survey records do (11.6 percent and $603, respectively).

However, administrative- and survey-reported earnings of SSI-only recipients differ less than those of DI beneficiaries, particularly when viewed in proportional terms. For example, average earnings among all SSI-only recipients is 30.3 percent higher in administrative sources than in survey sources; for all DI beneficiaries, the administrative estimate is 119.0 percent higher than the survey estimate. These widely differing proportions might reflect several substantive differences between DI and SSI, including differences in the characteristics of the beneficiary populations and in program requirements for reporting earnings. For example, SSI recipients tend to be younger and have considerably less work experience than DI beneficiaries. In addition, SSI recipients are required to report their earnings more frequently than DI beneficiaries do, because SSI has lower caps on allowable earnings and lower maximum benefit levels.

Discussion

We find that estimated employment rates and earnings levels are consistently higher in administrative data than in survey data, particularly among beneficiary subgroups with employment rates below the national beneficiary average. Further, the divergence in administrative data– and survey-based estimates is greater, in both absolute and proportional terms, for subgroups with lower survey-reported employment rates than that for beneficiaries overall.

We speculate that survey respondent recall error is the main factor driving the differences. Complete and accurate recall may be particularly difficult for individuals who work part-time and/or part-year. Furthermore, although all respondents are guaranteed confidentiality, some may be wary of potential negative consequences of fully disclosing their earnings. Hence, some respondents may be reluctant to provide accurate earnings information if they believe that it could jeopardize the benefits they receive from the same agency that sponsors the NBS.

The findings here can inform decisions about when and where to make the best use of survey questions related to employment and earnings. For example, administrative reports on earnings might be a valid substitute for survey data when the variable of interest is annual earnings. This substitution might be desirable to free up questionnaire space when a limited amount of survey data can be collected within a project's scope or budget. Such considerations may be especially useful for projects in which administrative records on annual earnings are accessible to program staff and can be linked to survey respondents.

These findings can also inform policymakers considering whether to use surveys to identify target populations for future demonstration projects to support working beneficiaries. Our findings indicate that administrative records might provide more reliable information on employment and earnings, but they do not diminish the need for survey information on measures not recorded in administrative data (for example, health status) and even on some employment-related measures.

Although administrative data appear to offer greater precision than survey data in measuring annual employment and earnings, they are extremely limited in measuring other characteristics of employment. For example, administrative data do not include information on hours, wage rates, monthly earnings, occupations, or the specific time period or duration of employment. Thus, researchers and policymakers can benefit from levering both sources to improve overall data quality and expand the coverage—from any source—for important employment-outcome estimates.

Future analyses could examine differences between administrative data and estimates from other SSA-sponsored surveys of beneficiaries. With the survey-sponsor variable held constant, such work could isolate certain aspects of the other surveys, such as an emphasis on employment or a linkage to a specific employment-focused demonstration project, and examine their potential effects on reported employment and earnings. Focusing on surveys conducted as part of employment-focused demonstration projects could provide insight into whether treatment- and control-group differences in reported earnings change with differing survey contexts. Recent examples of such SSA-sponsored surveys have been conducted for the Accelerated Benefits Demonstration, the Youth Transition Demonstration, and the Mental Health Treatment Study. Each survey focuses on employment with its own context. Comparing the results of these surveys with the administrative data could thus add contextual depth to our understanding of the differences in employment-outcome estimates.

Appendix A. Tables for SSI-Only Recipients

Table A-1. Descriptive statistics for the SSI-only recipient study sample, 2003–2005
Characteristic Number (weighted) a Percentage distribution Standard error b
Total 2,676,172 100.0 . . .
Primary disabling condition
Psychiatric 652,293 24.4 1.2
Intellectual 279,063 10.4 0.7
Musculoskeletal 407,657 15.2 1.0
Sensory 87,604 3.3 0.5
Other 1,060,306 39.6 1.1
Missing c 189,248 7.1 0.5
Sex
Men 1,528,509 57.1 1.2
Women 1,147,663 42.9 1.2
Age
21–29 433,213 16.2 0.5
30–39 457,720 17.1 0.5
40–49 674,485 25.2 0.8
50–59 719,101 26.9 1.1
60–64 391,654 14.6 1.0
Race
White only 1,578,527 59.0 2.9
Black or African American only 795,797 29.7 3.0
Other 301,848 11.3 1.6
Ethnicity
Non-Hispanic 2,282,085 85.3 2.6
Hispanic 394,087 14.7 2.6
Type of survey response
Self-report 1,962,696 73.3 1.1
Proxy report 713,476 26.7 1.1
SOURCE: Authors' calculations using linked MEF and NBS data.
NOTE: . . . = not applicable.
a. The weights for the survey data have been adjusted to reflect the three NBS rounds (2004, 2005, and 2006) combined into a single file. The unweighted total sample size is 5,054.
b. Estimated using the complex survey weights provided in the data, which control for the clustering and stratification of the survey.
c. Not reported in the matched MEF record.
Table A-2. Estimated mean annual employment rates of SSI-only recipients: Differences between administrative (MEF) and survey (NBS) data, 2003–2005
Characteristic Number (weighted) a Estimate based on— Absolute difference (MEF minus NBS) Proportional difference b (%)
MEF NBS
Employment rate (%) Standard error Employment rate (%) Standard error Percentage points Standard error p-value
Total 2,676,172 14.2 0.8 11.6 0.7 2.6 0.5 0.0 22.0
Primary disabling condition
Psychiatric 652,293 23.3 1.6 19.7 1.5 3.6 1.1 0.0 18.2
Intellectual 279,063 14.5 1.4 12.0 1.5 2.5 1.4 0.1 20.7
Musculoskeletal 407,657 20.9 2.1 16.5 1.7 4.4 1.0 0.0 26.8
Sensory 87,604 7.8 1.6 5.5 1.2 2.3 1.7 0.2 42.2
Other 1,060,306 20.0 4.5 19.3 4.4 0.7 1.7 0.7 3.8
Missing c 189,248 23.9 2.9 22.0 2.6 1.8 2.0 0.4 8.4
Sex
Men 1,528,509 12.3 1.0 9.7 0.8 2.6 0.6 0.0 26.8
Women 1,147,663 16.7 1.1 14.2 1.0 2.5 1.0 0.0 17.7
Age
21–29 433,213 29.5 1.4 24.4 1.3 5.1 0.9 0.0 21.0
30–39 457,720 20.2 1.3 15.6 1.2 4.7 0.9 0.0 29.9
40–49 674,485 14.2 1.1 11.2 1.0 3.0 0.8 0.0 26.5
50–59 719,101 6.5 1.3 6.1 1.6 0.3 1.5 0.8 5.6
60–64 391,654 4.4 1.7 3.8 1.4 0.7 1.7 0.7 17.5
Race
White only 1,578,527 14.9 1.1 12.7 0.8 2.2 0.6 0.0 17.5
Black or African American only 795,797 14.8 1.3 10.7 1.3 4.1 1.1 0.0 38.3
Other 301,848 8.9 1.4 8.6 1.7 0.3 1.6 0.8 3.8
Ethnicity
Non-Hispanic 2,282,085 14.9 0.9 12.2 0.8 2.7 0.6 0.0 22.6
Hispanic 394,087 10.0 1.6 8.5 1.3 1.5 1.1 0.2 17.7
Type of survey response
Self-report 1,962,696 13.2 1.0 11.2 0.8 2.0 0.7 0.0 17.8
Proxy report 713,476 17.0 1.4 12.9 1.1 4.2 0.9 0.0 32.3
SOURCE: Authors' calculations using linked MEF and NBS data.
NOTE: MEF estimates account for individuals with any earnings reported in the year. NBS estimates account for respondents who reported working at least one job held for 30 days or more in the year.
a. The weights for the survey data have been adjusted to reflect the three NBS rounds (2004, 2005, and 2006) combined into a single file. The unweighted total sample size is 5,054.
b. Calculated using unrounded employment-rate estimates.
c. Not reported in the matched MEF record.
Table A-3. Estimated mean annual earnings of SSI-only recipients: Differences between administrative (MEF) and survey (NBS) data, 2003–2005
Characteristic Number (weighted) a Estimate based on— Absolute difference (MEF minus NBS) Proportional difference (%)
MEF NBS
Earnings (nominal $) Standard error Earnings (nominal $) Standard error In dollars Standard error p-value
Total 2,676,172 785.79 57.05 603.07 64.39 182.72 42.66 0.0 30.3
Primary disabling condition
Psychiatric 652,293 625.85 103.02 487.90 97.60 137.96 99.88 0.2 28.3
Intellectual 279,063 799.15 120.60 525.11 96.99 274.03 90.22 0.0 52.2
Musculoskeletal 407,657 477.70 138.77 361.74 139.79 115.96 64.66 0.1 32.1
Sensory 87,604 1,552.40 562.85 971.44 304.40 580.96 501.27 0.3 59.8
Other 1,060,306 760.94 96.76 582.93 84.24 178.01 70.86 0.0 30.5
Missing b 189,248 1,765.46 303.14 1,577.18 330.78 188.27 117.64 0.1 11.9
Sex
Men 1,528,509 654.28 58.50 492.12 58.35 162.16 39.93 0.0 33.0
Women 1,147,663 960.96 109.30 750.84 110.45 210.12 81.39 0.0 28.0
Age
21–29 433,213 1,474.48 143.84 1,118.17 148.88 356.30 141.78 0.0 31.9
30–39 457,720 1,097.95 101.76 917.52 129.45 180.43 120.19 0.1 19.7
40–49 674,485 920.54 140.86 763.14 146.67 157.40 64.16 0.0 20.6
50–59 719,101 390.25 125.87 220.25 95.61 170.00 85.57 0.1 77.2
60–64 391,654 153.41 68.61 93.05 51.96 60.36 41.04 0.2 64.9
Race
White only 1,578,527 833.89 80.70 696.13 83.36 137.76 61.68 0.0 19.8
Black or African American only 795,797 737.79 88.72 468.89 82.59 268.90 67.73 0.0 57.3
Other 301,848 660.82 178.00 470.15 157.28 190.67 124.73 0.1 40.6
Ethnicity
Non-Hispanic 2,282,085 758.76 59.42 623.73 72.59 135.03 40.25 0.0 21.6
Hispanic 394,087 942.32 219.79 483.42 130.26 458.90 168.31 0.0 94.9
Type of survey response
Self-report 1,962,696 817.57 72.56 663.54 82.77 154.03 50.44 0.0 23.2
Proxy report 713,476 698.38 109.41 436.72 79.09 261.66 74.11 0.0 59.9
SOURCE: Authors' calculations using linked MEF and NBS data.
NOTES: Earnings estimates represent the mean amounts for all recipients, regardless of whether they had earnings during the year.
Of recipients with earnings, MEF estimates account for all individuals with nonzero earnings reported in the year, and NBS estimates account for respondents who reported working at least one job held for 30 days or more in the year; the NBS estimates reflect the sum of earnings from all such jobs.
. . . = not applicable.
a. The weights for the survey data have been adjusted to reflect the three NBS rounds (2004, 2005, and 2006) combined into a single file. The unweighted total sample size is 5,054.
b. Not reported in the matched MEF record.
Table A-4. Estimated mean annual earnings of employed SSI-only recipients: Differences between administrative (MEF) and survey (NBS) data, 2003–2005
Characteristic Number (weighted) a Estimate based on— Absolute difference (MEF minus NBS) Proportional difference (%)
MEF NBS
Earnings (nominal $) Standard error Earnings (nominal $) Standard error In dollars Standard error p-value
Total 240,835 7,147.56 483.35 6,338.34 596.56 809.22 390.66 0.0 12.8
Primary disabling condition
Psychiatric 52,575 5,755.93 894.78 5,451.56 1,115.26 304.37 801.60 0.7 5.6
Intellectual 41,405 4,663.59 662.33 3,472.30 575.70 1,191.29 512.63 0.0 34.3
Musculoskeletal 11,720 11,209.77 3,080.74 11,442.57 3,322.67 -232.80 842.61 0.8 -2.0
Sensory 14,653 9,131.43 1,993.39 5,426.61 1,455.94 3,704.82 2,551.04 0.2 68.3
Other 87,234 7,675.92 783.27 6,875.69 836.75 800.23 578.86 0.2 11.6
Missing b 33,248 8,749.02 924.62 8,502.49 1,165.07 246.53 612.67 0.7 2.9
Sex
Men 118,843 6,546.97 521.81 6,046.86 567.06 500.11 258.56 0.1 8.3
Women 121,992 7,732.65 696.95 6,622.29 866.21 1,110.36 659.08 0.1 16.8
Age
21–29 92,388 5,761.29 402.88 5,050.97 631.22 710.32 435.91 0.1 14.1
30–39 59,319 6,852.44 699.59 6,577.56 894.47 274.88 774.83 0.7 4.2
40–49 55,696 9,738.76 1,040.76 8,720.11 1,256.26 1,018.65 627.38 0.1 11.7
50–59 25,466 7,824.03 2,544.34 5,804.20 2,274.87 2,019.83 1,650.59 0.2 34.8
60–64 7,966 5,143.30 988.65 4,542.38 1,247.38 600.91 300.31 0.1 0.1
Race
White only 160,904 7,070.87 605.44 6,423.76 702.98 647.11 517.81 0.2 10.1
Black or African American only 63,985 6,964.29 639.67 5,577.96 734.80 1,386.33 484.22 0.0 24.9
Other 15,946 8,656.81 1,736.70 8,527.47 1,929.67 129.34 1,046.89 0.9 1.5
Ethnicity
Non-Hispanic 215,418 6,754.35 482.50 6,236.29 649.50 518.06 347.47 0.1 8.3
Hispanic 25,417 10,480.21 1,329.17 7,203.24 1,343.88 3,276.97 1,522.84 0.0 45.5
Type of survey response
Self-report 162,526 7,998.16 633.56 7,546.23 837.40 451.93 444.55 0.3 6.0
Proxy report 78,308 5,382.18 820.44 3,831.41 637.47 1,550.77 598.99 0.0 40.5
SOURCE: Authors' calculations using linked MEF and NBS data.
NOTES: MEF estimates account for individuals with nonzero earnings reported in the year. NBS estimates account for respondents who reported working at least one job held for 30 days or more in the year and reflect the sum of earnings from all such jobs.
. . . = not applicable.
a. The weights for the survey data have been adjusted to reflect the three NBS rounds (2004, 2005, and 2006) combined into a single file. The unweighted total sample size is 688.
b. Not reported in the matched MEF record.

Notes

1 Beneficiaries may have more than one disabling condition.

2 The specific questions from the NBS include:

In what month and year did you start working there? In what month and year did you stop working there? How many hours per week did you usually work at this job? How many weeks per year did you usually work at this job, including paid vacation and holidays? On your job were you paid by the hour? What was your regular hourly pay, including tips and commissions? Before taxes and other deductions, how much were you paid on this job, including tips and commissions? Were you paid daily, weekly, bi-weekly, twice a month, monthly, or annually?

We used the information on hours worked per week and number of weeks worked to construct the survey-based estimate of annual earnings.

3 Only for the relatively small “other” race subgroup do we see a greater proportional difference between administrative and survey records (102.3 percent).

References

Abowd, John M., and Martha H. Stinson. 2011. “Estimating Measurement Error in SIPP Annual Job Earnings: A Comparison of Census Bureau Survey and SSA Administrative Data.” SEHSD Working Paper 2011-19. Washington, DC: Census Bureau, Social, Economic, and Housing Statistics Division.

Abraham, Katharine G., John C. Haltiwanger, Kristin Sandusky, and James R. Spletzer. 2009. “Exploring Differences in Employment between Household and Establishment Data.” Journal of Labor Economics 31(2): 129–172.

Alwin, Duane F., Kristina Zeiser, and Don Gensimore. 2013. “Reliability of Self-Reports of Financial Data in Surveys: Results from the Health and Retirement Study.” Sociological Methods & Research 43(1): 98–136.

Barnow, Burt S., and David Greenberg. 2014. “Do Estimated Impacts on Earnings Depend on the Source of the Data Used to Measure Them? Evidence from Previous Social Experiments.” Evaluation Review 39(2): 179–228.

Bound, John, Charles Brown, Greg J. Duncan, and Willard L. Rodgers. 1994. “Evidence on the Validity of Cross-Sectional and Longitudinal Labor Market Data.” Journal of Labor Economics 12(3): 345–368.

Bridges, Benjamin, Linda Del Bene, and Michael V. Leonesio. 2003. “Evaluating the Accuracy of 1993 SIPP Earnings through the Use of Matched Social Security Administrative Data.” In Proceedings of the American Statistical Association, Survey Research Methods Section, 306–311. Alexandria, VA: American Statistical Association.

Coder, John, and Lydia S. Scoon-Rogers. 1996. “Evaluating the Quality of Income Data Collection in the Annual Supplement to the March Current Population Survey and the Survey of Income and Program Participation.” SIPP Working Paper No. 96-04. Washington DC: Census Bureau.

Davies, Paul S., and T. Lynn Fisher. 2009. “Measurement Issues Associated with Using Survey Data Matched with Administrative Data from the Social Security Administration.” Social Security Bulletin 69(2): 1–12.

Ford, Reuben, Douwere Grekou, Isaac Kwakye, and Claudia Nicholson. 2014. “The Sensitivity of Impact Estimates to the Range of Data Sources Used: Analysis from a Canadian Experiment.” Paper presented at the Association for Public Policy and Management Annual Conference, Albuquerque, NM (November 6–8).

Gottschalk, Peter T., and Minh Huynh. 2005. “Validation Study of Earnings Data in the SIPP—Do Older Workers Have Larger Measurement Error?” CRR Working Paper No. 2005-07. Chestnut Hill, MA: Center for Retirement Research at Boston College.

Hurd, Michael F., Thomas Juster, and James P. Smith. 2004. “Enhancing the Quality of Data on Income: Recent Developments in Survey Methodology.” Labor and Demography Working Paper No. 0412001. Munich: University Library of Munich, Germany.

Kornfeld, Robert, and Howard S. Bloom. 1999. “Measuring Program Impacts on Earnings and Employment: Do Unemployment Insurance Wage Reports from Employers Agree with Surveys of Individuals?” Journal of Labor Economics 17(1): 168–197.

Livermore, Gina A. 2009. “Work-Oriented Social Security Disability Beneficiaries: Characteristics and Employment-Related Activities.” Disability Policy Research Brief No. 09-05. Washington, DC: Mathematica Policy Research.

Livermore, Gina, Denise Whalen, Sarah Prenovitz, Raina Aggarwal, and Maura Bardos. 2011. “Disability Data in National Surveys.” Washington, DC: Mathematica Policy Research.

Mamun, Arif, Paul O'Leary, David Wittenburg, and Jesse Gregory. 2010. “Employment Among SSA Disability Program Beneficiaries: 1996–2007.” Washington, DC: Mathematica Policy Research.

Monti, Holly, and Graton Gathright. 2013. “Measuring Earnings Instability Using Survey and Administrative Data.” SIPP Working Paper No. 260. Washington, DC: Census Bureau.

Moore, Jeffrey C., Kent H. Marquis, and Karen Bogen. 1996. “The SIPP Cognitive Research Evaluation Experiment: Basic Results and Documentation.” Washington, DC: Census Bureau, Center for Survey Methods Research.

Olsen, Anya, and Russell Hudson. 2009. “Social Security Administration's Master Earnings File: Background Information.” Social Security Bulletin (69)3: 29–45.

Pedace, Roberto, and Nancy Bates. 2000. “Using Administrative Records to Assess Earnings Reporting Error in the Survey of Income and Program Participation.” Journal of Economic and Social Measurement 26(3/4): 173–192.

Pischke, Jörn-Steffen. 1995. “Measurement Error and Earnings Dynamics: Some Estimates from the PSID Validation Study.” Journal of Business & Economic Statistics 13(3): 305–314.

Rodgers, Willard L., Charles Brown, and Greg J. Duncan. 1993. “Errors in Survey Reports of Earnings, Hours Worked, and Hourly Wages.” Journal of the American Statistical Association 88(424): 1208–1218.