by John Jankowski and Brandon Shackelford[1]

The Business R&D and Innovation Survey (BRDIS) estimates that companies performed $356 billion of research and development in the United States in 2015. Data collected by BRDIS, which asks companies to describe their business operations and R&D activities in various ways, allow this R&D to be analyzed from a variety of perspectives. For example, the primary method that BRDIS uses to classify companies to industries—selecting the industry in which the company performs the most R&D in the United States—results in an estimate of manufacturing industries performing $236 billion of R&D in 2015. However, using an alternative classification method that relies on domestic payroll produces a manufacturing R&D estimate that is almost $100 billion smaller (table 1). This large difference is in part the result of companies typically thought of as manufacturers dedicating relatively more of their U.S. payroll to activities such as wholesale and retail trade than to manufacturing.

TABLE 1. Domestic R&D performed by the company, by company classification method and industry (detail for manufacturing): 2015
(Millions of U.S. dollars)
		Method for classifying companies to industry
Industry	NAICS code	By dominant domestic R&D		By dominant domestic sales		By dominant domestic payroll
i = > 50% of the estimate is a combination of imputation and reweighting to account for nonresponse; D = data withheld to avoid disclosing operations of individual companies; * = estimate is statistically significantly different from the estimate based on dominant domestic R&D. NAICS = 2012 North American Industry Classification System. NOTES: Detail may not add to total because of rounding. Industry classification in first column was based on dominant business code for domestic R&D performance where available. For companies that did not report business codes, the classification used for sampling was assigned. Industry classification in the second column was based on dominant business code for domestic sales where available. Industry classification in the third column was based on classification used for sampling (based on domestic payroll of business establishments) where available. Companies with unknown or incomplete industry assignment are classified in the "unknown industry" line. Statistics are representative of companies located in the United States with five or more domestic employees that performed or funded R&D. SOURCE: National Science Foundation, National Center for Science and Engineering Statistics, and U.S. Census Bureau, Business R&D and Innovation Survey, 2015. Table 1 Source Data: Excel file
All industries	21–23, 31–33, 42–81	355,821		355,821		355,821
Manufacturing industries	31–33	236,132		233,554	*	139,497	*
Food	311	4,838		5,682	*	3,142	*
Beverage and tobacco products	312	1,002		1,004		230	*
Textile, apparel, and leather products	313–316	748		736		349	*
Wood products	321	195		199		253	*
Paper	322	766		2,052	*	2,185	*
Printing and related support activities	323	196	i	183	i	128	*i
Petroleum and coal products	324	214		2,005	*	1,264	*
Chemicals	325	68,196		64,839	*	33,022	*
Pharmaceuticals and medicines	3254	58,675		55,848	*	22,701	*
Soap, cleaning compound, and toilet preparation	3256	1,398		1,449		2,000	*
Other chemicals, nec	other 325	64,257		60,939		28,246	*
Plastics and rubber products	326	2,541		2,451		2,776
Nonmetallic mineral products	327	1,259		1,260		1,356	*
Primary metals	331	628		793	*	735	*
Fabricated metal products	332	2,261	i	2,320		2,641	*
Machinery	333	13,426		13,243		11,017	*
Computer and electronic products	334	72,110		71,242	*	40,247	*
Communications equipment	3342	19,424		19,379		2,401	*i
Semiconductor and other electronic components	3344	30,451		29,091	*	17,201	*
Navigational, measuring, electromedical, and control instruments	3345	14,128		14,734	*	18,857	*
Electromedical, electrotherapeutic, and irradiation apparatus	334510, 334517	2,758		3,780	*	6,153	*
Search, detection, navigation, guidance, aeronautical, and nautical system and instruments	334511	7,360		6,683	*	7,406
Other measuring and controlling devices	other 3345	4,011		4,271	*	5,298	*
Other computer and electronic products	other 334	8,106		8,037		1,787	*
Electrical equipment, appliances, and components	335	4,335		3,662	*	3,114	*
Transportation equipment	336	49,274		49,468		30,982	*
Automobiles, bodies, trailers, and parts	3361, 3362, 3363	19,078		18,805	*	16,489	*
Aerospace products and parts	3364	27,464		29,755	*	12,099	*
Other transportation	other 336	2,732		907		D
Furniture and related products	337	452		452		523
Miscellaneous	339	13,690		11,963	*	5,534	*
Medical equipment and supplies	3391	11,382		9,646	*	4,777	*
Other miscellaneous manufacturing	3399	2,309	i	2,317	i	757	*
Nonmanufacturing industries (see table 2 for detail)	21–23, 42–81	119,690		122,137		215,776	*
Unknown industry	-	0		130	*i	548	*

This InfoBrief will explore the differences in industry patterns of R&D that arise from using alternative methods for classifying companies to industries and will highlight the various ways BRDIS data can be used to describe industry patterns of business R&D in the United States. This information is helpful when analyzing BRDIS data alongside data from surveys that use different methods to assign companies to industries.

R&D by Main Economic Activity of the Business

International guidelines[2] for conducting R&D surveys advise statistical agencies to classify businesses to industries according to their "main economic activity," which should be based on turnover (i.e., sales), gross value added,[3] or other sufficiently close criteria.[4] The method used by BRDIS to assign companies to industries may produce different results than methods used by business R&D surveys in other countries and non-R&D business surveys in the Unites States that frequently categorize businesses based on the industry that accounts for most of its economic outputs. Table 1 and table 2 show the official industry-level R&D estimates published by BRDIS alongside approximations of industry estimates using two alternate methods of classifying companies: by their dominant industry in terms of domestic sales and by their dominant industry in terms of domestic payroll.

TABLE 2. Domestic R&D performed by the company, by company classification method and industry (detail for nonmanufacturing): 2015
(Millions of U.S. dollars)
		Method for classifying companies to industry
Industry	NAICS code	By dominant domestic R&D		By dominant domestic sales		By dominant domestic payroll
i = > 50% of the estimate is a combination of imputation and reweighting to account for nonresponse; * = estimate is statistically significantly different from the estimate based on dominant domestic R&D. NAICS = 2012 North American Industry Classification System. NOTES: Detail may not add to total because of rounding. Industry classification in first column was based on dominant business code for domestic R&D performance where available. For companies that did not report business codes, the classification used for sampling was assigned. Industry classification in second column was based on dominant business code for domestic sales where available. Industry classification in third column was based on classification used for sampling (based on domestic payroll of business establishments) where available. Companies with unknown or incomplete industry assignment are classified in the "unknown industry" line. Statistics are representative of companies located in the United States with five or more domestic employees that performed or funded R&D. SOURCE: National Science Foundation, National Center for Science and Engineering Statistics, and U.S. Census Bureau, Business R&D and Innovation Survey, 2015. Table 2 Source Data: Excel file
All industries	21–23, 31–33, 42–81	355,821		355,821		355,821
Manufacturing industries (see table 1 for detail)	31–33	236,132		233,554	*	139,497	*
Nonmanufacturing industries	21–23, 42–81	119,690		122,137		215,776	*
Mining, extraction, and support activities	21	4,012		2,735	*	2,092	*
Utilities	22	480		485		497
Wholesale trade	42	1,768	i	4,729	*	64,818	*
Electronic shopping and electronic auctions	454111–12	1,533	i	1,591	i	377	*i
Transportation and warehousing	48–49	403		573	*	317	*
Information	51	65,513		52,620	*	64,383
Publishing	511	33,346		31,102	*	31,675
Newspaper, periodical, book, and directory publishers	5111	98	i	129		107
Software publishers	5112	33,248		30,973	*	31,568
Telecommunications	517	3,607		3,660		3,768
Data processing, hosting, and related services	518	9,416		9,354		7,136	*
Other information	other 51	19,144		8,504	*	21,804	*
Finance and insurance	52	5,366		5,446		5,944
Real estate and rental and leasing	53	301		343	*	651	*i
Professional, scientific, and technical services	54	38,626		48,795	*	62,247	*
Architectural, engineering, and related services	5413	4,128	i	4,123	i	4,811	i
Computer systems design and related services	5415	14,333		14,819		26,578	*
Scientific research and development services	5417	16,329		14,962		26,014	*
Biotechnology research and development	541711	5,693	i	4,405	*i	8,093	*
Physical, engineering, and life sciences (except biotechnology) research and development	541712	9,931		9,843		16,950	*
Social sciences and humanities research and development	541720	705		713	*	971	*
Other professional, scientific, and technical services	other 54	3,835		14,890	*	4,844	*
Health care services	621–23	758	i	2,445	*	2,621	*
Other nonmanufacturing	23, 44–45 (excluding 454111–12), 55–56, 624, 71–72, 81	930		2,375	*	11,829	*
Unknown industry	-	0		130	i	548

Companies estimated to have at least $1 million in total R&D expenditures are asked by BRDIS to report their R&D and sales in one or more business codes that are based on the North American Industrial Classification System (NAICS). For example, an oil and gas company may have both a refining business and an oilfield support business. This company could report the majority of its sales in petroleum refineries (NAICS 32401) but report the majority of its R&D in support activities for mining (NAICS 21300). Therefore, the R&D of this company would be included in the support activities for mining industry if classified according to R&D and in the petroleum refineries industry if classified according to sales.

Classification Based on Sales

Overall, the industry patterns of R&D in table 1 and table 2 are very similar between the classification based on domestic R&D (column 1) and the one based on domestic sales (column 2). This shows that most of the companies with large amounts of U.S. R&D activity reported the majority of their sales and R&D in the same business code or codes.[5] The largest differences in dollar terms between these two classification methods appear in two groups of nonmanufacturing industries. For the "other information" industries (NAICS 512, 515, and 519), the domestic R&D performance estimate drops by more than $10 billion when companies are classified by sales as opposed to by R&D performance. The "other information" industries includes the Internet publishing and broadcasting and Web search portals industry (NAICS 519130), which is one of the most R&D-intensive industries tracked by BRDIS (detail not shown).[6] In contrast, the R&D estimate for the other professional, scientific, and technical services industries (NAICS 5411, 5412, 5414, 5416, 5418, and 5419) is $11 billion higher when companies are classified by sales as opposed to by R&D. This reflects the R&D of companies that earn the majority of their revenue from consulting services but that have higher levels of R&D expenditures in other lines of business.

Classification Based on Payroll

Classifying companies to industries based on their dominant domestic payroll results in the estimate for manufacturing industries’ R&D dropping almost $100 billion to $139 billion (table 1, column 3), compared to the official BRDIS method of classifying companies to industries. Industries with the largest decreases in R&D when this payroll classification method was used include the pharmaceuticals and medicines (NAICS 3254), communications equipment (NAICS 3342), and semiconductor and other electronic components (NAICS 3344).

The classification method used to produce this result assigns companies to industries based on payroll data from the Census Business Register.[7] The Census Business Register uses administrative and survey data to identify what type of industry activity is taking place at each business establishment in the United States. The industry assigned to a company by this method reflects the dominant industry, according to payroll, for the company in the United States. This industry may even be in a different sector than what the company would report as its "primary business." For BRDIS industry estimates, these differences are most pronounced in three types of companies: multinational companies with foreign operations that differ in nature from their operations in the United States, companies that outsource business operations to third-party contractors, and startup or development-stage companies that do not yet have business operations in their target industry.

Wholesale trade (NAICS 42) was the industry sector with the largest increase in R&D estimates when classifying companies based on domestic payroll ($65 billion), compared to the official BRDIS classification method ($2 billion). This dramatic increase—along with the increase seen in the "other nonmanufacturing" industries, which includes retail trade—is the result of companies within those industry sectors that rely on contractors to manufacture their goods or that carry out most of their manufacturing outside the United States. Within the United States, the dominant business activity for these companies in terms of payroll is the sale of their goods, not their production. The dramatic growth of R&D classified in the wholesale trade sector by BRDIS’s predecessor survey, the Survey of Industrial Research and Development, influenced the decision of which industry classification method to use when presenting BRDIS estimates.[8] R&D estimates for the scientific research and development services industry (NAICS 5417) increase under a payroll-based classification, due in part to development stage companies and other companies within that industry that primarily carry out R&D in the Unites States and either have not commercialized products yet or conduct the rest of their business outside the United States or through contractors.[9] The computer systems design and related services industry (NAICS 5415) also saw an increase under a payroll-based classification partly because of diversified information and communications technology (ICT) companies within that industry that offer information technology consulting services alongside computer and electronic hardware. For many of these companies, the services business outweighs their manufacturing activities in terms of domestic payroll.

R&D Performance with External Funds

For companies that perform R&D that is paid for by other companies, the industries represented by the performers’ main economic activity may not match those of the funders. For this reason, BRDIS asks companies that perform R&D for other companies—including but not limited to contract research organizations (also known as CROs), U.S. subsidiaries performing R&D for their foreign parents, and companies engaged in collaborative R&D agreements— to report the industries of the companies that paid for the R&D. In 2015, companies performed $30.9 billion of R&D in the United States that was paid for by other companies. Figure 1 shows the distribution of this R&D by the industry subsector of both the performing companies and the funding companies (as reported by the performers). There were only two industry subsectors where the difference between the performerbased estimate and the funder-based estimate exceeded $1 billion (figure 1). Companies classified in the professional, scientific, and technical services (NAICS 541) subsector performed over a third of the $30.9 billion of R&D but paid for less than 10% of it. In contrast, companies classified in the chemicals manufacturing (NAICS 325) subsector performed less than a third of this R&D but paid for more than half of it. These differences reflect in part large amounts of R&D funds flowing between pharmaceutical companies and CROs performing clinical trials on their behalf.

FIGURE 1. Domestic R&D paid for by other companies and performed by the company, by industry subsector: 2015

R&D by Industry Orientation (That Is, by Industry Served)

Regardless of which industry classification method is used, industry R&D estimates based on company-level data will have reduced accuracy when diversified companies are included in the sample. For example, a company with $80 million of R&D in their pharmaceuticals business segment and $20 million in their medical devices business segment would have its total R&D of $100 million assigned to the pharmaceuticals industry by BRDIS because pharmaceuticals is the largest component of its R&D performance. To address the fact that many businesses—particularly larger businesses—operate in multiple lines of business, BRDIS tabulates estimates for each of the business codes reported by companies.[10]

Figure 2 illustrates the relationship between domestic R&D performance estimates for the medical equipment and supplies industry (NAICS 3391) and its corresponding business code. In this case, the industry R&D estimate ($11,382 million) and corresponding business code R&D estimate ($11,372 million) are very similar, but only 81% ($9,220 million) of the business code estimate is from companies that were classified in the corresponding industry. This means that the industry estimate for medical equipment and supplies includes more than $2 billion of R&D that was reported in other business codes. Similarly, more than $2 billion of R&D reported in the medical equipment and supplies business code was included in the estimates for other industries. The business codes most frequently reported alongside medical equipment and supplies by diversified companies included pharmaceuticals and medicines (NAICS 3254) and electrical equipment, appliances, and components (NAICS 335).

FIGURE 2. Relationship between industry and business code domestic R&D performance estimates for the medical equipment and supplies industry (NAICS 3391): 2015

Overall, $316 billion (89%) of the R&D performed by companies in 2015 was recorded in business codes that corresponded with the industry classification of the sampled companies that was based on their dominant domestic R&D (table 3). Table 3 shows the domestic R&D performance estimate for select industries based on the primary industry of sampled companies (company level; that is, the primary classification method used by BRDIS) and the estimate based on reported business codes (line-of-business level), as well as the correspondence between the two estimates.

TABLE 3. Domestic R&D paid for by the company and others and performed by the company and amount reported in corresponding business codes, by selected industry: 2015
(Millions of U.S. dollars)
Industry	NAICS code	Primary industry estimate^a	Business code estimate
Industry	NAICS code	Primary industry estimate^a	Total	From companies classified in corresponding industry
NAICS = 2012 North American Industry Classification System. ^a The data in the column "Primary industry estimate" in this table is the same as data shown in the "By dominant domestic R&D" column in tables 1 and 2. NOTES: Detail may not add to total because of rounding. Industry classification was based on dominant business code for domestic R&D performance where available. For companies that did not report business codes, the classification used for sampling was assigned. Statistics are representative of companies located in the United States that performed or funded R&D. SOURCE: National Science Foundation, National Center for Science and Engineering Statistics, and U.S. Census Bureau, Business R&D and Innovation Survey, 2015. Table 3 Source Data: Excel file
All industries	21–23, 31–33, 42–81	355,821	355,821	316,258
Pharmaceuticals and medicines	3254	58,675	56,029	55,629
Software publishers	5112	33,248	29,885	27,075
Semiconductor and other electronic components	3344	30,451	30,805	30,300
Aircraft, aircraft engine, and aircraft parts	336411–13	25,817	21,279	20,318
Communications equipment and other computer and electronic products	3342, other 334	27,530	26,680	21,546
Other information	other 51	19,144	21,576	19,048
Automobiles, bodies, trailers, and parts	3361, 3362, 3363	19,078	18,800	18,611
Scientific research and development services	5417	16,329	16,930	15,216
Computer systems design and related services	5415	14,333	13,647	12,634
Medical equipment and supplies	3391	11,382	11,372	9,220
Data processing, hosting, and related services	518	9,416	11,684	8,729
Search, detection, navigation, guidance, aeronautical, and nautical system and instrument	334511	7,360	6,507	4,801
Other machinery	other 333	6,146	5,650	5,569
Finance and insurance	52	5,366	5,381	5,285
Food	311	4,838	4,729	4,121
Electrical equipment, appliances, and components	335	4,335	4,353	3,790
Architectural, engineering, and related services	5413	4,128	4,172	4,020
Other measuring and controlling device	other 3345	4,011	4,034	3,266
Other professional, scientific, and technical services	other 54	3,835	4,696	3,761
Semiconductor machinery	333242	3,214	3,218	3,187
Basic chemicals	3251	3,042	3,072	2,562
Electromedical, electrotherapeutic, and irradiation apparatus	334510, 334517	2,758	3,477	2,250
Plastics and rubber products	326	2,541	2,903	2,415
Engine, turbine, and power transmission equipment	3336	2,535	2,306	1,231
Guided missile, space vehicle, and related parts	336414–15, 336419	1,647	3,012	1,611
Agricultural implement	33311	1,531	1,172	1,157
Soap, cleaning compound, and toilet preparation	3256	1,398	1,987	1,248
Nonmetallic mineral products	327	1,259	1,343	1,111
Resin, synthetic rubber, and artificial synthetic fibers and filaments	3252	1,201	1,214	907
Beverage and tobacco products	312	1,002	813	772
All other, not elsewhere classified	-	28,271	33,095	24,868

Of the largest industries in terms of business R&D, semiconductor and other electronic components (NAICS 3344) and automobiles, bodies, trailers, and parts (NAICS 3361, 3362, and 3363) had the highest correspondence between the R&D estimates based on the company level and those based on the line-of-business level. This indicates that companies reporting R&D in these industries reported little R&D in other business codes.

Companies classified in some BRDIS industries conduct a significant amount of their R&D in lines of business outside of their primary industry. Industries with the largest R&D totals that correspond least between the primary company level industry estimate and the related business code estimate include aircraft, aircraft engine, and aircraft parts (NAICS 336411–13) and software publishers (NAICS 5112). Less than 80% of the $25.8 billion of R&D reported in the aircraft manufacturing industries is from corresponding business codes. Companies in this industry include large defense contractors that are often diversified across many manufacturing and nonmanufacturing industries. The business codes most commonly reported alongside aircraft manufacturing by diversified companies include guided missile, space vehicle, and related parts manufacturing (NAICS 336414–15, 336419) and search, detection, navigation, guidance, aeronautical, and nautical system and instrument manufacturing (NAICS 334511). Similarly, over $6 billion of the $33.2 billion of R&D reported in the software publishing industry is from companies that are classified in industries other than software publishing but that reported R&D for the software publishing business code. Other business codes most frequently reported alongside software publishing include data processing, hosting, and related services (NAICS 518), such as cloud computing applications, and computer systems design and related services (NAICS 5415).

In cases where primary industry estimates and corresponding line-ofbusiness estimates diverge by large amounts, it may be helpful to analyze related industries as a group rather than individually, especially when evaluating year-to-year trends that may be influenced by companies shifting their focus from one line of business to another. For example, the R&D estimate for the group of medical products industries (medical equipment and supplies, pharmaceutical products, and electromedical apparatus) may be a more accurate measure than the estimates of the component industries individually. Data users may also confirm their analysis of industry and business code R&D patterns with other data items collected by BRDIS, such as R&D by application area.

Survey Information and Data Availability

The sample for BRDIS was selected to represent all for-profit, nonfarm companies that are publicly or privately held and have five or more employees in the United States. Estimates produced from the survey and presented in this Info- Brief are restricted to companies that perform or fund R&D, either domestically or abroad. Because the statistics from the survey are based on a sample, they are subject to both sampling and nonsampling errors (see technical notes in the data table reports at https://www. nsf.gov/statistics/industry/).

In 2015, companies estimated to have at least $1 million in R&D expenditures were asked to identify all the lines of businesses in which they operated using a list of business codes and subsequently were asked to report their sales and R&D for each line of business. Companies estimated to have less R&D, and some nonresponse cases, had all their R&D assigned to the business code that corresponds with the payrollbased industry assigned to them at the time the sample was selected.

The BRDIS questionnaire used the more generic term "business activities" instead of "lines of business." BRDIS asked companies to report their business activities using a set of business codes that were based on the 2012 NAICS codes. In some cases, these codes may not be sufficiently detailed to capture the full diversity of activities a business may have. For example, only one line-of-business code is provided for software publishers (51120), but a company may have more than one software business, such as games, business applications, and operating systems.

The full set of data tables from this survey are available in the report Business R&D and Innovation: 2015 (https://www.nsf.gov/statistics/industry/). Some estimates presented in this InfoBrief, such as for alternative classification methods and the correspondence between industry and business code estimates were prepared especially for this publication and are not available elsewhere. For further information, contact Raymond Wolfe (rwolfe@nsf.gov; 703-292-7789).

Notes

[1] John E. Jankowski is Director of the Research and Development Statistics Program, National Center for Science and Engineering Statistics, National Science Foundation, 2415 Eisenhower Avenue, Suite W14200, Alexandria, VA 22314 (jjankows@nsf.gov; 703-292-7781). Brandon Shackelford is the owner of Twin Ravens Consulting, Austin, TX.

[2] Organisation for Economic Co-Operation and Development (OECD). 2015. Frascati Manual 2015: Guidelines for Collecting and Reporting Data on Research and Experimental Development. 7th ed. Paris.

[3] Value added is an economic term equivalent to gross output (value of production) less the value of intermediate inputs (such as the cost of materials and electricity consumption). This measure is not available for all establishments in the Census Business Register.

[4] According to the Frascati Manual (OECD 2015), these recommended economic activity criteria are the basis for an "institutional classification" of companies’ R&D activities.

[5] The implication, therefore, is that for most industries, domestic R&D is a criterion that is "sufficiently close" to sales for classifying companies into their main economic activity, as per international guidelines.

[6] The proportion of R&D to sales, or R&D intensity, is a commonly used indicator for comparing the relative significance of R&D across industries and among companies in the same industry.

[7] For this InfoBrief, data in the column labeled "By dominant domestic payroll" in table 1 and table 2 were tabulated using the payroll-based NAICS code assigned to each BRDIS company for purposes of sampling. The process used to select this NAICS code is described in detail in the BRDIS Methodology Report, which is available by request from NCSES.

[8] Shackelford B. 2007. Revised Industry Classification Better Reflects Structure of Business R&D in the United States. InfoBrief NSF 07-313. Arlington, VA: National Science Foundation, Division of Science Resources Statistics. Available at https://www.nsf. gov/statistics/infbrief/nsf07313/.

[9] A U.S. research center for a foreignowned company would also be included in NAICS 5417 if classified according to domestic payroll.

[10] According to the Frascati Manual (OECD 2015), these criteria are the basis for a "functional distribution" of companies' R&D activities. Companies estimated to have less than $1 million of R&D are not asked to report business codes to BRDIS. Data from these companies, which account for a small percentage of total U.S. R&D, are assigned to the business code that corresponds with its payroll-based industry.