THE EFFECT OF THE USPTO'S QUALITY-IMPROVING..., 29 Tex. Intell. Prop....

*** Start Section

...
29 Tex. Intell. Prop. L.J. 67

Texas Intellectual Property Law Journal

March, 2021

Article
Teruki Amanod1

Copyright © 2021 by State Bar of Texas, Intellectual Property Law Section; Teruki Amano

THE EFFECT OF THE USPTO'S QUALITY-IMPROVING INITIATIVES IN 2000 ON THE CLAIM SCOPE OF
BUSINESS METHOD PATENTS

Table of Contents
Introduction 67
I.The USPTO's Quality-Improving Initiatives in 2000 for Business Method Patents 70
II.Methodology 71
A. Definitions 71
1. Treated Group and Control Group 71
2. Pre-treatment and Treatment Period 73
B. DID Models 75
C. Dataset 77
III.Results and Discussion 78
A. Comparison of Independent Claim Length Between Treated and Control Groups Across the Pre-treatment and 78
Treatment Period
B. Statistical Estimation of the Effects 81
Conclusion 83

Introduction

Patent quality and value has concerned society for over two decades while the U.S. patent system has undergone significant
changes in many ways. After the patentability of software patents was judicially confirmed in the State Street Bank decision,1
the granting of some arguably overly broad business method or software- *68 related patents, and subsequent litigation based
on those patents have caused concerns.2 The most recent debate on patent subject matter eligibility under 35 U.S.C. § 101 is
also within the scope of the issue of patent quality and value in the broadest sense.

Although there was a time when the U.S. Patent and Trademark Office (USPTO) was said to be regularly issuing bad patents,3
the agency had also shared concerns and adopted many initiatives to improve patent quality.4 Following these initiatives, why
is the discussion on patent quality still ongoing? It may be that the USPTO's attempts were not enough, or it may be that the
initial problems were adequately addressed, but new challenges have surfaced. Either way, knowing the effectiveness of each
of the USPTO's initiatives is essential to an informed debate on these issues.

One difficulty in assessing the effectiveness of the initiatives is that there are no definitive measures for patent quality. Various
measures have been developed and used in literature, including patent scope, patent family size, backward/forward citations,
patent renewal, and so on.5 Each measure naturally has its advantages and disadvantages, but among them, there is a strong legal
basis for using patent (claim) scope or breadth to determine patent quality. This is simply because patent claims determine
what is protected against competitors.6 Furthermore, given that every element of a claim must be present in another's product or
process to show patent infringement, a claim with fewer elements or words should, theoretically, mean that it is broader in the

© 2021 Thomson Reuters. No claim to original U.S. Government Works. 1

THE EFFECT OF THE USPTO'S QUALITY-IMPROVING..., 29 Tex. Intell. Prop....

same or similar technical field. This indicates that the claim length can represent the claim scope and be an important indicator
of the quality and value of the patent. Despite its logical clarity and simplicity, the use of claim length as a measure of patent
quality has not made much progress until recent large-scale analyses.

Recently, one study by Alan Marco, Joshua Sarnoff, and Charles DeGrazia analyzed the word count of the shortest independent
claims in U.S. patents.7 This study confirmed that the length of independent claims increased during the examination process,
and also showed the independent claim length had explanatory *69 power for other common indicators that had been used in
literatures.8 Kuhn and Thompson also analyzed the word count of the first claim and advocated the usefulness of the claim length
by showing the explanatory power for patent renewal rates and the correspondence with evaluations by patent attorneys.9
Similarly, Okada et al. analyzed the number of characters in the first claim of Japanese patents and found the claim length
increased through the patent examination process.10 These prior studies show the potential of using claim length as an indicator
of the stringency of patent examination process and the quality of patents granted.

On the other hand, empirical analysis using claim length has just begun, and its usefulness will be established through further
research. For example, as the Marco study pointed out, such analysis assumes that similar technical terms are used, and it is
important to note that it may or may not be useful in all fields of technology.11 Therefore, a series of case studies that apply
claim length to specific events and technologies are needed to examine and improve the utility of this indicator in empirical
research and policy considerations.

This study uses claim length to examine the effectiveness of the USPTO's quality-improving initiatives implemented in 2000 for
business method patents. I chose this event because the quality of business method patents remains an often-debated issue12
and, more importantly, because the event was aimed at a specific group of patents--business method patents--which allows for
the use of quasi-natural experimental methods. Specifically, Difference-in-Differences (DID) study design was used to compare
between a treated group of patents that were subject to the treatment, and a control group of patents that were untreated
but technologically related, across the pre-treatment and treatment period. This comparison is to control possible confounding
factors such as the terminological changes as well as the impact of other systemic changes existing in common in the technical
field like the introduction of pre-grant publication scheme13 in 2000 by the American Inventors Protection Act (AIPA).

This article is organized as follows. Section I outlines the event to be studied and relevant prior studies. Section II details the
methodology for comparing treated and control group in the Difference-in-Differences (DID) study design. Section III provides
the results of the comparison and estimations of the effects by the DID analysis. Concluding remarks follow.

*70 I. The USPTO's Quality-Improving Initiatives in 2000 for Business Method Patents

In the late 90's, patent filings for business-related data processing methods and technologies increased as the electronic
commerce business industry grew. The USPTO established a new class of United States Patent Classification (USPC), Class
705 for the business method patents in September 1997.14 The Class 705 covers variety of financial and management data
processing areas defined as “Data Processing: Financial, Business practice, Management, or Cost/Price determination”,15 i.e., a
prominent portion of business method patents. After the patentability of software was judicially confirmed in the State Street
Bank decision,16 the number of patent applications in the Class 705 reached 2658 in FY 1999 (Oct. 1998 - Sept. 1999). That
number is double the number of applications from FY 1998. In FY 1999, business method patent applications corresponded
to 1% of patent applications and less than 5% of patent applications filed in the communications and information technology
area.17

In response to an increasing concern for its quality, the USPTO announced several initiatives to improve the examination process
of the business method patents in March 2000.18 Those include a mandatory non-patent literature (NPL) search, a new second-

© 2021 Thomson Reuters. No claim to original U.S. Government Works. 2

THE EFFECT OF THE USPTO'S QUALITY-IMPROVING..., 29 Tex. Intell. Prop....

level review (Second Pair of Eyes Review: SPER), and a quality review with larger sampling size for patent applications in the
Class 705, in addition to other initiatives already in place.19 The mandatory NPL search initiative required patent examiners
search core and subject-specific NPL databases as well as U.S. and foreign patent documents in examining the Class 705 patent
applications.20 The SPER of all allowed patent applications in the Class 705 was carried out by the other examiner(s) in order
to ensure the compliance with the mandatory NPL search requirement and the adequacy of the examination.21

*71 Regarding the effectiveness of the initiatives, Allison and Hunter found that, in 2001-2002, the USPTO observed a
dramatic decrease in the number of defective patents reopened by the office of patent quality review after the implementation.22
They also found the number of patent and NPL documents referenced (by examiners and /or applicants) per a Class 705 patent
granted, and the number of Class 705 patent granted with at least one reference of foreign patents or NPL increased after
the implementation even in comparison to those of technologically related patents.23 Several scholars have cited the USPTO's
statement addressing the effectiveness of the SPER. For example, Lois Matelan noted that the allowance rate of the Class 705
patents dropped from 57% before the SPER to 47% at the end of the first quarter of the FY 2001 (i.e., at the end of 2000), and
continued to drop to 11% in FY 2004 and 19% in 2005-2006.24 Allison and Tiller also mentioned the allowance rate decreased
from 56% to 36% in a year from March 2000 to March 2001 and was stable around 35% by October 2002.25

Whereas these prior studies suggest the initiatives worked, no studies investigated the effects on claim length or claim scope
itself. Furthermore, more statistically robust results can be obtained by using a Difference-in-Differences (DID) analysis. Given
that a claim amendment to avoid prior arts or to make the claim clearer would be usually done by adding words to the claim, I
hypothesized that claim length of Class 705 patents (treated group) should have increased after the initiatives. More precisely,
every patent application needs to pass the examination by patent examiners before granted a patent. The patent applicants can
narrow the scope of the claim for example by adding other elements to avoid prior art or to correct deficiencies. Considering that
words describe the elements, one can reasonably expect the examination process to generally increase the claim word counts.
In that sense, the claim length added during the examination would reflect the quality of the examination process.

II. Methodology

A. Definitions

1. Treated Group and Control Group

This study used a Difference-in-Differences (DID) study design to control the impact of other systemic changes that could have
been caused by patent law *72 amendment or judicial decisions. First, I defined the treated group. Second, the control group
is defined as the group which is affected similarly to the treated group but does not experience the treatment.

With respect to the treated group, the quality-improving initiatives implemented in 2000 seems to be applied only to patents
classified into the United States Patent Classification (USPC) Class 705 as a main classification.26 Therefore, the treated group
was defined as regular utility patents assigned into Class 705 (Class 705 patents) in the “Patent Examination Research Dataset
(Public PAIR)”.27

Next, regarding the control group, I referred to prior studies showing that business method patents were classified not only
into Class 705 but also into other classes, such as Class 235, 340, 700, 707, 709, and 713.28 I further extracted those with the
secondary classification of Class 705 to choose technologically closer patents. In other words, the control group was defined
as patents assigned into any of Class 235, 340, 700, 707, 709 or 713 in the Public PAIR29 and also having classification of
Class 705 (not as the main classification) at patent issue date in the PatentsView dataset.30 Table 1 shows the technological
definitions of the classes.

© 2021 Thomson Reuters. No claim to original U.S. Government Works. 3

THE EFFECT OF THE USPTO'S QUALITY-IMPROVING..., 29 Tex. Intell. Prop....

*73 Table 1. Technological definitions31 of the USPC classes relevant to the treated and control group

GROUP USPC CLASS TECHNOLOGICAL DEFINITION

Treated 705 Data Processing: Financial, Business practice, Management, or Cost/

Price determination

Control 235 Registers

340 Communication: Electrical

700 Data Processing: Generic control systems or Specific applications

707 Data Processing: Database, Data mining, and File management

or Data structures, Appropriate subclasses for database structure,
Database and file management

709 Electrical computers and digital processing systems: Multicomputer

data transferring

713 Electrical computers and digital processing systems: Support

2. Pre-treatment and Treatment Period

The Class 705 was established in September 1997,32 and the initiatives to be studied were announced in March 2000.33 Although
the exact initiation date of the mandatory NPL search is unclear, a preliminary analysis revealed the percentage of Class 705
patents with at least one NPL documents referenced by examiners and /or applicants increased from 2000 (data not shown). The
SPER was initiated practically *74 around mid-2000, according to the USPTO.34 It should be also noted that other USPTO-
wide quality initiatives were also implemented afterward. The USPTO's 21st century strategic action plan that was updated
in 2003 included such initiatives as examiner trainings, in-process review, and the expansion of the SPER.35 These initiatives
were mainly implemented in FY 2004 (Oct 2003 - Sep 2004).36 The SPER was proposed to expand to some advanced field of
technologies such as semiconductors, telecommunications, and biotechnology.37 The expansion appears to be done on a trial
basis in 2004,38 although the exact target is unclear. Then, the SPER was finally expanded to all classes in May 2005.39 The
USPTO mentioned these general (not specific to Class 705 patents) quality initiatives improved the overall error rate of patent
allowance in the second half of FY 2005 (Apr. - Sept. 2005).40 This information is summarized in Figure 1 below. In a strict
sense, the treatment period would be from mid-2000 to before FY 2004, and it would be also possible to include 2004 if some
underestimation is allowed. The treatment period was defined as (a) 2001-2003 or (b) 2001-2004, and the pre-treatment period
was defined as 1998-1999 to exclude 2000 to simplify the analysis.

TABULAR OR GRAPHIC MATERIAL SET FORTH AT THIS POINT IS NOT DISPLAYABLE

© 2021 Thomson Reuters. No claim to original U.S. Government Works. 4

THE EFFECT OF THE USPTO'S QUALITY-IMPROVING..., 29 Tex. Intell. Prop....

*75 Figure 1. A diagram showing pre-treatment and treatment period

B. DID Models

A simple DID model can be expressed as the following equation:41

estimated independent claim lengthi = β0 + β1 • periodi + β2 • treatedi + β3 • periodi • treatedi + ei

Here, period...

*** Start Section

.... Finally, the coefficient of the interaction term β3 is the estimator of the treatment effect. The DID analysis requires a parallel
trend assumption that the treated and control groups show a similar trend besides the effect of the treatment.42 We further
incorporated other covariates into the equation to compare DID estimations. The covariates should work to adjust for the factors
that might violate the parallel trend assumption. If the DID estimation is not sensitive, the results would be considered more
credible.43 Some characteristics that could affect the independent claim length44 were selected (Table 2).

*76 Table 2. Covariates used in the DID estimation

COVARIATE DESCRIPTION

foreign _application dummy Dummy variable for claiming priority to foreign (non-US)
patent(s) (yes = 1, no = 0)

continuation _in_part dummy Dummy variable for continuation in part application (yes = 1,
no = 0)

continuation dummy Dummy variable for continuation application (yes = 1, no = 0)

divisional_continuation dummy Dummy variable for divisional continuation application (yes =

1, no = 0)

provisional_application dummy Dummy variable for claiming the benefit of us-provisional

application (yes = 1, no = 0)

small-entity dummy Dummy variable for applicants being an individual inventor, a

collaboration of individual inventors, a nonprofit organization,
or a company with fewer than 500 employees45 (yes = 1, no =
0)

post_implementation_filing dummy Dummy variable for applications filed after the initiatives were
implemented (filed in/after 2001 = 1, filed in/before 1999 = 0)

© 2021 Thomson Reuters. No claim to original U.S. Government Works. 5

THE EFFECT OF THE USPTO'S QUALITY-IMPROVING..., 29 Tex. Intell. Prop....

*77 C. Dataset

Claim length data of U.S. utility patents was retrieved from “Patent Claims Research Dataset”46 provided by the USPTO.
The “patent_document_ stats” file in the dataset contains minimum and average word count of independent claims at grant
of each patent from 1976 to middle of 2014. In this study, the minimum word count of independent claims in each patent
was analyzed as the “independent claim length.” This is because theoretically it should represent the broadest scope of the
patent. I also confirmed that both minimum and average word count of independent claims show a similar trend over time (data
not shown).47 Although the independent claim length added (or deleted) during the examination was my primary interest, the
data was not available for most of the Class 705 patents studied in this study. This is because these patents were filed before
the pre-grant publication scheme48 was introduced, and only the granted claims had been published. Therefore, I analyzed
the independent claim length at patent grant. The introductory claiming language such as “I claim” or “What is claimed is”
and numeric formulas are not included in the counts. Whereas US6506748 that is about drug compounds (USPC Class 514)
and apparently wrongly assigned into the Class 705 was excluded, it was impossible to check all the other patents manually.
Therefore, the classification used in the dataset was relied upon generally.

The Public PAIR dataset49 was used to retrieve other basic information. I used the allowance date showing that one or more
claims in the application have been allowed,50 instead of the patent issue date which generally appeared several months later.
Few (less than 5% of) patents had multiple allowance dates due to several reasons such as requests for continued examination.
I used the latest allowance date among the multiple dates to reflect the possible amendment because the independent claim
length was recorded based on the latest, issued patent. I also retrieved and calculated from the database the number of Non-
Final and Final Rejections issued.51 Some patents did not contain transactions-related data such as the allowance date, which
is presumably due to the errors in data entry into the USPTO's datasets used *78 or the original database. These observations
were also excluded, and there was eventually on average 6%, at most 17% difference in the number of Class 705 patents granted
in a year between USPTO's statistics52 and our dataset.

III. Results and Discussion

A. Comparison of Independent Claim Length Between Treated and Control Groups Across the Pre-treatment and
Treatment Period

Given that a claim amendment to avoid prior arts or to make the claim clearer would be usually done by adding words to the
claim, I hypothesized that claim length of the treated group should have increased after the initiatives.

Figure 2 shows yearly averages of independent claim length of Class 705 patents between 1998 and 2005, across the
implementation of the initiatives in middle of 2000. The duration covers most of the period from the establishment of the Class
705 to the implementation of other USPTO-wide quality initiatives. The control group that was not subject to the initiatives but is
technologically related is used to represent a general trend in the field. As assumed, both groups similarly show decreasing trend
in the pre-treatment period. From 1998 to 2000, Class 705 patents showed a decrease by 21.0 words (from 161.8 words to 140.8
words) and the control group by 17.6 words (from 156.8 words to 139.2 words). This result further supports the assumption
that the control group reflects a general trend that could include changes in terminology at the filing or other general policy
changes not specific to the Class 705. In contrast to the decreasing trend in the pre-treatment period, the independent claim
length of the Class 705 patents increased from 138.5 words in 2000 to 190.9 words in 2005, whereas that of the control group
remained in almost the same level. As a result, the difference between the two groups which would have reflected the effects
of the initiatives increased, reaching around 40 words in 2003-2005.

TABULAR OR GRAPHIC MATERIAL SET FORTH AT THIS POINT IS NOT DISPLAYABLE

© 2021 Thomson Reuters. No claim to original U.S. Government Works. 6

THE EFFECT OF THE USPTO'S QUALITY-IMPROVING..., 29 Tex. Intell. Prop....

*79 Figure 2. Changes in independent claim length of Class 705 patents between 1998 and 2005, in comparison with
the control group53

Interestingly, contrary to the intuition that a certain effect might emerge immediately in the treatment period, the difference
between the two groups was found to be increasing gradually. The most straightforward interpretation would be that it took
time for the effect to show. The reasons for this are not entirely clear, but the analysis of the office actions issued against these
patents during the examination process provides one of interpretation.

Figure 3 shows changes in percentage over time between the control group and Class 705 patents which have received at least
one Non-Final Rejection or Final Rejection (in addition to Non-Final Rejection). Interestingly, over 70% of Class 705 patents
had received at least one Non-Final Rejection even in the pre-treatment period, as had the control group. On the other hand, the
percentage of having received at least one Final Rejection drastically increased after the initiatives in the Class 705 patents,
and the difference between the control group gradually increased as time passed since the initiatives.

TABULAR OR GRAPHIC MATERIAL SET FORTH AT THIS POINT IS NOT DISPLAYABLE

*80 Figure 3. Changes in percentage of Class 705 patents granted having received at least one Non-Final or Final
Rejection, in comparison with the control group

Given the nature of the Final Rejection as a second or any subsequent office action from the patent examiner,54 the gradual
increase of them indicates that the patent examiners became more cautious to grant a patent as time passed. This suggests a
scenario that patent examiners or reviewers for the SPER, or other quality reviews, needed time to catch up with the new or
higher internal rules or standards required by the initiatives. This scenario is not contradictory to the fact that the allowance rate
of Class 705 patents gradually increased after the initiatives were implemented.55 Future research on changes in the grounds
of rejection (e.g., patent *81 eligibility, novelty, non-obviousness etc.) may further clarify the mechanisms in detail.

Some might think of a possibility that applicants changed their filing behavior in response to the initiatives to increase the claim
length at the filing of the Class 705 patents aiming to avoid prior arts etc. This possibility cannot be directly examined because
the claim length at the filing is not available for the majority of the Class 705 patents studied.56 Nevertheless, Class 705 patents
filed in/after 2001 constituted only 5.1%, 12.0%, and 5.7% of those allowed in 2002, 2003, and 2004, respectively. In other
words, of the Class 705 patents analyzed in this study were filed before the implementation of the initiatives, indicating this
scenario is unlikely to have contributed significantly. Preliminary analysis with limited data to those filed before the initiatives
gave almost the same results with Figure 2 (data not shown).

Taken together, it is strongly suggested that the initiatives actually increased the independent claim length at grant of the Class
705 patents, and that independent claim length may be useful indicator of the stringency of patent examination process and
the claim scope in this area of technology as well.

B. Statistical Estimation of the Effects

Finally, the effects of the initiatives were statistically verified and estimated by the Difference-in-Differences (DID) analysis.
Estimating the effects in each year might be desirable given that the effects seem to have appeared gradually. However,
meaningful estimates could not be obtained due to high standard errors (data not shown). Instead, a two-group, two-period
DID design was used to compare the independent claim length between the treated and control groups before and after the
initiatives.57 The similar trend between the two groups in the pre-treatment period shown in Figure 2 supports the parallel trend
assumption required in the DID analysis, although it is not strictly testable without additional time periods.58

© 2021 Thomson Reuters. No claim to original U.S. Government Works. 7

THE EFFECT OF THE USPTO'S QUALITY-IMPROVING..., 29 Tex. Intell. Prop....

Table 3 summarizes the results of simple Ordinary Least Squares (OLS) regressions of the DID models. First, in the models
(1)-(3), the average treatment effect was estimated for the treatment period of 2001-2003 (Table 3(a)). The estimated effects
were positive and statistically significant with p<0.05 in every model. The most basic model (1) estimated 18.5 words increase,
indicating the initiatives increased the independent claim length at grant of the Class 705 patents by 18.5 words in average.
The models (2) and (3) with covariates set-1 and set-2 respectively gave similar estimations of 17.0 and 18.2 words increase,
which shows the covariates did not affect the results significantly. Furthermore, the result from *82 model (3) in which post_
implementation_filing dummy variable was included, confirmed that the application filed after the initiatives did not affect the
results significantly and supported the view that the filing-behavior-change scenario is unlikely to have contributed.59

Next, in the models (4)-(6), the treatment period was expanded to include the year of 2004 when the other USPTO-wide quality
initiatives were also implemented. The inclusion of 2004 in the treatment period should theoretically lead to an...

*** Start Section

...
(A) (1) (2) (3)

Estimated average treatment effect (β3)(i) 18.5473**(ii) 17.01441** 18.2062**

(8.548966) (8.47422) (8.807039)

Pre-treatment period 1998-1999

Treatment period 2001-2003

Covariates No set-1(iii) set-2(iv)

Number of observations 3627 3176(v)

(b) (4) (5) (6)

Estimated average treatment effect (β3) 23.79674*** 22.68979*** 23.46318***

(8.870706) (8.163467) (8.409304)

Pre-treatment period 1998-1999

Treatment period 2001-2004

Covariates No set-1 set-2

Number of observations 4747 3879

*83 Models (5) and (6) gave similar results as model (4), as did models (2) and (3) compared to model (1). These results
indicate that the selection of the control group is appropriate and that this DID analysis is reliable. Although it was difficult to
make a precise estimation due to the above reasons, DID analysis confirmed that the quality-improving initiatives implemented
in 2000 increased the independent claim length at grant of the Class 705 patents statistically significantly, suggesting that the
initiatives contributed to narrow the claim scope of the Class 705 patents. The estimated average treatment effects were in
a range of 17.0 to 23.8 words increase, as the independent claim length at patent grant. Given that the examination process
typically increases the independent claim length by a few dozen words,62 it seems quite significant that the initiatives increased

© 2021 Thomson Reuters. No claim to original U.S. Government Works. 8

THE EFFECT OF THE USPTO'S QUALITY-IMPROVING..., 29 Tex. Intell. Prop....

the length by 17.0 to 23.8 words. Future studies may further clarify the relationship between the independent claim length,
claim scope, and the fate and value of the patent.

Conclusion

The purpose of this article was to see and improve the practical usefulness of claim length as an indicator of patent quality
through a case study.

In this study, the effectiveness of USPTO's quality-improving initiatives implemented in 2000 for business method patents was
examined by using claim length in a Difference-in-Differences (DID) study design. The DID analysis revealed that independent
claim length of the Class 705 patents increased, whereas that of the control group remained in the almost same level. As a
result, the difference between the two groups which would have reflected the effects of the initiatives increased, reaching around
40 words in 2003-2005. Taken together with the changes in percentage of Class 705 patents granted having received at least
one Final Rejection, it was suggested that patent examiners or reviewers for the SPER or other quality reviews needed time
to catch up with the new or higher internal rules or standards required by the initiatives. Finally, the effects of the initiatives
were statistically verified and estimated by simple OLS regressions of DID models. The estimated *84 average treatment
effects were positive and statistically significant with p<0.05 in every model, being in a range of 17.0 to 23.8 words increase,
as the independent claim length at patent grant. These effects appeared quite significant considering the typical increase by
the examination process.

This article provides an empirical basis regarding the effects of USPTO's quality-improving initiatives in 2000 on the claim
scope of business method patents, and at the same time, shows the practical usefulness of the independent claim length as an
indicator of the stringency of patent examination process and the claim scope in this area of technology.

Footnotes
d1 Deputy director of policy planning and research division, Japan Patent Office. The author conducted this research as a visiting scholar
at Northwestern University Pritzker School of Law. The opinions in this article are the author's independent academic views and are
not attributable to any of his organization or affiliations. The author is grateful for Prof. David L. Schwartz for helpful comments
and suggestions.
1 State Street Bank & Trust Co. v. Signature Fin. Grp., Inc., 149 F.3d 1368, 1375 (Fed. Cir. 1998), abrogated by In re Bilski, 545
F.3d 943, 959 (Fed. Cir. 2008) (holding that the “machine-or-transformation” test should be used to determine patent eligibility of
a process claim instead of the “useful, concrete, and tangible result” test).
2 See, e.g., Mark A. Lemley & Bhaven Sampat, Is the Patent Office a Rubber Stamp?, 58 Emory Law J. 181, 181-82 (2008); Executive
Office of the President, Patent Assertion and U.S. Innovation 1, 2-4 (June 2013), https://obamawhitehouse.archives.gov/sites/default/
files/docs/patent_report.pdf
3 Lemley & Sampat, supra note 2.
4 E.g., USPTO, 2010-2015 Strategic Plan 1, 8-9 (2010), http://www.uspto.gov/about/stratplan/
USPTO_2010-2015_Strategic_Plan.pdf; USPTO, The 21st Century Strategic Plan 1, 7 (updated on Feb. 3, 2003).
5 See generally Mariagrazia Squicciarini, Hélène Dernis & Chiara Criscuolo, Measuring Patent Quality: Indicators of Technological
and Economic Value, OECD Sci. Tech. & Ind. Working. Papers. 70 (June 6, 2013).
6 See 35 U.S.C. § 271(a); 35 U.S.C. § 112(b) or pre-AIA 35 U.S.C. § 112.
7 Alan C. Marco, Joshua D. Sarnoff & Charles A.W. DeGrazia, Patent Claims and Patent Scope, 48 Res. Pol'y., no. 103790, 2019.
8 See, e.g., Id. at 1, 10-12.

© 2021 Thomson Reuters. No claim to original U.S. Government Works. 9

THE EFFECT OF THE USPTO'S QUALITY-IMPROVING..., 29 Tex. Intell. Prop....

9 Jeffrey M. Kuhn & Neil C. Thompson, How to Measure and Draw Causal Inferences with Patent Scope, 26 Int. J. Econ. Bus. 5 (2019).
10 Yoshimi Okada, Yusuke Naito & Sadao Nagaoka, Making the Patent Scope Consistent with the Invention: Evidence from Japan, 27
J. Econ. Mgmt. Strategy 607 (2018).
11 Marco, Sarnoff, & DeGrazia, supra note 7, at 10-12.
12 See Exec. Office of the President, supra note 2.
13 American Inventors Protection Act of 1999, Pub. L. No. 106-113 (1999).
14 USPTO, USPTO's classification order 1659 (issued on September 2, 1997), https://www.uspto.gov/sites/default/files/documents/
archiverpt.pdf.
15 USPTO, Classification Resources, https://www.uspto.gov/web/patents/classification/uspc705/defs705.htm; see also John R Allison
& Starling David Hunter, On the Feasibility of Improving Patent Quality One Technology at a Time: The Case of Business Methods,
Berkeley Tech. L. J., 21(2), 729, 734 n.15 (2006).
16 See State Street Bank, 149 F.3d 1368.
17 USPTO, A USPTO white paper: Automated financial or management data processing methods (business methods), https://
www.uspto.gov/sites/default/files/web/menu/busmethp/whitepaper.pdf [hereinafter USPTO White Paper].
18 Id.
19 Id. (noting the other initiatives in place prior to March 2000 include the hiring of more and experienced examiners, training efforts/
partnerships with industry associations and companies, and customer partnerships).
20 Id.
21 Id. (noting the SPER was required “to ensure compliance with the mandatory search requirements, clarity and completeness of reasons
for allowance, and to determine whether the scope of the claims should be reconsidered.”).
22 Allison & Hunter, supra note 15, at 737.
23 Id. at 748-57, 758-64 (at the same time, pointing out the SPER was not sufficient to cover all range of business method patents that
could be classified into other main classification than Class 705, questioning the effectiveness of these patent classification-based
approach to improve overall patent quality.).
24 Lois Matelan, The Continuing Controversy over Business Methods, 18 Fordham Intell. Prop. Media Ent. L.J. 189, 207-08 (2007).
25 John R. Allison & Emerson H. Tiller, The Business Method Patent Myth, 18 Berkeley Tech. L.J. 987, 1026 (2003).
26 See USPTO, White Paper, supra note 17; Allison & Hunter, supra note 15, at 734 n.14 (“The program, referred to as the Second
Pair of Eyes Review (SPER) because of the second-level examination, applies only to allowed applications with a main classification
of 705.”).
27 USPTO, Patent Examination Research Dataset (Public PAIR), https://www.uspto.gov/learning-and-resources/electronic-
data-products/patent-examination-research-dataset-public-pair [hereinafter Public PAIR] (the variable “uspc_class” in the
APPLICATION DATA file denotes “Invention U.S. Classification”); see also Stuart J.H. Graham, Alan C. Marco & Richard Miller,
The USPTO Patent Examination Research Dataset: A window on patent processing, 27 J. Econ. Manag. Strateg. 554-78 (2018).
28 Allison & Tiller, supra note 25, at 1035 n.161; Allison & Hunter, supra note 15, at 758-764.
29 USPTO, Public PAIR, supra note 27.
30 The PatentsView, http://data.patentsview.org/20200630/download/uspc.tsv.zip (the table “uspc” contains USPC classification data
for patents at patent issue date).

© 2021 Thomson Reuters. No claim to original U.S. Government Works. 10

THE EFFECT OF THE USPTO'S QUALITY-IMPROVING..., 29 Tex. Intell. Prop....

31 USPTO, Classification Resources, supra note 15.

32 USPTO, Classification Order 1659, supra note 14.
33 USPTO, White Paper, supra note 17.
34 E-mail from Associate Counsel of the Office of General Law, USPTO (October 16, 2019). (“I found out that all Class 705 applications
that were “pending” when the program was initiated (around mid-2000) were subject to review.”).
35 USPTO, 2010-2015 Strategic Plan, supra note 4.
36 USPTO, FY 2004 Performance and Accountability Report, https://www.uspto.gov/sites/default/files/about/stratplan/ar/
USPTOFY2004PAR.pdf.
37 USPTO, 2010-2015 Strategic Plan, supra note 4.
38 See NATIONAL ACADEMY OF PUBLIC ADMINISTRATION, US Patent and Trademark Office: Transforming to Meet the
Challenges of the 21st Century. (2005), https://www.napawash.org/uploads/Academy_Studies/05USPatentandTrademarkOffice.pdf
(“This was begun in the business methods areas in March 2000 and has been expanded to other areas on a trial basis to determine
best practices.” As of April 30, 2005, “[s]ome of the TC directors and supervisory PEs indicated that the second-pair-of-eyes reviews
had been very helpful in improving the patent quality. For example, some art unit directors credited the reviews with reducing their
error rates by half during the last two quarters of FY 2004.”).
39 See Gene Quinn, An Interview with the Acting Commissioner for Patents, ipwatchdog (May 12, 2009), https://
www.ipwatchdog.com/2009/05/12/an-interview-with-the-acting-commissioner-for-patents/id=3381/ (““Second Pair of Eyes”
enhanced reviews were implemented in all areas of technology in May of 2005.”); see also USPTO, FY 2006 Performance and
Accountability Report, https://www.uspto.gov/sites/default/files/about/stratplan/ar/USPTOFY2006PAR.pdf.
40 USPTO, FY 2005 Performance and Accountability Report, https://www.uspto.gov/sites/default/files/about/stratplan/ar/
USPTOFY2005PAR.pdf
41 See generally Coady Wing, Kosali Simon & Ricardo A. Bello-Gomez, Designing Difference in Difference Studies: Best Practices
for Public Health Policy Research, 39 Annu. Rev. Public Health 453, 456 (2018).
42 See generally id. at 455-57.
43 See generally id. at 459.
44 See, e.g., Marco, Sarnoff, and DeGrazia, supra note 7; Kuhn and Thompson, supra note 9; Okada, Naito, and Nagaoka, supra note 10.
45 Graham, Marco, and Miller, supra note 27.
46 USPTO, Patent Claims Research Dataset, https://www.uspto.gov/learning-and-resources/electronic-data-products/patent-claims-
research-dataset; see also Marco, Sarnoff, and DeGrazia, supra note 7.
47 USPTO, Patent Claims Research Dataset, supra note 46.
48 See American Inventors Protection Act of 1999, Pub. L. No. 106-113; see also USPTO, USPTO Will Begin Publishing Patent
Applications (November 27, 2000), https://www.uspto.gov/about-us/news-updates/uspto-will-begin-publishing-patent-applications
(noting patent applications filed on or after November 29, 2000 is published eighteen months after the effective filing date of the
application. The claim length at the filing, precisely speaking, at the publication is available only for those patent applications.).
49 USPTO, Public PAIR, supra note 27.
50 Id. (the event code of “N/=.” in the TRANSACTIONS file denotes “Notice of Allowance Data Verification Completed”).
51 Id. (the event code of “CTNF” and “CTFR” in the TRANSACTIONS file denotes “Non-Final Rejection” and “Final Rejection”,
respectively).

© 2021 Thomson Reuters. No claim to original U.S. Government Works. 11

THE EFFECT OF THE USPTO'S QUALITY-IMPROVING..., 29 Tex. Intell. Prop....

52 USPTO, Class 705 Application Filing and Patents Issued Data (as of December 14, 2016), https://www.uspto.gov/patents-getting-
started/patent-basics/types-patent-applications/utility-patent/business-methods-18.
53 Note the allowance year of 1997 was omitted due to limited number of observations for the control group.
54 USPTO, 706.07(a) Final Rejection, When Proper on Second Action, Manual of Patent Examining Procedure (MPEP), 7th Ed.,
(Jul. 1998), https://www.uspto.gov/web/offices/pac/mpep/old/E7R0_700.pdf; USPTO, 706.07(a) Final Rejection, When Proper on
Second Action, Manual of Patent Examining Procedure (MPEP), 8th Ed., (Aug. 2001), https://www.uspto.gov/web/offices/pac/mpep/
old/E8R0_700.pdf (Any edition during 1998-2005 describes Final Rejection as follows: “Under present practice, second or any
subsequent actions on the merits shall be final, except where the examiner introduces a new ground of rejection that is neither
necessitated by applicant's amendment of the claims nor based on information submitted in an information disclosure statement filed
during the period set forth in 37 CFR 1.97(c) with the fee set forth in 37 CFR 1.17(p).”).
55 See Matelan, supra note 24, at 207-208; Allison & Tiller, supra note 25, at 1026 (Taken together, the allowance rate was around 55%
before the initiatives and decreased to around 35% in 2001-2002 and then to around 10-20% in 2004-2005.).
56 American Inventors Protection Act of 1999, Pub. L. No. 106-113; USPTO, USPTO Will Begin Publishing Patent Applications, supra
note 48 and accompanying text.
57 See supra section II. A, B.
58 See supra section II. B.
59 See supra section III. A.
60 See supra section III. A.
61 (i) The coefficient in each model was estimated by simple OLS regression. Standard errors in parentheses.
(ii) Inference: *** p<0.01; ** p<0.05
(iii) Covariates set-1 includes all the foreign_application, continuation_in_ part, continuation, divisional_continuation,
provisional_application, and small-entity dummy variables to further control patent and applicant characteristics (See Table 2 in
section II. B.).
(iv) Covariates set-2 includes post_implementation filing dummy variable in addition to the variables of Covariates set-1.
(v) The difference in the number of observations between models (1)-(2) and (3), (4)-(5) and (6) comes from the applications filed in
2000. The post_ implementation_filing dummy variable was assigned for patents filed before and after 2000, excluding 2000 (See
Table 2 in section II. B.).
62 Marco, Sarnoff, and DeGrazia, supra note 7 (“The examination process increases the mean ICL from 106 words at publication to
156 words at issuance for application years 2001-2014.”).

29 TXIPLJ 67

End of Document © 2021 Thomson Reuters. No claim to original U.S. Government

Works.

...

© 2021 Thomson Reuters. No claim to original U.S. Government Works. 12