"Academic Ranking Score": A New Reproducible Metric of Thought Leadership

Alexander Kutikov*, Boris Rozenfeld*, Brian L. Egleston ^, Mohit Sirohi*, Raymond W. Hwang #, and Robert G. Uzzo*

* - Division of Urologic Oncology, Department of Surgical Oncology, Fox Chase Cancer Center, Philadelphia, PA
# - Department of Orthopedic Surgery, New England Baptist Hospital, Boston, MA
^ - Department of Biostatistics, Fox Chase Cancer Center, Philadelphia, PA

Background: In a 2010 article by Sehgal in the Annals of Internal Medicine,1 the process of the U.S. News and World Report  (USNWR) rankings was deconstructed and analyzed.  Perhaps the biggest concern is that the USNWR relies heavily on 'reputation score'.   This is particularly true in urology where 100% of top 10 hospitals and 95% of top 20 hospitals were classified identically by the reputation score and by the overall ranking.1  The reputation score is based on a non-validated questionnaire sent to approximately 250 urologists.  The response rate is only 40-50%, meaning the urology rank in USNWR is nearly completely based on the opinion of 100-125 urologists.1  A primary reason for this is the magazine's need for 'face validity' - that is the rank list has to be 'believable' to potential readers.2, 3  In an effort to develop a more meaningful and reproducible measure of a urology department/division's contribution to the field, we calculated the “Academic Ranking Score" based entirely on publically available data.

Academic Ranking Score: A comprehensive faculty list was assembled for each urology department/division using individual hospital websites. A list of all publications produced by each hospital's urology department/division from 2005 to 2010 was then compiled. Only publications with faculty as first or last author were considered. The Academic Ranking Score was obtained by identifying the number of publications originating from a specific department/division within an institution, normalized by the Impact FactorTM of the peer-reviewed journal. The 2010 U.S. News & World Report top-fifty urology hospitals were then re-ranked based on an institution's Academic Ranking Score (Table 1).

"Academic Ranking"

“Academic Ranking” Normalized by FTE

US News & World Report  Ranking 2010

Change in Ranking

Institution

Number of Publications Adjusted by Impact FactorTM

1

1

8

7

Memorial Sloan-Kettering Cancer Center

2055.154

2

14

2

0

Cleveland Clinic

1214.559

3

2

15

12

University of Texas Southwestern Medical Center

1204.921

4

11

11

7

University of Michigan Hospitals and Health Centers

1183.854

5

13

4

-1

Ronald Reagan UCLA Medical Center

1167.714

6

3

6*

N/A*

New York Presbyterian University Hospital: Cornell

1128.542

7

4

5

-2

University of California, San Francisco

1034.897

8

5

7

-1

Duke University Medical Center

959.715

9

12

26

17

Northwestern Memorial Hospital

877.208

10

6

10

0

University of Texas M.D. Anderson Cancer Center

827.655

11

7

1

-10

Johns Hopkins Hospital

795.633

12

10

9

-3

Vanderbilt University Medical Center

654.082

13

15

3

-10

Mayo Clinic, Rochester

622.688

14

8

16

2

USC University Hospital

617.344

15

17

17

2

Barnes-Jewish Hospital/Washington University

513.64

16

18

6*

N/A*

New York - Presbyterian University Hospital: Columbia

433.947

17

26

12

-5

Massachusetts General Hospital

408.22

18

34

23

5

NYU Langone Medical Center

389.835

19

9

28

9

University of Chicago Medical Center

375.953

20

27

30

10

University of Washington Medical Center

365.847

21

20

25

4

Shands at the University of Florida

333.072

22

23

34

12

Henry Ford Hospital

286.151

23

28

32

9

University of California, Irvine Medical Center

274.631

24

19

40

16

University of Wisconsin Hospital and Clinics

267.214

25

42

39

14

St. Luke's Episcopal Hospital

262.209

26

25

18

-8

Methodist Hospital

222.864

27

33

46

19

Wake Forest University Baptist Medical Center

215.96

28

21

21

-7

Brigham and Women's Hospital

212.56

29

35

48

19

Memorial Hermann - Texas Medical Center

204.324

30

16

N/A

N/A

Fox Chase Cancer Center

202.12

31

24

22

-9

Stanford Hospital and Clinics

199.725

32

29

13

-19

Clarian Health

194.623

33

30

19

-14

University of Iowa Hospitals and Clinics

192.585

34

38

20

-14

UPMC - University of Pittsburgh Medical Center

187.639

35

22

31

-4

University of Virginia Medical Center

176.647

36

31

14

-22

Hospital of the University of Pennsylvania

162.963

37

47

45

8

Mount Sinai Medical Center

144.871

38

36

43

5

University of Kansas Hospital

127.096

39

32

37

-2

University of Maryland Medical Center

120.008

40

39

29

-11

Ohio State University Hospital

118.009

41

37

41

0

Rush University Medical Center

113.241

42

45

27

-15

Lahey Clinic

99.54

43

40

33

-10

Emory University Hospital

98.176

44

43

44

0

Loyola University Medical Center

92.064

45

41

24

-21

University of Alabama Hospital at Birmingham

75.804

46

48

36

-10

Beaumont Hospital

57.297

47

44

42

-5

Yale - New Haven Hospital

42.171

48

46

35

-13

Tampa General Hospital

24.676

49

49

38

-11

City of Hope (Duarte, Ca)

24.208

50

50

49

-1

St. Cloud Hospital (Saint Cloud, MN)

18.285

51

51

47

-4

Baylor University Medical Center (Dallas, TX)

12.97

52

52

50

-2

Christiana Care (Newark, DE)

11.33

* Columbia and Cornell Departments of Urology at New York - Presbyterian University Hospital were considered as a single entity in the USNWR analysis

Table 1: Top 50 U.S. Hospitals as determined by the 2010 USNWR rankings re-ranked using the Academic Ranking Score (2005-2010) -- a novel metric, which quantitates a hospital's contribution to the academic literature.

  1. Sehgal AR. The role of reputation in U.S. News & World Report's rankings of the top 50 American hospitals. Ann Intern Med. 2010 Apr 20;152(8):521-5.
  2. Sweitzer K, Volkwein J. Prestige among graduate and professional schools: Comparing the U.S. News' Graduate School Reputation Ratings Between Disciplines. Res High Ed. 2009;50:812-36.
  3. Brooks R. Measuring University Quality. Rev High Educ. 2005;29:1-21.