GENWiki

     Neural Nets Improve Hospital Treatment Quality and Reduce Expenses

                                June 9, 1990
                            by Jeannette Lawrence

  A new hospital information and patient prediction system has improved
  the quality of care, reduced the death rates and saved millions of
  dollars in resources at Anderson Memorial Hospital in South Carolina.
  The CRTS/QURI system uses neural networks trained with BrainMaker (from
  California Scientific Software) to predict the severity of illness and
  use of hospital resources.  Developed by Steven Epstein, Director of
  Systems Development and Data Research, the CRTS/QURI system's goal is to
  provide educational information and feedback to physicians and others to
  improve resource efficiency and patient care quality.

  The first study showed that the program was directly responsible for
  saving half a million dollars in the first 15 months even though the
  program only included half of the physicians and three diagnoses.  Since
  then, the number of diagnoses and physicians included in the program
  have increased.  The quality of care has improved such that there are
  fewer deaths, fewer complications, and a lowered readmission rate.
  Expenses have been reduced by fewer unnecessary tests and procedures,
  lowered length of stays, and procedural changes.

  Over the past five months the hospital's accounting department has
  reported a savings in the millions of dollars, although it is difficult
  to say exactly how much of this is directly due to CRTS/QURI.  According
  to Epstein, "The hospital may now be experiencing a "spillover effect"
  of new physician knowledge and behavior into areas not initially
  targeted by the CRTS/QURI system." The reported success has motivated
  several other hospitals to join in the program and has provided the
  impetus to begin a quality program with the state of South Carolina.

  At the core of this new system lies several back-propagation neural
  networks which were designed using the BrainMaker program on a PC.
  Individually trained neural networks learn how to classify and predict
  the severity of illness for particular diagnoses so that quality and
  cost issues can be addressed fairly.  After attempts to use regression
  analysis to predict severity levels for several diagnoses failed,
  Epstein turned to the BrainMaker program for a new approach and taught
  his neural networks to classify and predict severity with 95% accuracy.
  The neural networks are also used to predict the mode of discharge -
  routine through death - for particular diagnoses.

  Without these neural networks, the system could not automatically
  compare cases with the same diagnoses and like severity levels.  Most
  commercially available severity programs provide mortality as the only
  measure of quality.  Physicians typically reject systems which only
  compare costs or mortality rates as being too crude or unfair.

  In order to provide more meaningful feedback and valid comparisons
  within homogeneous groups of patients, the CRTS Acuity Indexing Scale
  was developed.  Patients are rated for the severity of illness on a
  scale from -3 to +3.  Training information is based upon the length of
  stay in the hospital which has a direct relationship to the severity of
  the illness (acuity).  When making predictions, a -3 patient is expected
  to require the least hospitalization and resources, a 0 patient average,
  and a +3 patient the most.  Cases which are run through the trained
  network can then be indexed in a database by their severity level and
  retrieved by such for comparisons.

                           Neural Network Designs

  The neural network was trained to make the severity prediction using
  variables of seven major types: diagnosis, complications/comorbity, body
  systems involved (e.g., cardiac and respiratory), procedure codes and
  their relationships (surgical or nonsurgical), general health indicators
  (smoking, obesity, anemia, etc.), patient demographics (race, age, sex,
  etc.), and admission category.

  Using a neural network to learn the effects that these variables have on
  the severity of illness takes the CRTS/QURI a step beyond the other
  programs available for indexing severity of illness.  With a neural
  network, it is possible to discover relationships that were previously
  unrecognized.  There is no need to define how the variables are related
  to the output, the neural network learns it on its own.  BrainMaker
  provides the ability to view how any one of these variables effects the
  output of the network.  Using BrainMaker's neuron graphics any input can
  be varied over some range and the effect that this has can be plotted.

  The neural network learns (just as profession literature states) that
  elderly patients with the same diagnosis require more care than younger
  patients and that patients with family support tend to require shorter
  hospital stays.  Unlike traditional programs, there is no need to
  translate premises like these into program statements or formulas.  The
  neural network learns these and other associations by viewing case after
  case until it picks up the patterns using the back-propagation
  algorithm.

  Three years of patient data was chosen for training.  There were
  approximately 80,000 patients to choose from and 473 primary diagnoses.
  The cases were pre-selected with multiple regression techniques to
  eliminate outliers so that no "bizarre" cases were used for training the
  neural network.  For a given diagnosis, about 400 to 1000 cases were
  used to train on.

  Data was collected from automated medical records, the QURI program
  (explained in the next section), and Federally required files for
  Medicare patients.  These were downloaded and read into the CRTS
  database which is in an rBase format on a PC.  The selected training
  data was output from the rBase file as a dBase file because there is a
  direct link to BrainMaker this way.  The dBase file was read by
  BrainMaker's preprocessor called NetMaker which automatically translates
  the data into neural network files for BrainMaker.  These files define
  the network architecture, the screen display, and the training and
  testing data in BrainMaker format.  After these automatic conversions
  were done, the neural network was trained in BrainMaker.

  During the initial neural network design phase, decisions had to be made
  as to which variables were important so that unnecessary data collection
  could be avoided.  In order to help make this decision, Epstein used
  BrainMaker's Hinton diagrams to see which inputs affected the trained
  network the most.  These diagrams present a picture of connection
  strengths (weights) at the hidden and output layers.  A neuron which
  connects to all the neurons in the next layer with the same strength
  transmits no useful information.

  Two neural networks for each diagnosis were trained - one to predict the
  use of resources and the other to predict the type of discharge.  For a
  single diagnosis network, there are 26 input variables and one output
  variable.  BrainMaker trained in about 4 hours using a NorthGate 386
  running at 33 MHz.  Epstein began training with a .1 tolerance which
  means every prediction for every case must be 90% accurate.  Then he
  lowered the tolerance to .085 and eventually stopped with the network
  trained at .05 training tolerance (95% accurate).  Then a test set of
  cases was run through.  The BrainMaker program has built-in testing
  capability.  In addition, Epstein and Jones verified the results of the
  network before using the network to categorize patients and make
  predictions.

                            The CRTS/QURI System

  Once the cases are indexed according to severity level, the CRTS
  (Computerized Resource Tracking System) can be used to provide
  educational feedback to physicians concerning quality care issues and
  the use of hospital resources.  CRTS combines computer software which
  produces automated graphs, reports and summaries along with a
  comprehensive educational format.  An artificial intelligence program,
  VP Expert, is used to generate individual physician's reports.

  The CRTS program is more affordable than other indexing systems in that
  no additional chart abstracting is necessary.  The majority of
  information is retrieved from the hospital's "mainframe" computer.
  Information sources include the medical records abstract, UB82 files
  (medicare), and QURI data.

  The "QURI" portion of the CRTS/QURI system is an integrated data
  collection and reporting program.  Epstein developed this stand-alone
  menu-driven, user-friendly software program which provides computer
  integration for departments which provide additional data for quality
  analysis.

  QURI stands for Quality assurance, Utilization review, Risk management
  and Infection control.  Quality Assurance monitors various criteria for
  specific procedures and diseases.  Utilization Review monitors resource
  related items such as admission justification and intensity of services.
  Risk Management determines things that put a patient or hospital at risk
  (such as a fall out of bed), and Infection Control monitors infections
  acquired during the hospital stay.

  CRTS is used for evaluation at various levels.  At the hospital level,
  comparisons are made to data published by the profession to identify
  areas of major concern for the hospital.  Individual diagnoses that show
  problems of quality and/or finance are identified.  At the physician
  specialty group level, treatment for various severity levels of a
  diagnosis are discussed.  At the individual physician level, one page of
  written summary and five pages of comparative graphs are provided.  This
  confidential report shows how a physician's patients compared to the
  hospital wide average for a particular diagnosis.  Comparisons are
  provided for areas such as the types of procedures used, ancillary
  services used, patient demographics, infection rate, complications,
  readmissions, and predicted (versus actual) illness severity level,
  charges, and mortality rate.  A summary of problem areas and a list of
  suggestions are also included in the report.

  CRTS is implemented in two primary modules 1) collecting and analyzing
  the data with neural networks, and 2) conducting the physicians'
  education program.  Epstein, who originally designed the system for his
  dissertation, is in charge of the data-related portions.  Dr. Fred
  Jones, Executive Vice President of Medical Affairs at Anderson Memorial,
  sets up physician education (feedback) programs, trains them in the use
  of system, and verifies trained neural networks results.

                             Physician Response

  Getting the physicians to accept the new system was no easy task.
  "Initially some of the physicians must have thought the program was a
  Communist Plot.  One even stood up in an early meeting to declare the
  whole thing 'crap'," says Epstein.  He explains that Jones has done a
  tremendous job in getting the physicians involved.  The physicians are
  given a say as to what information is important, what should be in the
  reports, and how to use the data.  Epstein continues, "The problem we're
  having now is that we're OVERWHELMED with requests for more diagnosis
  studies and more special reports from the physicians."

                     What Can Be Learned from CRTS/QURI

  The QURI system includes the following components: infection
  surveillance, drug use evaluation, blood usage review, surgical case
  review, monitoring and evaluation, physician demographics, case
  review/peer review, incident/occurrence monitoring, claims management
  and utilization review.  This information, as well as information from
  patient's charts is used to provide reports, graphs and charts to
  individual physicians or specialty groups.

  Hospital-wide comparisons can be trended over time and be compared to
  statewide averages or averages reported in the literature when
  available.  Comparisons between the 473 diagnoses can be evaluated and
  compared to literature or statewide averages when available.  Using the
  severity groups, computer generated graphs and charts can be presented
  to physician specialty groups and/or individual physicians displaying
  how they compare to their average hospitalwide peer or to their average
  within a specialty group for many resource and quality issues (see
  Comparative Information sidebar).  Cases with similar severity levels
  can be found and treatments compared.  Trends are also revealed in these
  same graphs depicting how a physician compares throughout severity
  levels.  A physician or group may practice more efficiently and with
  improved quality when treating higher severity level patients.  More
  indepth studies can be provided, such as how a physician's expired
  patient with a certain severity level was treated, what procedures and
  services were used, etc., versus other similarly ill patients.
  During a report review a question may come up such as why did Risk
  Management note that there was a problem?  One possible scenario might
  be that the patient fell out of bed and a closer look at the report
  would show if Quality assurance reported that too much sedation was
  given.  One report from the CRTS brings it all together.

                      How It Began at Anderson Memorial

  The first diagnosis chosen for the program was DRG89 - pleurisy/
  pneumonia.  Compared to the national norm, the hospital mortality rate
  was higher.  In addition, the hospital was losing money from cost
  overruns since insurance companies and medicare coverage have limits on
  what they will pay.  It was learned that pneumonia of unknown etiology
  had the highest mortality.  The reports showed that patients of family
  practitioners had longer lengths of stay.

  A typical scenario unfolded.  Physicians would start their patients on
  antibiotics, wait for lab results (which were inconclusive more than 25%
  of the time) then ask for an internal medicine consult on the third day.
  Then treatment started all over.  These patients stayed an average of 4
  days longer and the hospital averaged a $2764 loss.  Since this
  discovery, Family Practitioners now get an earlier consult.  Further
  research found that the sputum collection procedure was a problem.  The
  hospital now uses respiratory therapy to collect specimens.  Samples are
  quickly tested and thrown out if they are spit instead of sputum from
  the lungs.  The inadequate sputum rate decreased from 26% to 5%.  The
  improvements caused a drop of $1091 in treating an average case of
  pneumonia during the pilot program even though only half of the
  physicians were involved in the program.

  Other improvements for DRG89 followed.  The length of stay and charges
  continued to fall through 1989.  Complications and mortality rates also
  decreased.  The use of ancillary services and ICU decreased.  When
  another diagnosis was added to the program, cerebovascular disease, the
  hospital experienced a decrease in total charge of $1043 per patient,
  even though CAT scans increased.  Average length of stay also decreased.
  Anderson Memorial is now using nine diagnoses in its program.

                              What's Downstream

  In response to the national concern for the quality of health care, the
  South Carolina Hospital Association voted to approve a statewide Quality
  Indicators Project with the state.  The tracking method was developed by
  the Systems Development and Data Research department at Anderson
  Memorial.  Epstein is currently working with the state of South Carolina
  to develop a software package which will look at quality issues for
  statewide comparisons.

  Seen as only the beginning, Epstein hopes to use the results of this
  statewide quality project as a basis for designing another neural
  network for use in the hospital.  This neural network would screen all
  patient charts and other data to provide a quality index (based upon the
  statewide issues such as mortality, infections, etc.).  He proposes that
  the results could be used to decide which cases should be reviewed,
  instead of randomly picking charts to look at.  "There are too many
  patients and too much data for someone to look at every single one and
  decide which ones have quality issues that need looking into with a peer
  review," Epstein says.  This new neural network would make educating
  physicians a much more efficient process as well as improve the quality
  of care even further.

  Currently, Anderson Memorial is working with three other hospitals as
  they start their own program.  A recent presentation of the CRTS/QURI
  system at an American College of Physician Executives conference
  produced an overwhelming response.  The hospital has been beseiged with
  requests for more information from other interested hospitals.

  In the very near future at a meeting with seven hospitals from a
  hospital network, Epstein will propose a hospital-group program.  In
  this program, there would be one neural network trained for each
  hospital, plus one for all the hospitals using one diagnosis initially.
  An overall advisory group would be formed.  Comparisons could be made
  between hospitals, between physicians and the seven-hospital average, as
  well as all the currently implemented comparisons.  If approved, he
  plans to start the program in a few months.  After a six-month pilot
  program, results will be checked and verified.  And this is only the
  beginning.

*

  The CRTS program can provide comparative information for the following:

      Resource Indicators                     Quality Indicators
      -------------------                     ------------------
      total patient charge                    inpatient mortality
      length of stay                          unplanned readmissions
      ancillary charge                        hospital-acquired infections
      ancillary ratio                         surgical-wound infections
      ICU days                                neonatal mortality
      number of consultants                   perioperative mortality
      cardiopulmonary charges                 Cesaerean sections
      physical therapy charges                unplanned returns to special
      laboratory charges                              care unit
      radiology charges                       unplanned returns to O.R.
      catscan charges
      pharmacy charges
      IV solutions required

*

  Note: The bulk of this material is expected to be printed in PCAI
  approximately November, 1990.