Ethics for the Information Age, Eighth Edition, Chapter 8, Computer Reliability
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Computer Reliability
Chapter 8
Eighth Edition
Ethics for the Information Age
If this PowerPoint presentation contains mathematical equations, you may need to check that your computer has the following installed:
1) MathType Plugin
2) Math Player (free versions available)
3) NVDA Reader (free versions available)
1
Learning Objectives (1 of 2)
8.1 Introduction
8.2 Data-entry or data-retrieval e
ors
8.3 Software and billing e
ors
8.4 Notable software system failures
8.5 Therac-25
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Learning Objectives (2 of 2)
8.6 Tesla Version 7.0 (Autopilot)
8.7 Uber Test-vehicle accident
8.8 Computer simulations
8.9 Software engineering
8.10 Software wa
anties and vendor liability
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
8.1 Introduction
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
8.1 Introduction
Computers an integral part of modern communication, transportation, retail, banking, finance, health-care systems
Save time, money and enable greater productivity when working co
ectly
Failures can result in lost time, lost money, injury, or even death
Studying failures a way to appreciate complexity of building reliable computerized systems
Examine increasingly important area of computer simulations
High-level overview of software engineering
Software wa
anties and issue of responsibility of software manufacturers for quality of their products
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
8.2 Data-Entry or Data-Retrieval E
ors
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Two Kinds of Data-Related Failure
A computerized system may fail because wrong data entered into it
A computerized system may fail because people inco
ectly interpret data they retrieve
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
November 2000 general election
Florida disqualified thousands of voters
Reason: People mistakenly identified as felons
Cause: Inco
ect records entered in voter database
Consequence: May have affected outcome of national presidential election
Disfranchised Voters
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Sheila Jackson Stossier mistaken for Shirley Jackson
A
ested and spent five days in detention
Roberto Hernandez mistaken for another Roberto Hernandez
A
ested twice and spent 12 days in jail
Te
y Dean Rogan a
ested after someone stole his identity
A
ested five times, three times at gun point
False A
ests
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
March 2003: Justice Dept. announces F B I not responsible for accuracy of N C I C information
Exempts N C I C from some provisions of Privacy Act of 1974
Should government take responsibility for data co
ectness?
Accuracy of N C I C Records
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Impractical for F B I to be responsible for data’s accuracy
Much information provided by other law enforcement and intelligence agencies
Agents should be able to use discretion
If provisions of Privacy Act strictly followed, much less information would be in N C I C
Result: fewer a
ests
Dept. of Justice Position
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Number of records is increasing
More e
oneous records → more false a
ests
Accuracy of N C I C records more important than eve
Position of Privacy Advocates
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Over 1 million cars stolen every yea
Just over half are recovered, say 500,000
Assume N C I C is responsible for at least 20%
100,000 cars recovered because of N C I C
Benefit of $5,000 per car (owner gets car back; effects on national insurance rates; criminal doesn’t profit)
Total value of N C I C stolen vehicle database: $500 million/yea
Act Utilitarian Analysis: Database of Stolen Vehicles (1 of 2)
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Only a few stories of false a
ests
Assume 1 false a
est per year (probably high)
Assume harm caused by false a
est $55,000 (size of award to Rogan)
Benefit surpasses harm by $499,945,000/yea
Conclusion: Good to have N C I C stolen vehicles database
Act Utilitarian Analysis: Database of Stolen Vehicles (2 of 2)
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
8.3 Software and Billing E
ors
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Assume data co
ectly fed into computerized system
System may still fail if there is an e
or in its programming
E
ors When Data Are Co
ect
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Qwest sent inco
ect bills to cell phone customers
Faulty U S D A beef price reports
U.S. Postal Service returned mail addressed to Patent and Trademark Office
Spelling and grammar e
or checkers increased e
ors
New York City Housing authority overcharged renters
About 450 California prison inmates mistakenly released
E
ors Leading to System Malfunctions
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Ambulance dispatch system in London
Japan’s air traffic control system
Los Angeles County + U S C Medical Center laboratory computer system
Boeing 777 (Malaysia Airlines flight)
N A S D A Q stock exchange shut down
Insulin pump demo at Black Hat conference
Jeep Cherokee
E
ors Leading to System Failures
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Amazon.com in Britain offered iPaq for £7 instead of £275
Orders flooded in
Amazon.com shut down site, refused to deliver unless customers paid true price
Was Amazon.com wrong to refuse to fill the orders?
Analysis: E-Retailer Posts Wrong Price, Refuses to Delive
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Imagine rule: A company must always honor the advertised price
Consequences
More time spent proofreading advertisements
Companies would take out insurance policies
Higher costs → higher prices
All consumers would pay higher prices
Few customers would benefit from e
ors
Conclusion
Rule has more harms than benefits
Amazon.com did the right thing
Rule-Utilitarian Analysis
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Buyers knew 97.5% markdown was an e
o
They attempted to take advantage of Amazon.com’s stockholders
They were not acting in “good faith”
Buyers were in the wrong, not Amazon.com
Kantian Analysis
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
8.4 Notable Software System Failures
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Designed as anti-aircraft missile
Used in 1991 Gulf War to intercept Scud missiles
One battery failed to shoot at Scud that killed 28 soldiers
Designed to operate only a few hours at a time
Kept in operation > 100 hours
Tiny truncation e
ors added up
Clock e
or of XXXXXXXXXXseconds → tracking e
or of 687 meters
Patriot Missile
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
(1) The radar system doing a wide area search picks up the Scud missile. (2) The radar system isolates the proposed target. (3) A software e
or causes the system to produce a faulty range gate. The system loses track of the missile, because it does not fly through this gate. (Figure from Science 255:1347. Copyright ©1992 by the American Association for the Advancement of Science. Reprinted with permission.)
Patriot Missile Failure
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Satellite launch vehicle
40 seconds into maiden flight, rocket self-destructed
$500 million of uninsured satellites lost
Statement assigning floating-point value to integer raised exception
Exception not caught and computer crashed
Code reused from Ariane 4
Slower rocket
Smaller values being manipulated
Exception was impossible
Ariane 5
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Significant service disruption
About half of telephone-routing switches crashed
70 million calls not put through
60,000 people lost all service
A T&T lost revenue and credibility
Cause
Single line of code in e
or-recovery procedure
Most switches running same software
Crashes propagated through switching network
A T&T Long-Distance Network
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
(1) A single switch in New York City detects an e
or condition and reboots. When it comes back up, it sends an “OK” message to other switches. (2) Switches in Detroit, St. Louis, and Atlanta are so busy that handling the “OK” message causes them to detect an e
or condition and reboot. When they come back up, they send out “OK” messages to other switches, causing some of them to fail, and so on.
A T&T Long Distance Network Failure
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Mars Climate O
ite
Disintegrated in Martian atmosphere
Lockheed Martin design used English units
Jet Propulsion Lab design used metric units
Mars Polar Lande
Crashed into Martian surface
Engines shut off too soon
False signal from landing gea
Robot Missions to Mars
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
B A E built automated baggage handling system
Problems
Airport designed before automated system chosen
Timeline too short
System complexity exceeded development team’s ability
Results
Added conventional baggage system
16-month delay in opening airport
Cost Denver $1 million a day
Denver International Airport
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
First day of trading for J-Com
Mizuho Securities employee mistakenly enters order to sell 610,00 shares at 1 yen, instead of 1 share at 610,000 yen
Employee ove
ides computer warning
After sell order posted on exchange’s display board, Mizuho tries to cancel order several times; software bug causes attempts to fail
Mizuho loses $225 million buying back shares
Tokyo Stock Exchange
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
After problems with 2000 election, Congress passed Help America Vote Act of 2002
H A V A provided money to states to replace punch card voting systems
Many states used H A V A funds to purchase direct recording electronic (D R E) voting machines
Brazil and India have run national elections using D R E voting machines exclusively
In November 2006 one-third of U S voters used D R E voting machines
Direct-Recording Electronic Voting Machines
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
This Diebold voting machine uses a touch-sensitive screen to capture each voter’s choices. (A P photo/Rogelio Solis)
Direct-Recording Electronic Voting Machine
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
Voting i
egularities
Failure to record votes
Overcounting votes
Misrecording votes
Lack of a paper audit trail
Vulnerability to tampering
Source code a trade secret, can’t be examined
Possibility of widespread fraud through malicious programming
Issues with D R E Voting Machines
Copyright © 2020, 2017, 2015 Pearson Education, Inc. All Rights Reserved
States had second thoughts about D R E voting machines
May 2007: Florida legislature voted to replace D R E voting machines with optimal