Man-Machine Nightmares: Chaos buttons, human error and healthcare

The film, presented by UCL researcher Dominic Furniss, investigates how everyday and medical device design contributes to human error. Speaker C outlines classifications of error—knowledge‑based, slip, and omission—and argues that training alone cannot eliminate mistakes; design cues and resilient strategies are essential. Real‑world examples illustrate this: an ATM redesign that returns the card before dispensing cash, a syringe‑pump incident where a nurse entered the wrong infusion rate due to confusing numeric entry, and a simulated study of a pump’s arrow buttons that behave inconsistently across numeric ranges. The team then conducts a playful “microwave chaos button” experiment, highlighting how ambiguous interfaces lead to user frustration and unintended outcomes. Throughout, the participants debate whether blame and training are sufficient or whether systematic redesign should be the primary remedy for error‑prone technology.
Chapters
Hi, I'm Dominic and I'm a researcher at UCL. I'm interested in human behaviour and making technology more user-friendly, and I've recently started a project on medical devices, which in part is what the film's going to be about. Ironically, I have to get used to using this complicated bit of kit, the camera, but hopefully it will be alright. And, well, I hope you enjoy the film.
Pff, pff, pff.
The type of research that I'm involved in is understanding human error. Now, there's several different classifications of human error. There's error that's associated with a lack of knowledge. A user of a particular device, if we use the example that you've got a new mobile phone, you might not know all the functions, so maybe you know you might want to place a phone call, for example, but you don't know how to do that on a particular device. So there you can make an error when using devices that you're unfamiliar with, and these errors are often associated with a lack of knowledge. Slip errors are an everyday occurrence. They can be associated with everyday routine. For example, you might watch a weather forecast and realize that you should take your umbrella with you, but on the way out of the house, for example, you might realize that you've forgotten your umbrella. You have the knowledge that you needed your umbrella for a particular day, but for some reason you've forgotten it, and that, that's an error, and that can be down to a variety of reasons. But of course, people develop strategies to avoid these types of errors. I mean, a classic example is that if you know you want to take your umbrella with you, you put it in a position when you leave your house that you're going to walk past, so you'll see it and remember it as a cue to memory. I mean, so this type of cue is almost— can be conceptualized specialised as a resilient strategy that individuals can develop to remain guarded over making particular everyday errors. Training is important for the avoidance of knowledge-based errors. If a user simply doesn't know how to use a device, then they're more likely to make errors if they've got insufficient training. However, there's a whole set of human errors, including errors where of omission, where people forget a particular task step. They might have done that particular routine a number of times, but due to a circumstance such as an interruption or just a lack of attention, an error might occur. So we have to think about how do we safeguard against those types of errors, and one way of doing that is through device design. Now with device design, some of the modifications can be quite trivial, but as long as they support what a user's thinking and their intentions, then the likelihood of error could be minimized. A classic example is using an ATM machine, a cash machine. Before they were redesigned, users would often walk away from the machine without their card. So cash machines have been redesigned so that they give the card before dispensing the money. So a simple modification like that can actually avoid human error.
In hospitals, infusion pumps and syringe drivers are widely used to pump chemicals into people after accidents, operations, and for treatment. So it is important that device design is right to reduce the likelihood of error. I heard of a case where a nurse entered 15 mL per hour instead of 1.5 mL per hour. People on our project think this was influenced by bad design. They suspect the nurse pressed a 1, then a decimal, then accidentally pressed another decimal. To delete this, she pressed the delete key, which strangely deleted both decimals. And then she pressed the 5. The nurse was sacked for not checking this figure. Reflecting on this, I went down to Swansea to find out more about this sort of work.
Recently I've been looking at number entry in medical devices, and I've simulated one of the number entry systems seen in a syringe pump. So there are 4 arrows that are used to increase and decrease numbers in this particular number our interest system, and this 2 I use to increase the number, and this 2 I use to decrease the number. I think it is quite reasonable to assume that these 2 buttons, the small up chevron and the small down chevron, are sort of inverses of each other. However, this is not always the case in some of the values. So we have peculiar cases when you're increasing or changing the number between 9 and 10, and when you're changing the number between 90 and 100. So for instance, if we go up to 9, we're increasing by 0.1. If we try to decrease, it decreases by 0.3 there. If we go right up, well, 9.7, we're still increasing by 0.1, but suddenly we decrease by 0.8. I think this is very peculiar behavior. I don't think it was planned by the manufacturer, but it is the sort of thing we've discovered. I think it'd be interesting to do an experiment to find out how much people look at the screen. So for instance, I think personally that this particular user interface forces you to look at the screen. That might be one good thing about it, because you don't know, you're really not sure what is going to happen when you press any of these buttons.
Hello, I'm Rachel and this is our microwave oven.
So my name is Chris, I'm a PhD student at UCL, and today I'm going to be using the microwave.
Okay, so my name is Masha and I'm a PhD student and I'm going to cook today. The big microwave experiment.
Yes, I've been living here for about 5 years now and wouldn't say I use it every day, but use it quite a lot, so I'd like to think I know what to do with it.
Racers ready? Microwaves ready? 3, 2, 1, go!
Put the microwave on full power for 3 minutes, 3 and a half minutes.
Was it running on full power?
It's on high.
Is it done?
3 and a half.
Okay, so this is going to be interesting whether it actually does chaos.
It has a chaos button.
Um, okay, stop.
We're going to shoot its full power.
Oh, I could just start.
Oh, that's about 3 and a half minutes.
Let me just start. Oh, that doesn't seem to do anything. Okay, well, 3 and a half minutes, I can probably do that. So we have 3. Oh, that was 10 minutes.
Stop.
Okay, so that's 3 and a half. And the power, press start. Okay, high.
Okay, good.
I think that was it. Okay, maybe we should stop it because it's gonna—
ah!
Oh no! Oh, that's not good. Can you hold that and stop? Oh, that's really bad. There's nothing in there. Should we turn it off?
I think we broke it. No, it's fine.
There's nothing in it.
But it does.
My sister's microwave once exploded.
Okay, this is the frozen chicken already.
9 chickens.
See, this is—
to defrost, you actually need to put on the lowest setting. Press that.
Is that— oh, grab something.
Wait, there's something on the screen as well.
I don't know, but okay.
Over there, grab this.
Okay, I think I should do it on 40%.
Oh, well, I guess now this doesn't work.
Well, this It seems clear that device design could make a real difference in performance, especially where the design seems strange. Chaos!
It has a chaos button!
So if human errors seem to occur so naturally, is it right that we should just resort to blame and train, or should we think more about system design?










