One of the great unexpected twists from the classic movie, The Wizard of Oz, as a child was the revealing of the Wizard not being some magnificent god. Instead he was just a plumped, normal human. This deceiving act is the inspiration of an experimental set up used in human-computer interaction research: A Wizard of Oz Experiment or WoZ for short.
A difficult aspect in this field’s research is the time-intensive and expensive nature of creating and programming new software and algorithms. Many researchers in the field start with a hypothesis such as: a virtual assistant that can develop rapport with a human is more effective in aiding humans with their tasks than a a virtual assistant that does not develop rapport with a human such as Siri, Cortana, Google Now etc. To best verify this hypothesis, a virtual assistant would be built that can develop rapport. Then this rapport-building virtual assistant would be tested on humans to see if it more beneficial compared to a current virtual assistant by using a measured and controlled test.
This process seems straight forward until lets say that the tests reveal that the rapport building virtual assistant is not more effective in aiding humans with tasks than the normal virtual assistant. Now all this time, energy, and money has gone to waste into developing this brand new, never done before virtual assistance that is not more useful than older technology.
This is why WoZs are use! A WoZ experiment starts with telling a human that a computer program is fully autonomous while in reality a human in another room is controlling what the computer is doing. So in this example, instead of creating a automated rapport and dialogue selection system, all that is created is a user interface that a human called a Wizard can control to make it seem that this virtual assistant that is developing rapport is fully autonomous. This way is much cheaper and quicker than the prior description. If the hypothesis gets verified, then the actual automated computer software is created. A WoZ can be used throughout the design process by slowly getting rid of the scope of control that the Wizard has over the virtual assistant until the Wizard is no longer needed!
It is such a simple concept, but the benefits of it are huge! Instead of starting right away building technology that might not be useful, a person first tests a human (Wizard) controlled scenario that mimics what the computer would do without the subject knowing. If it proves effective, then the actual, autonomous software is programmed and created!
My main job at ArticuLab has been researching past and present WoZ design philosophy/systems and creating a WoZ interface to use on a lab project. Having the ability to learn and experiment with different techniques such as utilizing a WoZ has really helped me understand the wide variety of ways researchers conduct their research.
*** ArticuLab’s work on this hypothesis can be found here: SARA