Kris Henning

The research project focused on developing 3D form in an iterative loop of experiments, mediating between physical objects and digital space. The project investigated a sequence of experiments exploring the concept of linkage and slippage or control and serendipity, occurring at the boundaries of converging technologies. The experiments tested the use of 3D scanning, digital fabrication techniques (specifically 3D printing and laser cutting) and virtual immersion within stereoscopic environments. It also tested the concept of mediating between remote studios with new forms of hybrid designing and real-time online collaboration as a means of generating innovative and unexpected design outcomes – real and virtual, tangible and intangible.

3D Scanning, Laser-cutting and 3D Printing Experiments

3D Stereoscopic Environments and Digital Fabrication

Final Design

This was a Project I initiated throughout the 2007 year, officially kicking off in October. I presented the idea of researching the School of Design’s newly acquired 3D scanner to the Head of School Simon Fraser. At the time I was working with HitlabNZ, and we agreed it would be a good opportunity to develop the relationship between the two Schools and incorporate the use of the Design School’s also newly acquired 3D stereoscopic environment. I co-wrote the proposal in which we received $50,000 as a grant from the Victoria University Research Fund. The work led to the acceptance in two conferences (Siggraph Asia, and VRAP Portugal), and formed the basis of my future masters research work.

The work completed under the project was presented at two conferences and is shown below.

New Perspectives On Form :

Real-time, virtual modeling in networked 3D environments.

(Chinese Whispers: mediating oscillation between physical form and digital space)

Victoria University School of Design in Wellington in collaboration with HitLabNZ at Canterbury University in Christchurch. Conference Paper Presented at SIGGRAPH Asia in 2008 and developed further for presentation at VRAP Portugal in 2009.

From VUW: Simon Fraser , Kris Henning , Tim Miller , Morgan Barnard , Marcia Lyons , Johann Nortje
From HitlabNZ: Mark Billinghurst , Julian Looser , Hartmut Siechter , Michael Groufsky

Abstract:

This project investigates new approaches to developing complex 3D form through an iterative loop of experiments mediating between physical objects and digital space. It explores the concept of slippage occurring at the boundaries of converging technologies as a means of generating innovative and unexpected design outcomes, real and virtual, tangible and intangible. The sequence of experiments involved linking digital technologies that are not immediately compatible in remote networked environments. It brought together the ‘physical’ digital technologies of 3D scanning and digital making with the ‘virtual’ technologies of immersive stereoscopic environments and video conferencing in two separate locations. The paper describes this process and the outcomes, details the standard and custom developed hard- and soft-wares, and frames the project in a context of ‘play’ as a means of facilitating cross disciplinary collaboration and user-generated design.

Introduction

In this paper we describe the New Perspectives on Form project and its theoretical context. First we discuss the approach we followed to interpret the physical into a virtual form. Next, we present the experience of the participants in manipulating form in the virtual design space or ‘playing field’. Thirdly, we describe the technology developed to provide the experience and the continuous development of programming enabled process. We end with directions for future work and conclusions.

The project brought together interaction designers with industrial designers and computer scientists in two different geographic locations. It also brought together the ‘physical’ digital technologies of scanning and rapid prototyping specific to the VUW Industrial Design Program with the ‘virtual’ technologies of a 3D immersive environment developed by the HIT Lab NZ and shared at a distance as a parallel environment with the VUW Digital Media Design Program. The goal of this research was to explore how new physical digital technologies can be combined with collaborative virtual environments to investigate new approaches to manipulating form as a unique remote design experience.

In the design context, error or chance can be used as a source of opportunity. The notion is not new to design as a human centered discipline, and has its roots in the fundamental human fascination with the ‘flaw’ as a counterpoint to perfection [Kraus 1976]. This observation is equally applicable to design outcomes and the design process itself where unforeseen interventions can inform, influence and enrich the process and ultimately the outcome in new and unexpected ways.

In the spirit of chance, the inspiration for the project was opportunistic from the start; to attempt an ad hoc connection of a variety of digital technologies to investigate new forms of real time remote collaboration in a design setting.

“Linkage and slippage”

Unforeseen interventions are a theme that the design critic and commentator David Pye also discusses in relation to Industrial Design [Pye 1968]. Pye rejects the monotony of mass production and makes an appeal for greater diversity in our built environment through a process he calls the “Workmanship of Risk” where “The essential idea is that the quality of the result is continually at risk during the process of making…..”[Pye 1968]

While Pye was referring to analog methods of making (handcraft as opposed to making by machine) ironically the answer may well lie in the technology that he questions. More specifically in digital technologies which make it possible to reintroduce the concept of “Workmanship of Risk” by enabling us to engage ever greater levels of complexity, personalized and collaboration in the process of designing and making; thereby potentially achieving the richness, variety and diversity Pye suggests.

The vehicle of exploration is fundamental to design; 3D form and space, with specific reference to a new understanding as to how it can be developed, modified, manipulated, communicated and perceived in both physical and virtual realms. The not always seamless interfacing between a complexity of hard and software, along with the opportunity for, and disorientation of online real time immersive intervention or collaboration in the design process cast new light on Pye’s concept of ‘Workmanship of Risk’ and the opportunity to engage the unexpected in design. This process is demonstrated in a series of tests.

The first sequence of tests mediates between physical form and digital space. The purpose of these experiments was to scan a physical object using a Next Engine 3D Scanner [NextEngine 2008] in preparation for importing it into the Vision Space 3D immersive environment. They demonstrate the notion of digital slippage in the form of off-cuts that are all too easily discarded as flawed because they failed to meet a preconceived outcome and yet reveal unexpected design opportunities. See figure 1.

The quality of the scan is dependant on the material and/or surface quality of the scanned object. The same object with alternatively opaque, transparent or reflective surfaces will render very different results when scanned.

This phenomenon is explained by Schnabel et al. [Schnabel 2004] “Scanning is not a faithful replication of an object but a reinterpretation. Errors and occlusions are introduced that do not exist in the physical form.”  Whereupon their students are encouraged to “make use of the ‘shortcomings’ to create new forms.”

Figure 1. Scanned sheet of wire mesh

The scan initiates a transformation from an object with substance to a tessellated surface without thickness. This represents the first step in facilitating a shift in perception or alienation of the form allowing multiple readings from object to space and in between, particularly when the surface has been misread by the scanner and perforated with voids.

A further shift may occur when the digitally rendered scan results in an irregular simplification of the original forms by reinterpreting them as mediated surfaces. The complex mesh generated by the scan proved problematic. High resolution scans caused the server to crash. Rather than perceiving this as a constraint it was resolved by scanning a generic abstract form or proxy (a sphere) at low vertices count. See figure 2.

Figure 2. Scanned proxy highly simplified

While the outcomes of some of these ‘imperfect’ scans were triggers for design research in physical form, the scanning and prototyping created a procedural ‘framework’ of risk to lead into the full collaborative potential of digital exploration, and the scanned proxy with subsequent irregularities was imported into the remote, real time interactive and immersive environment.

“Framing the Error”: collaborative play in digital space

Importing the data into this environment immediately opened it up for new interpretations and interactions; it shifted the traditional industrial design focus on the 3D object and framed it within the interaction design concept of ‘play’.

This shift had particular significance for the project:“…psycho-sociological frames have more recently repositioned play as a force for change where participants can imagine alternatives, create possibilities and experience the loss of self through ‘a flow state’ where the activity has no extrinsic reward beyond its own existence. Play is, in itself, a reframing activity, a state of mind which allows us to access the intuitive and to envisage the (im)possible. As such, play permeates performance and design practice at all stages – conceptualization, development and realization all depend on what Sutton-Smith calls ‘the potentiality of adaptive variability’ offered by this affective state of mind.” [Bayliss, A., Mckinney, J., 2007]

The unexpected experience of the aesthetic quality of space and form as ‘play’ within the stereoscopic immersive environment, between the virtual object and the object as a vertigo spatial experience was profound; that getting lost in the act of designing, in this environment with  this intensity, suggested new and very different ways of designing.

Several experiments were conducted to see what results could be accomplished as multiple designers in remote locations pushed and pulled points of a single digital model in a virtual tug of war. Knowing how to design ‘too well’ with an expected result based on a set of fixed constraints within that substructure is challenged in these experiments. Assuming that ‘all play has meaning’, this active principle is as illusory as much in a virtual studio ‘space’ as it is in any play activity where a variety of possible outcomes exist [Huizinga 1971].

The result of the experiments was not to model a perfectly designed conclusion, but to capture the essence of the collaborative intention resulting in a collection of digital forms. Individual methods and disciplinary trajectories soon began to trigger interesting control issues. The participants competitively responded to a set of constraints and negotiated discipline compromises resulting in an emergent design experience, effectively leveling the playing field. The errors and flaws in this networked co-laboratory were guiding factors in the design output that forced situations where perfection had to be ignored.

In five minute experiments, the three disciplines (players) in two locations (remote-local) met in a stereoscopic immersive environment (as an interstitial space or playing field) and collaboratively engaged in a hybrid design process. The following images and notation describe this process. All players modified the same scanned proxy within their own remote studios to complete one round of ‘game play.’ See figure 3.

Figure 3. Collaborating in Immersive Stereo Environments
VisionSpace at HitLab NZ  and Victoria University Media Lab.

Collaborative test in digital space: 5 minute timed experiments

Communication Local/Remote

All players communicated both locally and remotely.
Error 1: An overload of vertices caused local/remote systems to crash.
Error 2: Orientation of local/remote designers to projected displays caused disorientation resulting in confusion.
Error 3: Crash at either end causes DESIGNER DOMINANCE
Coop 1: Both local and remote DOMINANT DESIGNERS instructed critical reflection.

Non-Cooperative / Local Communication

All players resisted relinquishing their design intent.
Error 1: “What part of the model is our design?”
Error 2: TUG OF WAR –Competitive control of vertices.
Coop 1: “Let’s See IT” Both local and remote begin to examine the design experience and process the outcome. Within the fissures of the form itself the research element of design discovery begins to reveal more interesting- NEXT MOVES
Error 3: Undo Option not working-process malfunction.
Coop 2: Key frame function ‘snap back’ working: MACHINE MOVES: Meaning there is a meta-controller, the server functions and backend designers respond.

Local_ Remote to Remote_ Local

Each designer directed ‘players’ of the opposing team creating a remote ‘doubles’ match.
Error 1: Telematic mashup: Confusion in communication between remote studios.
Coop 1: ‘Game Play’ Competitive Design “He stole my point”
Error 2: “Is that you? Or is that us?” Locative struggle of similar vertices control.
Error 3: System overload: Lagging! Yelling! Sweet!
Coop 2: Spin it! Local Global GAME SPENT

The Coffee Break- 10 minute local advantage

Remote team went for coffee break resulting in total local design dominance.
Error 1: Uneven time designing; one ‘player’ game.
Error 2: One sided game no communication.

Design From Within: STEALTH Designer Mode

The object became space; navigating within the object.
Error 1: Uneven time designing; no communicating verbally; dislocation between object and space.
Coop 2: ‘Being’ the space was a more holistic way to interpret form. Sliding inside the form and outside the form begins another context of “object into space”.
Hint 1: Position tracker ‘weighted’ location for designer to identify vertices from within.
Error 2: Designer Vertigo; remote triggered uncontrollable object spin causing local disorientation and physical sensations.

Test Conclusions

Despite high speed networks the manipulation of form in real time from remote locations imposed a low resolution polygon visual language on the objects/spaces generated.

The immersive environment changed the design experience by liberating the participant from the constraints of scale. Instead of concentrating on design iterations of an object the exploration shifts seamlessly from out- to in-side, between object and space.

Liberated from the laws of gravity or mechanical constraints, otherwise static structures become dynamic
Unexpected design forms could be printed at any point in the process offering further new perspectives on the outcomes.

Errors create an opportunity for documenting the design process and gesture; reactive and responsive, local and remote abstractions of space and form trigger another way of looking at the design process.

Game Play strategy; (tennis metaphor) Ad-vantage local or remote; became the strategy narrative for a competitive networking designer dominance and demanded a rapid response in the form of spontaneous and intuitive decision making.

In-space conceptualized the form into another environmental architecture that took the design process into the fantastic; installation and interaction design through sensor triggers create an out of the immersive space experience for an audience, a sense of design performance rather than design process.

Discipline Trajectories: Hitting ‘Print’/Future Work

The technological foundations enabling different disciplines to experience and control a formal investigation of form and space in real time, each from their own very different perspective and interpretation. We describe it from one perspective:

Industrial Design

The necessity for us to work in low resolution comes at a point in time where the manipulation of complex fluid forms in high resolution has never been better. However our initial misgivings with this perceived compromise proved to be unfounded if viewed within the context of contemporary Industrial Design form language where ironically, despite the availability of sophisticated high resolution modeling software, the fascination with low resolution tessellation and the polygon as a formal motif is finding increasing interest. This has parallels in other design disciplines such as the fascination with pixilation in communication design.

We did not have to look far to find immediate analogies to our early low resolution scans in the form of other design experiments or even manufactured products, or to find other manifestations of polygons as structural elements. See figure 4.

Figure 4. Julian Mayor ‘General Dynamic’ 2004
‘Sinterchair®’ Oliver Vogt and Hermann Weizenegger 2002

However, working between 3D scanning, stereo environments and physical prototyping also offers industrial design the unique opportunity to explore dynamic forms/spaces with real-time modeling processes. Our initial experimentation of scanned objects outputted highly complex surface geometries with densely triangulated curvatures. The forced error or ‘slippage’ of manipulating low resolution polygons in stereo also resulted in simpler planar faces that lent themselves to the notion of ‘folding’ as a modeling process to generate transformative surfaces.

Figure 5. ‘Transformer’ interactive object/ inhabitable space. (Linkage and Slippage)

The ‘Transformer’ (Figure 5) is an ambiguous multipurpose object that gives physical substance and a fixed scale to the virtual model. Modeling the shape of the object virtually involved the manipulation of each of the individual vertices to generate surfaces that could be unfolded and printed for reassembly. Giving physical substance and a fixed scale to the digital print through rapid prototyping at a micro scale or laser-cutting at a macro scale opens up the output to an interactive audience and invites new interpretations as to orientation and possible applications.

The production of the virtual geometries as physical models can be laser cut from planar materials and re assembled. Changes in the procedure between the virtual and physical, suggests a new reading of the form, and offers the opportunity to develop refined folding geometries. These processes can be used to develop and give direction to the development of each of the reconfigurable components as well as the overall design. This paradigm of translation, reconfiguration and reinterpretation has been used in related domains like digital architectural design. [Schnabel 2003]

Back End Design

The experiments used custom developed software that enabled participants to remotely view and edit 3D virtual objects in both desktop and immersive modeling interfaces. There were three main pieces of software involved:
a server application for maintaining model consistency
the VisonSpace immersive stereo-projection application
the Blender modeling application.

In this section we describe each of these technologies in more detail.

Verse Server Application

For shared 3D modeling applications it is important to maintain model consistency.  To achieve this we use the Verse server which is a piece of software that works as a hub, letting multiple graphical clients connect using the Verse protocol, upload 3D model data, and subscribe to changes that are made to the virtual models hosted on the server [Verse 2008]. In this way Verse is a framework for sharing geometry, contextual information and supporting avatar representations. The two client applications described next, VisionSpace and Blender, use Verse as a common language with which to communicate while providing fundamentally different user experiences.

VisionSpace

VisionSpace is the name of the immersive visualization installation built at the HIT Lab NZ. A similar satellite system running the same software was built at Victoria University of Wellington. These two systems were indirectly linked via simultaneous connections to a common Verse server.

VisionSpace is comprised of hardware and software that provide multi-sensor 6DOF tracking, rendering and interaction. Three large (8 foot by 6 foot) screens cover up to 180 degrees of the user’s view. Behind each screen is a computer and pair of projectors that display stereo images that the user views through passive polarized glasses. A wireless tracking system tracks the user’s head and hand pose. The general setup is similar to a CAVE [Cruz-Neira 1993] or a Responsive Workbench [Froehlich 1995], in which the tracking supports one spatially registered viewer and one input device in combination with a projected stereo view.

For user interaction we use a wireless handheld input device. This input device appears much like a standard joystick, although rather than sitting firmly on a surface, it has a tracking constellation mounted on top, and is manipulated freely in space. When using this tool, the user can literally reach out and directly manipulate the virtual model shown on the screens.

Custom software was developed for the VisionSpace environment to enable multiple users to remotely view and manipulate 3D forms. The software communicates with a Verse server to display and manipulate geometry, coordinates the generation of stereo views between three workstations, collects head and hand tracking data, and provides a plug-in architecture for rapidly exploring new prototype applications.

Figure 7. Collaborative schematic

VisionSpace software utilizes the Verse protocol to subscribe to and manipulate 3D content stored on the server. This allows the immersive clients to be equally enabled as the desktop 3D modeling applications, described next. See figure 7.

Blender

Blender [BLENDER 2008] is an open-source desktop 3D modeling application that is commonly used by the design students at the Victoria University of Wellington. The Verse and Blender projects are tightly related. The source code of Blender provides a plug-in which synchronizes data with a Verse server as the user creates or edits models in Blender. In this way the students can use their conventional desktop modeling tool to collaborate in real time with each other and remote users in the immersive 3D theatre.

Conclusions

Schnabel et. al. [Schnabel 2001] reports on the first successful attempt to conduct a Joint Design Studio, which uses Virtual Environment (VE) as tool of design and communication between the remote partners. Until this time shared immersive spaces had been used for design review [Davidson 1996], but not for active joint design sessions which involved content creation. In these cases the interfaces were designed for shared model viewing and manipulation, but not low level editing and design critique.

New Perspectives on Form has a number of unique features. Unlike earlier work in this area this constellation of software supports shared polygon level editing on a remote network which enables the designer greater freedom in creation and enhanced collaborative methods of learning. In particular it combines 3D scanning, rapid prototyping, a shared desktop modeling tool, and immersive stereoscopic environments and integrated video conferencing as a critique tool. In this way, the participants can use familiar interfaces for collaborative design exploration together with a virtual reality environment. Unlike earlier applications, a methodology of play theory applied to design processes created a sense of leveling the playing field for the participants in an activated studio environment.

We can see that there is a compelling argument for exploring the new essence of physical and virtual networked vocabularies. Collaborative roles of designers and other disciplines in our network focused realm created unexpected connections and expanded boundaries cross-pollinating disciplines in our physical and increasingly
virtual world.

Experiments extensively investigating 3D scanning.

References

Bayliss, A. & Mckinney, J., 2007 Emergent Objects: Design   and Performance Research Cluster. In Designing for the 21st Century. Interdisciplinary Questions and Insights. Gower December 2007. pp.150-165 Edited by Tom Inns, University of Dundee, UK

Blender Website: http://blender.org/ Accessed June 2008

Cruz-Neira, C, & Sandin, D.J. & Defanti, T.A., 1993, Surround-screen projection-based virtual reality: the design and implementation of the CAVE, International Conference on Computer Graphics and Interactive Techniques, pp. 135 – 142.

Davidson, James N. and Campbell, Dace A.: 1996, Collaborative Design in Virtual Space – GreenSpace II: A Shared Environment for Architectural Design Review, in Design Computation: Collaboration, Reasoning, Pedagogy, ACADIA Conference Proceedings, Tucson, USA, 31/10–2/11/96, pp. 165-179

Huizinga, J. Homo Ludens. Beacon Press, 1971

Kraus, K. 1976. Half-truths & one-and-a-half truths: selected aphorisms. Montreal : Engendra Press,

Krueger, W., Bohn, C.A., Froehlich, B., Schlueth, H., Strauss, W. & Wesche, G. 1995, The Responsive Workbench: A Virtual Work Environment, Computer, 28(7), pp 42-48.

Nextengine Next Engine company website:
https://www.nextengine.com/ Accessed June 2008

Pye, D. 1968 The Nature and Art of Workmanship. London, Cambridge U.P., ISBN: 0521060168

Schnabel, M.A, Kuan, S.S. & Li, W. 2004. 3D Scanning, Digital Modelling, Rapid Prototyping and Physical Depiction.  Department of Architecture, The University of Hong Kong, Hong Kong; P.R. China

Schnabel, M.A., Kvan, T., Kruijff, E. & Donath, D. 2001 The First Virtual Environment Design Studio, Architectural Information Management, 19th eCAADe Conference Proceedings, Helsinki, Finland, 29-31 August, pp. 394-400.

Schnabel, M.A., 2003, Creation and Translation, Proceedings 8th CAADRIA Conference, Rangsit University Thailand, 371-383.

Verse Website: http://verse.blender.org/ Accessed June 2008

(C) 3D Transformations
marcaurel@hku.hk  kskuan@hkusua.hku.hk wdli@hkusua.hku.hk

IMAGE CREDITS:
Figure 4. Julian Mayor  2004 ‘General Dynamic’ (Left) source:     http://www.julianmayor.com (Copyright being sought)
Oliver Vogt and Hermann Weizenegger 2002  ‘Sinterchair®’     (Right) source: http://www.vogtweizenegger.de (Copyright being     sought)