These are the notes from a presentation I delivered at the 2nd Message Symposium Against Orthodoxy: Why Publish? back in April of this year.
Children ask great questions. Each present us with a seemingly spontaneous idea — something unconstrained by preconceptions of how the world works.
In The Dog Exercising Machine, a Study of Children as Inventors, Edward de Bono (1971) states that “Children are good at generating ideas…They are not yet aware of how things should be done”. He purports that because children have yet to learn the rules by which the world operates, they are much more likely to think laterally, that is to arrive at an idea without building upon previous solutions.
The tablet computer is different enough from previous forms to provide an opportunity for devising new ‘book’ concepts in the way that a child might. The task however is being dealt with by adults, meaning that solutions are arrived at in a typically linear way. Current results remediate desktop computing design conventions with printed media metaphors. We are left with image and text that requires repeated left, right, up and down swipe gestures, a loose hybrid of the turn of a physical page and the mouse initiated scroll.
This is to be expected. This is how it usually works. Would it be idealistic to hope for anything more?
The following is about unashamed idealism. It is sometimes also about children. In fact, the story of the tablet computer begins with children.
What is a Tablet Computer?
A tablet computer is a lightweight, portable computer that uses a touch screen as it’s primary input mechanism.
The tablet computer was conceived in 1968 by Alan Kay, an American computer scientist who amongst other things invented the concept of overlapping windows and object oriented programming. A child prodigy, he had purportedly read 150 books before he started school and “knew that the teachers were lying to him”. Perhaps because of this, he became interested in the use of computers for education very early in his career. He was influenced in particular by the cognitive learning theories of Jean Piaget that outlines young children as ‘sensorimotor’ learners rather than passive receptors of information.
When working at the Xerox Palo Alto Research Center, and contributing to the invention of the ‘Windows Icons Mouse Pointer’ graphic interface, he delivered a paper titled A Personal Computer for Children of All Ages (1972), in which he essentially describes the iPad, 12 years before the release of the first Apple Mac. The ‘Dynabook’ as it was called, was intended to encourage active learning in children through problem-based interactions performed on a cheap, portable, wirelessly networked flat-panel computer.
Kay acknowledges at the start of his paper that the idea is “science fiction” but that at some point it would become possible. It takes 32 years before his vision is realised and marketed unsuccessfully by Microsoft in 2002 as a tool for business executives. The concept at this point is largely written off by IT analysts.
Apple, capitalising on the powerful brand association with the iPhone and a devoted core customer base, revive the concept of the tablet computer as a ‘family friendly media consumption device’ in 2010 with the launch of the ‘iPad’. This effectively creates a new market that other manufacturers quickly populate with competing devices.
Kay’s original vision closely resembles Apple’s plans for the iPad, to the extent that a current strategic aim is to try and replace textbooks with iPads in classrooms across the world.
Apple has a view of what a tablet should be used for. But what about book and periodical publishers, authors and designers? For them it presents a puzzle – something that could offer a supplement to languishing print sales, but which presents enough of a leap from existing ways of working to cause confusion in terms of how best to achieve this aim.
What is Tablet Publishing?
- A medium, point-of-sale and distribution platform all rolled into one
- Possibility for authors to sell directly to a (global) audience without an intermediary (i.e. a publisher…)
- Something that still requires quality content and effective promotion to be a success
- Android Tablets (including Kindle Fire)
- Microsoft Surface?
- PDF – viewed in ‘reader’ software, fixed layout and orientation, can be designed, easy to copy and share illegally, not naturally suited to screen based presentation, easy to produce.
- eBook– viewed in ‘reader’ software, ‘flowed’ content, orientation aware, can’t be designed, can contain DRM (but possible to crack this), best suited for long-form publications such as novels and textbooks, requires some basic technical skills to produce.
- eBook (fixed layout) — viewed in ‘reader’ software, fixed layout and orientation, can be designed, can contain DRM (but possible to crack this), best suited to simple illustrated publications, can be produced by a number of DTP-like authoring packages, not compatible with all eBook readers.
- eBook (iBook) — proprietorial format exported by Apple’s iBooks author that can contain multimedia content but can only be viewed by the iBooks application on the iPad 2+
- Book App– standalone application, complete control over layout, orientation and design, next to impossible to obtain without payment, best suited to ‘media-rich’ interactive publications, require specialist skills to produce. Will only work on the platform it has been produced for.
- Newsstand App – Apple’s platform for delivering periodicals through an app, that can be paid for via a subscription. The app acts as a shop window, library and viewer of purchased titles. Many Newsstand publications are straight copies of the printed version, with limited interactivity, rather like a PDF. This is presumably due to financial/resourcing constraints. Technically a Newsstand ‘issue’ can do anything that a Book App can do.
Looking at this, it would appear that app-based publications in particular, offer the most interesting potential in terms of commerce and flexibility of function and design.
It is also worth noting that publishers currently are focussing for the majority on producing apps for the iPad platform only. This is because it has the largest market share, and users have a greater tendency to actually pay for apps. Fragmentation on the Android platform also makes the development of high-end titles problematic.
What Isn’t Tablet Publishing?
- The Web – a pool of content, viewed as ‘free’, not necessarily optimised for tablets.
- eReaders – devices designed specifically for the task of reading eBooks. Include the Kindle and Nook (although tablet versions of these are also available).
- A Printed Publication – tactile, has a nice smell, looks good on a shelf, doesn’t run out of batteries or become technically obsolete. Could still be around and functioning in 100 years.
Why Publish (for a Tablet)?
Since the iPad launch in 2012, nearly 60 million iPads and 12 million Android tablets have been sold, with 300 million tablets predicted to be in use by 2015. Apple currently sells more iPads each week than other manufacturers sells PCs and iPads have sold faster than the iPhone when it was launched in 2007. 3.5 billion apps have been downloaded across both platforms, with books being the largest category in the Apple app store, being 19% of total number of available apps.
Tablet publications could be used to target a different demographic, used in conjunction with and in support of other channels (such as print), to promote brand awareness and to cross-sell.
Tablet apps enable you to do some things with a publication that aren’t possible any other way. More of this later on.
The Origins of Tablet Design Conventions
When designing for print, you have a fixed frame (the edges of the paper) within which to work.
When designing digitally for desktop/laptop computers, the frame is indeterminate – dimensions are determined by the proportions and scale of the display, and by the application ‘window’ that can be adjusted by the user.
Tablet apps differ in the sense that the display proportions are known, and the window paradigm is not used, so they are much more akin to a printed ‘page’. They can, however invoke content from beyond the frame using interactive features not possible with a static print.
In Dec 2009 (five months before the release of the first iPad), Swedish media group, Bonnier, publisher of Popular Science, presented a video for Mag+, a tablet magazine concept their R&D department had been working on with London design studio BERG:
What is most interesting about this video is that many of the proposed paradigms for presenting and interacting with the content are very much present in publications being released now, over two years later.
In particular, the choice of arranging articles or sections in a swipe-able horizontal sequence of ‘pages’, which in turn can be scrolled vertically has seen wide adoption. This method combines two major GUI models that have been in use since the introduction of WIMP interfaces in the 1980’s, namely ‘cards’ and ‘scrolling’. Cards have a fixed frame and as content increases, text and image flow into the next card and the total number of cards increases. Scrolling uses a flexibly sized frame, that acts like a ‘window’ to content that extends beyond the viewable area, which can be brought in to view using ‘scrollbars’.
The web has adopted a strong bias towards (vertical) scrolling over card-based or other presentational methods. Oliver Reichenstien offers an explanation for why this might be.
Mag+ and consequent iPad app books tend the favour the left/right card model, combined with vertical scrolling in some instances.
Whether this is the best way to present content in an app book is perhaps open to debate.
Bret Victor, designer of the initial user interface concepts for the iPad, in his article ‘A Brief Rant on the Future of Interaction Design’, argues that focussing primarily on “sliding a finger along a flat surface” fails to exploit the full range of possible interactions that are available within the tablet and future computing devices. Worried that we are travelling down too narrow a tunnel, he advocates a more “ambitious, long range vision”.
The Nielsen Norman Group Studies
Some key themes arise from the two NNG reports on iPad usability:
- App designers should ensure perceived affordances / discoverability
- There is a lack of consistency between apps, lots of ‘wacky’ interaction methods. Designers should draw upon existing conventions (either OS or web) or users won’t know what to do.
These are practical interaction design observations, but from a particular perspective, that of perceptual psychology. These conclusions are arrived at through a linear, rather than lateral process. By giving weight to building upon existing convention, because they are familiar to the user, there is a danger that genuinely new ideas (and the kind of ambition called for by Victor Bret) within tablet design will be suppressed.
Kay’s vision of the Dynabook came from lateral thinking, and thinking about how children learn. Shouldn’t the items that we design for this device be generated in the same way?
Recently an iPhone app Clear received a lot of attention not because of what it was (a ‘to do’ list manager, of which there are dozens) but because it’s interface design went against the Nielsen/Norman stance on providing perceived affordances and relying on convention.
In order to operate the app, Clear requires the user to learn an unfamiliar set of interaction methods, without any clues or as to what these might be. It’s positive reception shows that app design does not necessarily need to rely on prior user interface conventions, like NNG suggest, as long as the interactions are simple, distinct from one another, follow some form of logic, and provide feedback to the user.
The Apple iOS Human Interface Guidelines
These are Apple’s attempt to define how one should design for interactivity within iOS apps. Some of it is technical, some of it ideological. If you don’t adhere to it, your app may get rejected when submitted to the App Store.
One section of note is under the ‘User Experience Guidelines’ heading. Under “Consider Adding Physicality and Realism” and “Delight People With Stunning Graphics”, Apple places an emphasis on Skeuomorphism in the appearance of iOS interface elements. It gives default iOS apps such as the Contacts and Notes apps as examples of how this could be achieved.
The document argues that the use of illusionary surfaces and objects will lead to a more intuitive and pleasurable experience for the user. It also, yet again, looks to prior conventions for solutions rather than starting afresh.
Current Examples and Critique
Current iPad book/magazine publications use a combination of methods borrowed from older digital forms such as the web and smartphone apps, the Mag+ prototype, Nielsen’s recommendations, Apple’s human interface guidelines and some other seemingly unique elements.
Wired was one of the first attempts at an iPad magazine app, and perhaps because of the subject matter, there has been an editorial decision to make the experience as multimedia heavy as possible. Each article begins at least with an animated headline and sometimes an animated sequence too. These tend to grate by the third or fourth viewing.
Within articles, there are sometimes slideshow or carousel elements, which work best for images and don’t seem a good idea for copy text. Sometimes these interfere with the swiping action of moving between pages. The method for operating these elements is also inconsistent. Some adverts use multi-column text which is a problem when the top or bottom of a column is cropped during scrolling.
Landscape orientation is not supported. Each layout is fixed and text can’t be resized or copied. Because of the multimedia content each issue is quite a large download.
This publication is similar in approach to Wired, but with less reliance on multimedia elements. There are some strange interaction choices, such as having some text columns scrollable, and others fixed within a single layout, for example.
A key feature is being able to tap hotspots within a photograph to view descriptive annotations. These are handled a bit clumsily, and the type treatment and position of these is inconsistent, meaning the experience feels at times awkward rather than pleasurable and informative.
The Guardian iPad Edition
The app for the Guardian newspaper was criticised on it’s initial release for being too static, but wisely chooses to focus more on the experience of reading than on overstated interaction methods. Overall, this approach seems a more natural fit within the tablet form whereas Wired and Wallpaper* feel more ‘forced’. The app experience is also distinct from that of reading a printed paper or a website, although isn’t a huge leap from either.
The hierarchy of the grid based layout can get a bit confusing however, doubly so with the two-tiered scrollable menu — what is the difference between the top categories and a ‘section’, for instance? Once buried within an article, it is also hard to get a sense of where you are in relation to other articles or sections.
This independent publication stands out with its use of enigmatic full screen video backgrounds, creating a unique tone that could not be achieved in print. This technique somehow appears more integral to the reading experience than the Wired ‘skip to read’ approach, even though they are essentially doing the same thing.
Navigation once again can be confusing, with different swipe conventions being used across different sections. Some in-page interactions also interfere with the swiping between pages gesture. The use of parallax scrolling pull-quotes works well though.
This is another independent publication that seems better thought out than some of the more mainstream contenders. This employs consistent interaction methods, although swiping image slideshows within a page can cause the entire page to move, and the title page for each article uses the triangle ‘play button’ convention as a static graphic, which is misleading.
What does work well is the combination of video and text in a way that is normally associated with image and text. It seems quite natural to watch a video and casually scroll through some copy relating to it at the same time.
This publication promotes the Exploratorium museum in San Francisco. It contains a series of interactive articles and experiments relating to the perception of colour. What makes it unusual is that is acknowledges the portable and physical nature of a tablet and plays with this, asking users to put drops of water on the screen or hold the device parallel to the floor and swing it about.
Skulls by Simon Winchester
Skulls is by Touchpress, who mostly produce illustrated reference titles in the Dorling Kindersley vein. Their work is celebrated as demonstrating the potential of the medium, as well as being profitable, proving that tablet publishing can have a viable business model.
The book has a vast amount of content and is technically impressive. There are some clever touches, such as utilising the device orientation as a way of switching between a richer level of interaction or more reading-focussed form of presentation.
My main criticism is that it whiffs slightly of late 1990’s CD-ROMs made in Macromedia Director. In other words there is a sense that this is drawing from past models rather than finding truly innovative solutions.
Certain aspects of these examples are successful whereas others seem problematic. The problems arise from either usability errors or an over-reliance on multimedia elements that detract from the reading/viewing experience.
On a larger scale, there seems to be a lack of real invention or exploration of what may be possible. Most examples are only an iteration or two away from pre-existing forms.
The worry here is that as current solutions become established as convention, true innovation within the form may be stifled.
Jessica Helfand in her essay The Dematerialism of Screen Space (2001) critiques the phenomenon of design practise being led by developments in software engineering. She argues that designers should take the initiative:
“design must submit to a series of commands and regulations as rigourous as those that once defined Swiss typography. Aesthetic innovation, if it indeed exists at all, occurs within ridiculously preordained parameters: a new plug-in, a modified code, the capacity to make picture and words ‘flash’ with a mouse in a non-sensical little dance. We are all little filmmakers, directing on a pathetically small screen – yet broadcasting to a potentially infinite audience. This in itself is conflicting (not to mention corrupting), but more importantly, what are we making? What are we inventing? What are we saying that has not been said before?”
Helfand here is referring to the web, but her argument applies equally well to designing tablet publications. Designers of book and magazine apps should be asking themselves those last three questions. Since tablet publishing conventions are in the process of being formed (like child invention), we have a unique opportunity right now to influence their direction.
Approaching The Problem Like A Child
By looking at a tablet as if encountering it for the first time, we might note:
- That it is a small metal and plastic rectangle that contains a panel that shows text and images (still and moving)
- That it can sometimes do things when I touch the panel with my finger
By perhaps trying to generate ideas without any pre-conceived notion of what is or isn’t possible, we might arrive at a number of ideas, such as:
- A publication that contains all it’s content on a single sheet, and is navigated either horizontally or vertically.
- A publication where pages connect and take you in multiple directions, like following paths in a maze.
- A publication where you have to pan and zoom around a large map-like content area (Yes, a ZUI)
- A publication that requires you to rotate the device because different pages are facing in different directions.
These may not all be practical, but at least suggest that there are new avenues that aren’t currently being explored. Can we take this any further?
By exploring aspects of what a tablet can do, we can discover that also:
- The display panel emits coloured light from almost the entirety of one side
- It can react to environmental light levels (using an ambient light sensor)
- It can record and process still and moving images (with inbuilt cameras)
- It can record, process and playback sounds (with an inbuilt microphone and speaker)
- It can work out where I am in the world (by triangulating Wifi hotspots)
- It can send to and receive information from to a vast worldwide network of information
- It can tell what direction it is pointing (using a built in magnetometer)
- It can tell what speed it is travelling (using a built in accelerometer)
- It can tell the direction and rate at which it is being moved within 3D space (using a built in gyroscope)
Developing New Ideas For App Publications
Looking at the device capabilities suggests a categorisation between possible methods that might either process received data to create new material (generative) or respond to a stimulus (responsive). Furthermore, the results could focus on drawing the user into the virtual app space (immersive) or could use the portable nature of tablet to extend the experience into the physical space inhabited by the user (emersive).
- Books that project coloured light and audio into a darkened space
- Books that display abstracted video/audio from cameras/microphone collaged with pre-designed content
- Books that contain location specific content from the internet combined with pre-authored/designed content
- Books that require you to move between environments to experience content
- Books that require you to perform physical tasks and record the results with the camera/microphone
- Books where content adapts in response to movement and orientation of the device rather than touch interaction
By undertaking this simple exercise, it conjures up some intriguing concepts for app publications that aren’t being fully explored. Not all may be workable, but they at least raise a question — is the BERG/MAG+ solution the only way that this can be done? Probably not. Should we also be exploring other possibilities? I think so.
Innovation in current app book design is being constrained by a reliance on convention, resulting in ‘repetitive swipe injury’. By approaching app book design as if one has a limited awareness of prior convention (like a child), a more ambitious, challenging outcome could be reached. Jessica Helfand again:
“…our response has been a reactive one: to technological imperatives, to pragmatic considerations, and to each other. To think beyond these practicalities is to respond to a broader and more compelling challenge: the idea that, as designers, we might begin to tackle the enormous opportunities to be had in staking claim to and shaping a new and unprecedented universe.”