The Future of Creative Applications

After holding out a year and a half, I just bought my first iPad.

I remember when the iPad came out. My initial reaction was different than most people. My thinking was “what good is it?”. Yeah. It was cool, but it didn’t do anything special. I remember talking to my friend David Blatner at the CS Developer conference in Seattle last year soon after the iPad came out. He showed me his iPad and I asked him what’s so great about it. “What do you mean? It’s great! The kids love it! You have pictures, videos…” I just didn’t get it. I don’t need toys. What does it do for me?

What was I missing that everyone else seemed to get?

The Scroll Direction

Seriously.

What??? Yep. Of course the scroll direction is not that significant in it of itself. In fact the whole iPad on its own does not have a lot of value in my eyes. What is truly valuable is the iPad as a platform. What the iPad really represents is a concept much larger than itself. It’s a concept that transforms the essence of what computers are and will be in the future.

Computers have advanced tremendously in the last twenty years. The capabilities today were not imaginable then. But, the basic way we interact with computers has not changed much since Apple popularized mice. Yeah. Wacoms are nice, but the interaction with computers has been by definition clumsy. There’s a level of indirection built in to the very interface. You don’t interact with things on the computer. You tell the computer to interact with them. Case in point: The scroll direction. Why do mice scroll down a web page to display lower content? Because what’s really going on is you are telling the computer “move my view down”. We’re so used to this idea that we don’t even question it.

The iPad changes all that. Instead of telling the computer to move the view down, you grab the content and move it up.

Without reflection, you don’t realize how significant this point really is.

Classic computing has all been about making it easier to communicate with the computer what you want to do. The iPad is all about eliminating the computer from the experience. It gives the illusion of dealing with the content directly. You don’t need to “tell” the computer anything. You just do it yourself.

Of course this is all about touch. In the real world, when you want to do something, you just do it with your hands (or with tools — more about that later). The more indirection when doing something, the more difficult the task becomes. Think about the carnival game you might have played as a kid. You know, the one where you look in a mirror and try to walk along a curvy line. Walking on a line is not hard, but doing it while looking through a mirror becomes challenging. That’s more or less what we are doing in our design programs. We look at a display and then manipulate objects using a different device and at a different spacial orientation no less! It’s testimony to the human mind that we can do any precision work like this, and we don’t even think about it any more!

On the fundamental level, touch changes all this. Using touch, we interact with what we are editing directly without any indirection. We manipulate objects the way we would in the real world. This is an extremely powerful concept!

Of course when the iPad came out, this was only a concept. There were no valuable practical applications. You can also argue that the iPad is not even the right form factor for efficient use of creative applications.

But, I think people sensed this change in the computing paradigm even if not consciously. That, I believe is what was so compelling about the platform. The iPad shipped with nice applications which gave the feeling for the paradigm, but as in all platforms, it’s the job of the application developers to really harness the potential of the platform.

Now, I don’t believe that the story is really about iPads or tablet devices vs. conventional computers. I think this is a shift from the conventional computing paradigm as a whole. This shift started with tablets, but it will unquestionably make its way back to desktop computers. Desktop computers in 5 years from now will probably look different than the ones of today. Make no mistake, touch will not remain in the small form factor of a tablet.

This shift is already happening. Case in point: Lion changes the scroll direction in the Mac OS. This change was not by mistake. Apple is making a conscious effort to move the new paradigm to the desktop because is a much more natural mode of computing. I’m quite sure that Apple is working on new designs for desktop and laptop computers to enable this shift ergonomically. I’m pretty sure Adobe is thinking the same way as is evident in the comment by Kevin Lynch in this article by the New York Times. “It’s the Future”. They are clearly thinking beyond simple tablet devices. A 50 inch drafting table is definitely not a tablet device!

Okay. So we are in a transition period to the future of computing. One no less significant than the mouse of the first Macintosh computers of 20 years ago. What about applications?

Well, I think Adobe made that very clear at MAX this year. They are rethinking the way designers interact with creative applications on a fundamental level. They showed some concrete results in the form of their touch apps. That’s a great start, but I think it’s very clear that it’s only a start. These touch apps are apps in their infancy on a platform paradigm which is also in its infancy. As the paradigm makes it back to the desktop, I think we are going to see some really interesting developments.

Here’s some ideas floating around my head:

Photoshop

I’m going to start with Photoshop because Adobe has already shown some work in that area.

Expect to see huge drafting tables. They are coming. It’s only a matter of time.

After the showing of Photoshop Touch, I heard some comments about how difficult it is to create using your fingers. It seems to me that such a comment is short-sighted. Yes. Of course fingers are imprecise tools. That’s why artists do not use fingerprinting to create masterpieces. People have designed tools to allow precision work beyond what’s possible using your hands alone.

Yes. Paintbrushes allow for much more precise work than finger-painting. You’d have to be a fool to say that a canvas is not a good medium for painting because the results of finger-painting are less than impressive. The answer is to buy good brushes.

For the touch interface to become really powerful, we are going to need the tools to enable that. We are going to get back to the basics where artists keep at their disposal physical brushes and other tools where each one allows for creation of different tpyes of art. Painting in Photoshop with real brushes directly on the screen which reacts using pressure sensitivity. That sounds really exciting, and I believe it’s coming!

Beyond painting, I believe touch will allow for many other improvements such as improved selection tools. Imagine soft touch painting of selections. We have yet to become accustomed to this new way of thinking, and I’m sure as it sinks in, there will be many new developments.

Of course there’s manipulation of objects as well. Warping and other filters are a much more natural fit using touch than the classic UI. I’m sure the list of improvements will go on and on. We have exciting times ahead of us!

Illustrator

If I had a dollar for each time I’ve heard people complain about the pen tool in Illustrator, I’d be a rich man. But, I think my friend Mordy Golding has the right way of looking at the issue: We don’t need to fix the pen tool. We need a replacement. One that uses a totally different way of working with paths. To drive the point home, I can’t help but point out that Mordy just published an entire course on drawing without the pen tool!

There’s really no reason that we need to move anchor points and control points to work with paths. Yes. Moving these points allow for very precise control, but there has to be a better way to go about controlling the shape of paths.

In other words, the paradigm used for the pen tool is broken. Any improvements you can make to the pen tool is just attempting to fix a broken paradigm. I don’t think Adobe is to blame for this broken paradigm. Moving anchor points and control points is about the best you can do using our computer interface. Yes. There can probably be small improvements to make using the tools a little easier, but fixing it fundamentally is a serious problem.

I think that touch might just be that new paradigm that we are looking for. Think about manipulating objects is real life. Can you imagine trying to shape anything using one finger? Here’s an exercise: Take a string and place it in a straight line on your table. Now use one finger and try to shape it into a spiral. You push on one point, and another point goes out of whack. Even if you pin down one end, it’s almost impossible! Well, that’s sort of what we are trying to do using a mouse or Wacom. The fact that we are able to shape things as well as we can using a single point of interaction is actually quite amazing!

This problem is very similar to the mouse vs. touch interface problem. Dealing with anchor points and control handles is a level of indirection in our work. In real life there are no anchor points and there are no control handles. Making us manipulate anchor points and control panels adds a level of indirection to the object manipulation which does not exist in the real world. We enter the world of computers and tell the computer: “I know you only understand anchor points and control panels, so do me a favor and move these objects around to give me the results I need.” In other words, to work you need to work in computer terms rather than vice versa.

With touch, this all changes. We will now have the full use of both of our hands to be able to shape objects. This will give us a level of control that we can’t even think about today. Control handles can easily become a thing of the past. We can control the shape directly, and it will be the job of the computer to store that data in a way that the computer finds useful. The computers will allow us to work in our terms rather than keeping us bondage to thinking like a computer. Empowering!

What else? Think about almost any pain point in Illustrator, and it can probably be improved by touch. Perspectives might become something the average person can use. Gradient mesh? Just imagine being able to manipulate that using touch! I’m sure the list will go on and on!

InDesign

Here’s the topic closest to my heart, but one much harder to pinpoint areas that will gain from touch technology.

Thinking into the past, you think of exacto knives and tape. I think everyone is happy we left those days behind! In fact, after reading the New York Times article I quipped: “I can see it now: Drawing in Photoshop on a huge digital easel using REAL brushes! As long as we don’t have to use exacto knives in InDesign, I’m cool with that!”

Well, it’s true that no one wants exacto knives and band-aids, but there very well might be what to look to in the past.

Discussing InDesign is a difficult thing to do. InDesign is used in a wide range of applications. Designers are not looking for the features of book compositors. Magazine publishers are not using it the same way as newspaper publishers. And ePub is a totally different game than eMagazines. The fact that InDesign caters to so many different uses is amazing, and it’s very hard to generalize on features.

With that aside, let me try to analyze some of the advantages of exacto knives. When you cut apart text, it allows you do deal with different parts of the text as individual objects. As it stands in InDesign, text composition and text containers are two completely separate concepts. The text composition has nothing to do with the text containers.

Image if we did have a digital exacto knife. Text could be dealt with like first class objects and positioned accordingly. If done correctly, this could be a very powerful design tool. There’s other aspects of manual typsetting that I wish InDesign would include (for example quads), but it’s not clear exactly what some of these features have to do with touch.

One area which could definitely benefit from touch technology is digital publishing. If you are including content which is supposed to react to touch, what better way to define the gestures than by using touch? You could just place the content and define how to interact with the content using touch! This is potentially an extremely powerful feature!

This past year, David Blatner and Anne Marie Concepcion posted an April Fools post on InDesign for iPad and Android. I’m not sure how much of a joke that really was. Of course, it does not exist yet, but based on how things are moving, it could just be a matter of time before we can work on InDesign documents on tablet devices.

I think with InDesign, it’s going to be largely a matter of sitting back and watching how the technology develops to see where touch should fit into an InDesign workflow. Stay tuned…

Other Applications

There’s a lot to look forward to across the board. We’ve already seen some of the ideas on wire-framing websites. Cool new ways to work with photos. Kuler for tablet devices. Wild stuff!

It’s hard for me to do an analysis of other pro applications especially because I’m not too familiar with many of them. One thing I’m certain about: Just about every app we work with will change due to the new computing paradigm. Some drastically, some less so.

Get ready for the interesting years ahead, because all our tools are about to change and I’m convinced it will be a change for the better!


2 Comments

  1. Daddy says:

    Interesting

  2. Zev says:

    I can’t say I fully agree. I believe that there will be two types of platforms now with two types of programs. There will be the programs that are best with touch (paying bills, todo lists, task oriented, tracking expenses, balancing checkbooks). Then there will be the programs that still require desktop computers with keyboards and mice.

    The reason all-in-one computers and laptops are becoming so much more popular is because of how small and compact they are. No one needs or wants huge drafting tables. We can’t even find a place for our desktops. We need Mac Minis and MacBook Air’s. The programs that can be manipulated by touch easily on a small screen will be very successful on touch platforms whereas the programs that need precision selection will stay with keyboards and mice. Just my 2 cents.

Leave feedback