The Future of Computer User Interface is Physics Based

In the early '70s the research lab of Xerox at Palo Alto has developed the so called "WIMP user interface": Windows, Icons, Menu and Pointer. It took almost 15 years until Apple started to integrate such functionalities in the first Macintosh, and couple of years later Microsoft's Windows started to use the same metaphors.

Today's operating systems still use the same GUI which is based on the ideas already available in the '90s. But computer resources are growing exponentially -- couldn't we use this extra computing power for a better desktop experience? We could. The following projects are such initiatives, and their eye candy demo videos are definitely worth watching.

XGL for Linux

Novell started a project called XGL in late 2004. Their goal is to create an X server architecture for Linux, which allows fancier, more intuitive user interaction. XGL puts different desktops on the sides of a cube. Switching between the desktop happens through rotating this cube. The other features stay in the realm of 2D, but include nice effects: windows deform like jelly rectangles when one moves them around; transparency of windows can be fine-tuned, etc. Check out this video for a short demo, or burn a kororaa live CD to try it out yourself. Now, XGL does not bring revolutionary things, but it enhances current GUI which starts to behave much more physics based, and it is a bit less abstract.

Click here to view the XGL demo video


Project Looking Glass

A similar project was started by Sun, as well. Not surprisingly, Project Looking Glass is based on Java Technology, and its aim is to enrich the desktop experience with 3D elements. One can, for example, tilt windows and put them on the side. A neat function is that one can turn around windows and attach notes to the back. Some other features include: a jukebox-like music browser, control over the translucency of windows. You can view the video of Jonathan Schwartz presenting a short demo of Project Looking Glass here. Again, we did not see fundamental innovations how we interact with the computer, but mostly neat enhancements were introduced to improve the user experience.

Click here to view the Looking Glass demo video


BumpTop - real physics

Research communities strive to reinvent the way people interact with computers, as well. Anand Agarawala from the University of Toronto's Dynamic Graphics Project had a really cool master's project: BumpTop. He created a 3D computer desktop using a real physics engine, AGEIA PhysX SDK. Anand thinks the computer desktop should simulate the real desktop as well as possible, and he goes as far as trying to simulate even the untidiness of a real desktop. Of course, BumpTop supports a lot of smart functions: One can drag, throw around icons which collide and behave exactly as one expects in real life. After organizing icons into piles you can lay them on a regular grid, browse through them with a fisheye function or leaf through the pile exactly like you would with a real life book. Dozens of other functions are available, but what is the most important is that all of them resemble to real-life manipulations. He has a very thorough video presenting the features of BumpTop. Moreover, according to his website, he is planning to build a full blown desktop replacement that can be installed on any machine.

Click here to view the BumpTop demo video

But it does not take teams of research and state-of-the art physics engines to enrich the desktop environment. Kristian Høgsberg implemented a small and simple physics engine, Akamaru and created a physics based Mac OS X-like dock for Gnome. This short video demonstrates what a single programmer can do with a bit of programming time invested.


Conclusions

All the projects presented above have at least one thing in common: physics simulations. They use simulation of real-life world in order to bring something to the user which is less abstract and it is closer to the world we are used to.

The question is: Will users the users accept it? Innovating user interfaces is not an easy task. Users don't want to learn a lot of new things in order to use the computer, so there will be always a resistance toward new GUIs. One way to go around this problem is by not making drastic changes (like XGL and Project Glass), thus allowing an easier transition. Microsoft has learned this the hard way, through Microsoft Bob. So, this time the Redmond-based company takes smaller steps with the upcoming Vista operating system: Windows Aero. According to this demo video, they introduce smooth transitions, improved switching between windows, but nothing too radical.

Even if one makes a bigger leap (like BumpTop) the first and foremost criteria for success is that the new system should offer improved usability. Joel Spolsky defined usability in a single sentence: Something is usable if it behaves exactly as expected. But what could look more natural than the physics of the world we live in? So, if we accept this definition, these projects are definitely on the right track to get closer of tomorrow's mainstream desktop environment. I predict that there is a still long way ahead of us, but maybe in a couple of years Microsoft or Apple makes the step, just like in the '80s Steve Jobs went to Xerox's research center and came back with a new user interface.

---

Thanks to Gabor Cselle for reviewing drafts of this.

Google's advice for January the 1st

January the first 2006... I am wondering what should I do today... and of course, Google knows everything, so I let go a search for "what to do on january 1st":
So... Google tells us to learn to be a great kissers on the first day of the new year... at least the customers of Google AdWords are proposing this to us... :)

PS. These are the results for a Swiss IP, Google might propose something else to you on the other side of the planet. Actually, I am really disappointed that a couple of hours later I had no more sponsored link for this search...

Geometry in computers

I think that the computer is a great tool to ease understanding/solving of geometry problems. Going one step further, I suppose a lot of problems solved by current computer solutions could be improved by a more realistic 3d interface. Just think of the difference between maps.google.com and Google Earth. Similarly, Microsoft improved their map service with bird's eye view pictures, and it really makes a huge difference.

Today, computer systems using 3d are mainly CAD systems for engineering, 3d animations and content creation tools. "Normal" users can not do anything geometry related, usually one needs to learn quite a lot before one can use such an animation tools like those from Alias, Softimage, not to speak about the CAD systems.
Initiatives like Cosmic Blobs are really interesting. This software from SolidWorks is a 3d graphics and animation tools designed for kids. All operations are simple, intuitive enough that a simple child should be able to use them. If such tools will be available in the mainstream animation and modeling software packages, much more content creators will find their way to create interesting content -- not only those who learned all the cumbersome technical terms and tools in today's modeling tools.

But let's go back to my original observation. Geometry is mainly present in today's computer only in animation and engineering domains. Aren't there somewhere some other domains? Since we live in a world where everything around us is basically a 3d scene, there have to be other natural computer application domains for 3d geometry...

So, why don't we find/create them? To find the answer just think of the technology how we are programming geometry today. There are available libraries/geometry engines like CGAL, OpenMesh, SMLib, Parasolid and others... but if we compare this number to the multitude of tools available to create database applications, we will see a huge difference. And for example when one debugs an algorithm working on triangle meshes, one sees pointers and numbers which do not make any sense without geometric interpretations. Why isn't there some kind of visual debugger where the debugger interprets the state of data structures into the geometry and renders it? The way I see it, today's geometry programming is in a "baby-phase". We should be able to create better tools for ourselves and then we have a much better basis to take geometry in computers to a different level.

As a small conclusion to this brainstorming I would say that there is a lot of potential to be explored. I would love to see 3d computer graphics used for a lot more than engineering, games and animation movies. It's clear that a lot of serious mathematics and work is needed, but it would be worth investing into it.

Your own... candle??

If you are around Zürich in December it might be a good idea to go to the Kerzenziehen am Bürkliplatz. With a bit of patience you can make your own candle... or something like that. This is what I came home with. Maybe it's exaggerateded to call it a candle... but it's something funny and it smells like honey... Actually, my creature was upside down during "production time", this is why he has his "tale" so nicely straight up in the air. Some other guys really take this thing very seriously, and make reaallly huge candles which weigh several kilograms at the end. They invest many-many hours over several days, weeks probably... so that in the end... I don't know what they do with those huge candles. Whatever... actually, I don't know either what I should do with my "something-like-a-candle"..
And since I am still at recommendations, let me give you a tip for your next movie: Broken Flowers. It reminded me in a way to Mulholland Drive, which I liked a lot, as well. So, enjoy your winter time!

Brian Barsky: See what you see

Yesterday I attended an excellent talk of Brian Barsky. He's professor at UC Berkeley with affiliation to both computer graphics and optometry group. This mixture resulted some really interesting projects which contributed to both, developing medical tools to test and improve the human sight, and rendering scenes in computer graphics by modeling the eye instead of using a simple camera model.
A small part was extremely interesting for me: For detecting the aberrations, irregularities, defects of the cornea, they have developed a tool which resulted images of a human eye like this (CWhatUC paper):

The irregularities of the cornea surface can be seen by the deviations of the circular reflection lines. Basically, the very same tests is used in CAD or other geometric modeling tools to asses the quality of surfaces generated. Compare the picture above to this Solidworks example:
Actually, I am working right now on exactly such a surface quality estimation test in my internship at Autoform, who is producing simulation software mainly for automotive industry. I always love to see different research areas meeting at some points, it is great to know that things can be reused, combined almost to infinity, and the results can be really astonishing.
And I want to note again that the talk was just great. Brian Barsky seems really a funny and easy going guy... He could really put some highlights of his research work into a presentation which was really fun to listen to and interesting. What I especially like about such established professors like him, is that they are really working and researching in an area what actually their passion/hobby is. So listening to his talk on Friday really made my day.

What is the best job?

Being close to the end of my studies, I am thinking quite a lot about this issue... what is the greatest job?
So.. computer scientist.. for most of the people it means simply software developer. Ok, I would be okay with that. But I want to have control over my work, I want to do it in a great environment, do really quality work... and.. of course, what the most important is... to do challenging stuff. The whole thing smells like I do not want to be just one developer for a big corporate company.. then the small ones might be to unstable... one could "not advance"... slowly I end up with the idea, that if I want to do software development, then I should do it for my own company.. tech start-up... it is not an unusual thing nowadays.
But what else... academia.. that could be cool. Challenging stuff, less stress because of market constraints, one does really mostly what is really interesting... It is definitely not easy, especially if one would want to go all the way on the academia track.. actually it is damn hard and one needs the intelligence/talent... otherwise no chance. There's kind of a disadvantage... usually one is much farther away from the "customers".. it is more difficult to see the "product work".
But there is more..
Today, as I came home with the bus, I've decided, the busdriver has the best job. He probably doesn't love it that much (what is there to love about driving the whole day the bus), so he can not get annoyed about how he would do it better... he has a stable job... he just does it... he can switch off his ambitions during the day.. he drives the bus, and tries to find the fun in his life somewhere else...
Or... pro snowboarder.. that is cool, as well. Lot of bullshit to be done... a lot of parties.. unconventional stuff.. injury for 9 months... again videos, party, you love what you're doing... After 20 years, you are a piece of junk physically, but you always did what you wanted.. you partied...
I could probably go on... and I still wouldn't be much smarter... but yeah.. slowly I arrived to that situation when this "what do I wanna do with my life" question is around... Something will happen, we'll see, at least I realized that there is the question to be answered ;)

First 2005 snow in Zürich

It's cool, it's snowing.. the first time this year in Zürich, and again I have made the picture from my window, just like last year... I hope you enjoy snow, as well.. it always gives me good mood and stuff.

So, have fun!