MICHA.ELMUELLER

During autumn last year I had the chance to work as an assistant within a research project at university. The idea was to conduct a study on broken smartphone displays: how often do displays break? Where do they break most often? How does this affect the interaction of users with the phone and—most interestingly—what coping strategies have users developed in order to handle those limitations of the display?

I am quite proud to say that the results have been published as an academic paper at the CHI conference: “Broken Display = Broken Interface? The Impact of Display Damage on Smartphone Interaction.“—yay!

As part of the study we asked people to send us photos of their smartphones with broken displays. There were certain criteria which one had to follow in order to send us an acceptable photo—e.g. a green/white checkerboard image had to be displayed in full screen.

The first image shows the processed version of a submitted photo. It has been prepared using various techniques (cropping, white-balance, perspective alignment, etc.). The second and third photo show the manual annotations which we did as preparations to further analyze the photos.

A part of my work on the project focused on analyzing these photos. I used Matlab and the imagemagick suite to automate a part of this process (Unix style!). One late night, I was working on developing a “contiguous-area-search” algorithm. In order to better retrace if this process was correctly working I started rendering images with the resulting contiguous areas. I was quite surprised to see how interesting this looked and before I knew it I was diving into this. The hours went by and I kept working on improving the algorithm and the color scheme. Eventually I got to the results below, which in my opinion look really interesting. To further process them in an artistic manner I made a selection of twelve of these photos (there are about a hundred of them in total), vectorized them and scaled them to common proportions:

From my perspective the relation of science and art is really interesting and I aim to explore this space more. The way in which I see it, art and science depend heavily on each other. Art inspires and encourages to dream. Just think about the way in which e.g. Jules Verne or Isaac Asimov have influenced science.
On the other hand, science influences art by providing new insights and findings. Take the enormous area of art inspired by psychedelic substances for example. None of this would exist without the findings of scientists. Science provides new instruments and tools as means to create art. We have come a long way since caveman paintings: modern artistic expression has many forms, be it photography or e.g. electronic music.

The artistic process I used in order to create the images above is called Generative Design. It clearly separates from the way by which traditional artists operate. From caveman paintings to the modern process of creating illustrations (Photoshop and a cursor) there wasn’t as much change as one might think: it still breaks down to the same basic principles.

Generative Design is an entirely different process. The artist creates an algorithm which renders the results. But he doesn’t define specific images, drawings, shapes or colors. All of this is generated by the algorithm. This is an entirely different approach and we get some advantages which a “normal” artistic process does not posses. E.g. we get the possibilities of using the calculation power of a computer to create things which are not possible for a human (or at least only possible under the investment of a lot of energy). Examples of this are e.g. enormously complex forms or shapes which can be generated using a generative design process.

In my case the algorithm has been used to automatically process about a hundred of those photos. After seeing the results I adapted the algorithm as a mean to further influence the results. This is a typical generative design workflow: developing an algorithm, analyzing the results and iteratively adapting the algorithm.

I would love to present these generative works on an exhibition, gallery or something similar. If you are aware of any possibilities where this could be a fitting content I would very much appreciate to hear from you.

I really like the title Scratches for these artworks, Pasi deserves recognition for coming up with it.