Media Technology

ColourMatch

This game was created for the course Creative Programming in the code language Processing. The game rules are simple: the player has to match three big balls with three smaller balls of the same colour. Each level you succeed gives you one point. The higher the level, the faster you need to match the balls. If your time is up, you lose the game and get to see your final level and score.

Music Machine Gary

This is a project made in Pure Data for the course Creative Programming. Concept: You are listening to Gary. Gary really wants you to clap and sing along to his music. When you open the application, he will tell you so if you press a bang (see patch). When you clap, the trigger value of 70dB is usually reached, which triggers some oscillator tones and a phasor tone: the beginning of his song! If you clap very loudly or sing, you reach either 80, or even 90 dB. When you reach 80dB a rhythmic noise is added and Gary will tell you ‘Wow, I love this!”. When you reach 90, you will hear both the rhythmic noise and some more oscillators and Gary will say “This is amazing!”. If you reach 90dB in one go, only “This is Amazing!” will play, but if you first reach 80 and later 90 you will hear him say both things separately. Whenever the rest level is reached, a random reply from Gary is banged. All 3 replies have to do with what he wants you to (clap/sing or be loud(er)).

Note: This work is very dependent on how loud your speakers (and environment) are. If your speaker volume is too loud, the voice of Gary will already reach the threshold level that starts the music. In the demonstrating video I actually have slightly different threshold values then mentioned in the concept explanation above.

Horizons

For the course Meta Media, a media artwork based on the concept “Lighthouse” was created. We defined lighthouse as a tower that has a light that often turns. We took inspiration from a real lighthouse, where red plexiglass was used for signalling. The finished work consists of a box with different colours of plexiglass, which stands in front of a round, white wall. Inside the box is a lamp, which can turn from side to side. The result is a wall with light colours that move slightly from left to right. People standing in front of the wall can interact with the lights by masking parts and playing with their shadows.

Sense Floor

see Me

For the course Artificial Creature, the project “See Me” was created. The assignment of the course was to take a characteristic of a(living) creature and turn it into an artificial creature. My group created a creature that is invisible, but wants to be seen (characteristic = wanting to be seen). We created a box which has a small ventilator in it and some holes on top. On top of the box is a large glass compartment. Inside the glass we put incense. Attached to the front of the box is a distance sensor. When you are standing close by the work, the ventilator is blowing quite fast, which makes incense smoke less visible. Meanwhile, you hear some interesting, yet calm sounds. Once you walk away, the creature becomes agitated, because it wants to have your attention and be seen. The creature’s sounds become louder and unnerving. Meanwhile, the ventilator in the box is blowing slower, making the smoke of the incense more visible then before. The idea is that the creature is trying to be seen through the smoke, as well as attract your attention through sound. We made this project using Arduino with a distance sensor and Pure Data.

Horror Soundscape

For this project, the aim was to create a creepy story using sounds. Our objective was to create an immersive experience by using a four speaker set up and different ambience noises. Furthermore, our narrative reacts to the voice of the audience, which is detected by a microphone. Based on the decibel of the input, threshold will be activated and  the narrative will keep on unfolding. Ideally the experience takes place in a dark room, the audience should focus on the sound as much as possible. This code for this project was fully created in Pure Data.

The soundscape starts in a forest, where you here wind and birds chirping. Slowly, the atmosphere takes a darker turn as you start hearing an owl cooing and a wolf howl; the night begins. Crows are cawing and you here something that could be the wind, or possibly a ghost. Next, you hear a little girl’s laugh. After a while, you hear the laughing voice say “”Hello?”. If you reply to this voice, it will again call out “Hello?” and later “Who are you?”. The replies eventually trigger a whispering sound and at the end a horrifying scream.

Dog TV: Can Media Improve a Lonely Dog’s Wellbeing?

For the course Non-Human Cognition, I conducted literature research with two fellow students about the interest of dogs in television and other media. Furthermore, we aimed to answer the question whether television can help dogs that are often home alone feel less distressed. Although dogs do show some interest in television, they are not able to pay attention to it for a long time and follow a storyline. Currently, better options are available for keeping a dog company at home, such as interactive games. Yet, more research needs to be done about the possibilities that television could offer for dogs. We have compiled our findings of this research in a poster.

SoundCave: a Game Where You Need to Listen

Video games are a popular pastime, but for visually impaired people there are not nearly as many video games on the market as for sighted people. Visually impaired people often experience more symptoms of loneliness and depression, while video games have been correlated to an improved mental wellbeing. This was a reason to develop SoundCave, a multiplayer audio-based computer game, which uses stereo channels to provide each player with a unique set of sounds through one earphone of a set of two. Its aim is to be accessible and enjoyable for both sighted and visually impaired people, contributing to inclusivity and combating social exclusion. Based on the results of two user evaluations, it is concluded that, although SoundCave has proven to be accessible for both demographic groups, more research should be done in order to make the game more enjoyable and reach its full potential. SoundCave was created for the course Human-Computer Interaction. We used the program Unity to create this game.

Listen to some parts of the game demo (in Dutch) here :

SoundCave: a Game Where You Need to Listen

The great whales are an important part of the deep-sea ecosystem – not only when alive, but also when they are dead. The sunken carcasses of great whales have a big role in the lives of a long list of other species – whale-fall specialists – and are a necessity in order for these species to survive. Commercial whaling has shown to have big consequences on both the population of whales and their length – and thereby also on the amount of sunken carcasses available to the whale-fall specialists. 

In other words – the human population and its tendency to overuse the resources of the globe causes problems. It is not only the case in the most obvious places, such as melting poles and extreme weather conditions – it is also seen in places which are not directly visible to us. 

For the course Playful and Creative science, we tried to contribute to the visibility of studies like this with a new type of output: we are expressing the amount of whales in the sea in weight. Furthermore, we created an infographic to illustrate the diminishing length of whales and the decreasing population of whales compared to the increased population of humans.

The Smartest Human

“The Smartest Human” is a recorded screenplay that re-enacts the Dutch trivia game show “De Slimste Mens”. What sets it apart from De Slimste Mens is the focus on philisophy: the goal of this screenplay is to communicate information about philosophy in an original and interesting way. Next to the presenter, the game show features three participants: philosphers Karl Popper, Thomas Kuhn, and Paul Feyerabend. For each role, the philosopher’s view on the world and way of thinking has been exagerrated, whenever a question is asked to them. This is done to give the viewer a better understanding of the different stances that these philosophers have had on science and the world in general. This video was created for the course Sciences and Humanities.

Fuse

Fuse is a group project made for the course Media Technology and Interaction Design (taken at KTH). The design prompt for this course was to create an interactive digital application for a world that is dominated by introverted people. Through extensive user research regarding social needs, preferences and rules in public spaces, we developed Fuse. Fuse is an app and wearable that matches other people based on interests and hobbies. You only match with someone when this person is physically nearby and currently open to interacting. The goal is to make it easier to have social interaction in a world where approaching another person in a public space is (even more) difficult. We designed most of this project in Figma.

Photo Planet

For the course User Experience Design and Evaluation (taken at KTH). I had to design an interactive digital project based around the theme of Sustainable Data. I decided to create an app that spreads awareness about the amount of data that is saved through photos on your camera roll, by guiding the user through an interactive story. When the user has gone through the story, ways to limit the amount of photo data are mentioned, and the user is occasionally reminded by a notification to delete photos. This project was created using Figma.

LangBot

LangBot is a multimodal interactive virtual agent that is used to learn a new language. By interacting with the agent, the user can learn new words in a more natural way than most language learning apps, since the agent can make conversation and explain things that are unclear for the user, just like a human could react to a question. LangBot uses speech and haptic input, as the user can name a word or click a button. Furthermore, LangBot has both speech and visual output, as it demonstrates words through speech and words are depicted in images and text. The images make the user more engaged, since they can choose which words they are interested in learning.

This interface was developed for the course Multimodal Interaction and Interfaces. We used the virtual version of the social robot Furhat and its SDK, which works with the programming languages Kotlin and Javascript. For the Graphical User Interface (GUI) we used the FurGUI skill as a base.

TeleCombo

For the exhibition project Science to Experience we created an art installation called TeleCombo.

People stand in front of a board containing ten buttons, where each button corresponds to a specific object. In the beginning, all buttons are lit up. When you click one button, the other buttons blink. When you click on your second choice, two things can happen:

1. The combination prompts a video pitch promoting the product of about 1 minute 
2. There is no video for this specific combination yet, so you are asked to film your own video

Next to or above the item-choosing display, there is a television on which the different pitches for different product combinations are being played in a loop in our television program called Telecombo. There is a couch in front of the television on which you can sit and watch. When you are prompted to record your own pitch, this pitch is added to the list of videos.

Behind the display there is a small recording studio including a crew giving instructions. This was made by creating a square of beams and putting a white sheet and curtains around it. Inside the recording studio there are prompts and tips to help the participant think of a good commercial. Once the pitch is recorded, it is edited and the user has the option to add a bit of text or a drawing in our commercial template. The pitch is then added to the Telecombo commercial television program.

The commercial film studio:

The display:

People in commercials:

Applying the Principles of Japanese Garden Design to a Virtual City

According to the Attention Restoration Theory, natural environments have the potential to restore an individual by decreasing fatigue and stress of everyday life. Incorporating nature in urban settings can thus be beneficial for the mental health of people living in the city. The Japanese garden is an example of a small-scale garden in an urban environment that has restorative  potential. In this research, the restorative potential of the Japanese garden is re-assessed and commonly used design principles of the Japanese garden are defined. After this, an existing neighbourhood in Amsterdam is replicated in a 3-dimensional model and is redesigned according to the design principles of a Japanese garden. The goal of this new design is to investigate whether an urban environment can be restorative through the placement and form of urban elements only, rather than relying on the incorporation of nature. To evaluate this question, the virtual neighbourhood is compared to the original neighbourhood in a VR walking video. The paper concludes that the city can have a restorative potential when Japanese gardening principles are applied on the built environment. However, more research should be done about the incorporation of nature in the urban environment and the addition of detail in the built environment, as well as other people’s perception of the virtual city.

This research was carried out by me individually as my graduation project. I have used the open-source software Blender to create 3D models and used cardboard VR-glasses to watch the videos.