reculture

What is Reculture?

Reculture aims to help educate British people about different cultures in a safe and comfortable environment—the living room! Each day is assigned a different culture and throughout the day users control their TV by performing gestures native to the culture of that day.

When the user walks in front of the TV, their smartphone receives a notification to inform the user what the culture of the day is. The user then launches the Reculture app to learn about the different cultures and to find out which gestures they will need to adopt to control their TV on that particular day.

Along with finishing off the smaller parts of my project, such as the videos used in the app, I have also been trying to obtain a developer mobile handset for the Degree Show for visitors to use. I have frantically been trying to call a whole host of people: Microsoft, Nokia, Samsung, LG, Three UK. However, it turns out that they’re either really busy, or don’t want to sponsor students… Understandable.

However, recently, Microsoft Students UK came to the University of Dundee to hold a Windows Phone camp. We were informed that we had a chance of winning a brand new Nokia Lumia 800 handset. This was the opportunity I’d been looking for!

The day arrived, and in order to impress, I wanted to show Microsoft my project so far. This meant having to haul my full workstation, including the 32″ TV, across campus to the computing building where we were going to be for the day. I got it set up and Microsoft arrived. We all expectantly walked into the lecture theatre for the short(ish) lecture which walked us through the plan for the day and the basics of Windows Phone and Windows Phone app development. It was at this point when I realised that bringing my full workstation to the computing building was irrelevant: the prize phones would be given at random to those who attended the phone camp. I was quite disappointed at this stage.

At the end of the day, I went to speak to the representative about my project and what could be done to gain a Windows Phone. She informed me that if I submitted an app to the Windows Phone Marketplace, I had a very, very good chance of winning a handset! Apparently, there were 30 handsets to give away and at that point, less than 30 entries. The odds seemed good!

Thursday 19th April rolled round and as impatience crept in, I went to Microsoft Student UK’s Facebook page and questioned when we’d find out about the competition. They replied with a two comment reply: “Just sent out the winners emails…” and “and you were on it ”!

I was very, very pleased! Thank you, Microsoft!

The initial idea of the video was to primarily communicate how the project works, and secondarily give some background to the project. Logically, it made sense to explain the problem first before trying to communicate the solution. Visuals would not have quite illustrated properly the problem of how people of one culture are ignorant to people of other cultures.

One idea I had was to show an Englishman meeting a Japanese-man, but with the greeting being very awkward. The Englishman would reach his hand out for a handshake, but the Japanese-man would have started performing a Japanese bow. They would then quickly swapped actions to reflect the greeting of the other person’s culture. However, this could have been very clichéd if not done properly. Also casting would have been a major problem as I do not know any Japanese people who aren’t of a stereotypical student age.

As these visuals were not going to work, this meant that I had to have either narration or titles in order for the user to properly comprehend the problem.

I chose to make the video as easy to understand as possible and so used narration so that the user didn’t have to read. This was overlayed onto images of David Cameron visiting Japan in 2012. The reasoning behind this was to make the first ‘problem’ part of the video congruent with the second ‘solution’ part: the culture used to demonstrate the project was Japanese.

The visuals and narration then brought the actual project into the foreground of the video. Caitlin, my pretend user, walked into the room and was prompted with a push notification being sent to her mobile phone. She then went into the Reculture app, was informed that the culture of the day was Japanese, and then found out what the Japanese gesture to turn the TV on was. She then performs this gesture and the TV turns on. She then realises that the programme which is on the TV is not what she wants and then delves back into the app to figure out what the gesture to change the channel is. She then performs this gesture. She repeats this process to increase the volume and then sits down with a beaming smile upon her face.

When critically reflecting upon this, I have learnt a few things:

Lighting

My brother helped me film the video and when we both arrived, it was daylight outside. We started filming and all was well. An hour and a half quickly passed and before we knew it, the sun had set. Once I got home, I reviewed the footage and the mistake I’d made was obvious: the white balance and general colouring of the various different takes were drastically different. We started filming during daylight, when there was a natural bluey-white light pouring in from the window. Caitlin is well illuminated and there is relatively little noise on the video image. However, toward the end of the shoot, there was only the filament light on in the living room which provided a dim, yellowy light. This meant that my brother had to increase the ISO, which also increased the noise on the video image. This is noticeable from shot to shot in the final video, and even after colour correction, it is obvious that the light had changed.

Everything Matters

The first time we performed the gestures, the content on the TV was not good. I wasn’t going to say anything as I didn’t want to take up any more of their time than we had to. In the end, it was the mother of the family who questioned the content on the TV. We did it again and selected more appropriate channels. This is what can be seen in the final video. There are still issues with the content, but I chose to go with it anyway.

As final year draws to a close, it’s time to look back and reflect on what exactly has happened over this period of time.

Phase 0

When we started this all the way back in September, none of us really knew what we wanted to do. I was no exception, though I knew I wanted it to involve controlling the TV. I didn’t really know why, and it didn’t really have much of a purpose. This has always been one of my weak points as a designer. In a lot of ways, I’d class myself as more of a developer: taking someone else’s ideas which had a clear purpose, and making them work. After having Ideas Day, and speaking to some amazing people, it became clear to me which way I should be going: the gestural TV control interface route. However, at this point, it was just a gestural TV control interface project without much of a purpose. After all, I wasn’t studying Applied Computing, but I am studying Digital Interaction Design.

What I was doing had to have purpose. As I started thinking about TV and how programming is consumed, the content became more of a prevalent topic for me. It then occurred to me that the British are quite ignorant of different cultures.

Phase 1

Knowing I wanted to investigate different cultures, I needed a method to do this. After brainstorming a few ideas, I chose to use little stickmen to send out. It took a long time to get these packs made up due to the custom packaging and actually making the stickmen. However, the results were great! I was very pleased with the results and then had to start looking at how to implement the results. I realised a needed a more powerful computer and I received a more powerful computer! I managed to program one culture’s gesture set and send commands to the TV depending on the gestures performed. This was big news and I was felt confident.

Phase 2

With so much of the heavy programming being completed in Phase 1, I could now spend Phase 2 working on the phone app. As this was the primary part of the project where any type of visual design language could be implemented, I had to make sure that the visual aesthetic was perfect. I chose to use the same font and general design language which I have used all throughout this project, but also kept within the Windows Phone UI Design Guidelines. I chose to represent different cultures by the countries which are generally associated with these cultures. Since I was slightly ahead, I also spent a lot of this time helping others with their coding and electronics. I am now proficient in RFID technologies, SOMO modules, AppleScript, current sensors, basic Xcode, Arduino code and general determination!

When it came to integrating the Kinect sensor and the Windows Phone app together, there was a steep learning curve. I had to tell the computer side of the code to send a message containing the culture of the day to all the registered phones, once a user was detected by the Kinect. Once I overcame this battle, I had managed to successfully implement push notifications on the handset and have a full backend system ready to accept more Windows Phones’ subscriptions to these notifications. This means that at the Degree Show, other people with Windows Phones can download the app and receive the push notifications on their own handsets. I was very proud of that achievement and I just hope Windows Phone users attend the Degree Show!

The end?

The project is almost completely over. It is a sad time. Am I glad to see the back of it? No. I will hopefully continue to develop the application. I wouldn’t mind commercialising the project, but in the same way, I’m not going to actively seek funding. I’m very thankful for all of the things that this project has taught me. The ethnography, the research, the technology, the presentation skills. It’s been a wonderful time.

It’s time to start my video and so I must first analyse existing videos before creating my own. Below are a few concept videos in varying styles. I will analyse them in the sandwich format: bread, filling, bread. The bread is the intro and outro before and after the filler, that is, the main concept.

Narration accompanied by video

This is a relatively long concept video. Too much bread and too much filler. As with any large sandwich, it leaves you feeling slightly bloated and questioning why you felt the need to consume such a large portion.

Lots of talking from the same monotonous American accented voice—not good.

Titles followed by narrative and video

This is a really long concept video, but the intro communicates the idea well. Narration is not left for too long without visuals shortly following, keeping the viewer attentive. Very creative visuals further into the video.

Easily digestible and beautifully tasty bread followed by a tasty filler.

The video says it all

No bread here. Just a tasty filler. Communicates everything naturally, but perhaps leaves the viewer without much context and wanting a little bit more. Perhaps should be in the format of an open sandwich—filler on top of a bed of bread to bring it all together.

Narrated overview followed by explanation

Lovely bread. Lovely filler. I like this format.

One of the initial ideas for the app was going to be a tool to compare the gestures of each culture for each command. For instance, there would be a ‘page’ to show all of the different cultures’ gestures for the ‘power’ command. The initial idea was to include a ‘compare’ view in the app which would simultaneously play every cultures’ gesture for each command.

However, without post-processing of all the videos, this is not possible as the Windows Phone SDK does not allow for multiple MediaElement objects to be active at the same time. I came up with a slightly less impressive way of being able to compare gestures and that is to simply include a small amount of text on each gesture telling the user what other gestures are similar to the current gesture.

After the frantic push to make Mark I as functional as possible, I have been left with lots of little jobs to do: I’ve had to programme all of the culture’s gestures, test them, refine them, implement audio feedback for the users, house my FEZ Panda II and try and source a Windows Phone for the Degree Show. I’ve managed all but the last two on that list.

I’ve found that nobody’s really that keen on lending me a Windows Phone for use at the Degree Show. Unless I can get a Windows Phone for the Degree Show, I’m going to have to use my own and not have use of my phone for a week, or I’m going to have to buy another… Money is very tight right now and so buying one is out of the question.

Apathy

I haven’t even given much thought to housing the FEZ Panda II. I can’t quite rationalise why not, therefore, I’ll blame it on apathy. I think in my mind, I know how I want it to be housed, but I haven’t quite gotten round to making it. In essence, it’ll just be housed within a small plastic box with a micro USB port, and a 3.5mm jack socket for the infrared LED to plug into. This will then attach to the back of the TV using either magnets or suction cups. I’ll get it done soon!

Results of apathy

Also, in my apathy, I decided I’d try and get the word out about my project now so that it would have some type of public/web presence aside from this blog. The project managed to make it onto wpcentral and WMPoweruser! A relative success! At the time of writing, the short video I made has had over 1,700 views. I’m quite satisfied with this, but I hope to make it onto the likes of Engadget and The Verge. I just need to get a decent concept video made, and this should be good enough to be featured on those blogs. I think I have an idea for the video lined up, which is quite nice.

Other successes include being given my own 32″ TV to use for the duration of the project up to the Degree Show. This means no more disconnecting or dismounting of the Kinect and the FEZ Panda II!

Anyway, if you’d like to lend me a Windows Phone for a little while, then, please, get in touch!

Now that the Mark I build has finished, I set to work on Mark II.

So far, I’ve managed to externalise all of the data so that it is not stored on the phone, and managed to assign each five minute period of the day with its own culture. This is primarily for the degree show to simulate what happens each day.

I’ve also managed to program the British gestures. Once I’d programmed the British gestures, I sat down and compared the total results of the Phase 1 research. From the research, it turns out that a lot of cultures share similar gestures for similar things. I decided to chart these gestures, assigning each gesture its own symbol. The scribbled out symbols are the symbols which I’ve now programmed. Wow! I’ve managed to program the majority of gestures!

Mark I is at an end and I feel a lot has been achieved.

In my mind, there has always been three parts to the Make stage: detecting the gestures, sending the infrared (IR) commands, and creating the documentation/instructions.

Detecting the gestures

The gesture detection was perhaps the most challenging part, but for Mark I, I made the decision to focus on the Japanese culture, simply because it was the most dynamic and least ‘Western’ culture. I managed to program these gestures and, with varying success, detect the gestures. I have learnt that the Kinect is not so great at figuring out where your arms are when they are too close to one another and so this will perhaps need to be fine tuned in Mark II and it may be that some of the gestures will need to be totally reconsidered.

Sending the IR commands

Sending the IR commands was also quite demanding, both as a programmer and also as an electronics engineer. I had no idea about how IR signals worked before Mark I, but by the end of it, I now have more understanding. Currently, the FEZ Panda II is mounted to the back of the television, with the IR LED coming round to the front and pointing at the TV’s IR receiver. This is not the most elegant execution, and is something which will need refining in Mark II. If the project was ever commercialised, then the IR sender would definitely have to be housed externally from the TV and would need to incorporated into an object. This, perhaps, will need to be refined in Mark II.

Creating the documentation/instructions/educational information

As we’ve already established, the project’s documentation will be an app. Work on the app has begun and I feel that I’ve achieved a lot in the way of the functionality. I have built on my skills as a mobile app developer and have successfully leveraged the power of many web technologies in order to create a scaleable push notification service for the app — something which I have never done before. The video below shows this. For Mark II, the app will have to be refined, fully tested, and the gesture comparison feature will have to be implemented. I will also need to perform user testing on the app in order to make it as user friendly as possible.

At the beginning of Mark I, I thought that I would be able to access to various travel APIs in order to bring each cultures’ information straight into the app. However, after contacting many different websites, most of them do not allow access to their APIs. For Mark I, I’ve worked around this by simply linking the user to one of the websites. For Mark II, however, I will have to research ways to bring content into the application.

Below: A (very) quick run through of my progress so far.

The phone app has several use cases:

1. To notify the user what the culture of the day is.
2. To inform the user what the culture’s gestures are.
3. To enable the user to learn more about the different cultures.
4. To enable the user to compare different cultures’ gestures.

Notifying the user what the culture of the day is

Smartphones are great for this! When the user walks in front of the TV, I want to be able to send a notification to their smartphone telling them what the culture of the day is and asking them if they want to learn more. If they choose to learn more, the app initiates and shows them the gestures of that culture with a little bit of information about them.

Informing the user what the culture’s gestures are

This section of the app will let the user view short video clips of someone performing the different gestures for that culture. Each gesture will have a small description associated with it which will shed more light on the gesture and its roots.

Enabling the user to learn more about the different cultures

For the remit of my project, there is no way I can collate a whole database of information about various different cultures. In light of this, I plan to enable the user to be able to navigate to a website to learn more about each culture. Websites such as the Word Travel Guide are great for this.

Enabling the user to compare different cultures’ gestures

At some point during the Make phase, I hope to be able to add functionality in order to let the user compare the different gestures of each culture on one screen. This will probably be in the form of all the videos for each gesture playing at the same time.

Wireframe Design

The feature image details the flow of the application. The user interface utilises Microsoft’s Metro UI, and its associated components such as Panoramas and Pivots.