'The Blindfold Experience'
A binaural horror soundscape.
Note: Headphones are required for the most optimal experience.
----------------------------------
Link: https://youtu.be/nU751GO32aw
'Cipher 603'
Sound design for film project.
----------------------------------
Link: https://youtu.be/EhuC5niOY40
'April Moments'
5.1 Sound design for film project.
----------------------------------
Link: https://youtu.be/LTzQP1RRBpw
'Storytime Readings'
A children's audiobook company
----------------------------------
Link:
www.storytime-readings.jimdosite.com
The Blindfold Experience - Reflective Journal
Journal 1 - Getting Started.
Starting the year off is an assessment piece detailed 'The Work'. Featuring an open structure, this assessment is a great way to start the year and allows us to focus our creative potential on any work we wish to create. I have always wanted to create a piece of work that induces a variety of emotion just through sound. What better way to do that than with just a soundscape? Maybe with the inclusion of horror elements that perhaps promotes fear and anxiety?
Now even though I enjoy creating soundscapes similar to what I made last year with the 'A Cry for Nature' project, I can't help but feel that the overall sonic potential needs to be expanded a little. Stereo mixes are great and all, but the overall depth of the audio experience could be better, especially for a piece of work that can be so complex. Because of this, I would love to be able to create the horror soundscape as a binaural audio piece. Elements of the audio experience will now have the opportunity to present themselves within an equal space, surrounding the viewer in a more realistic environment rather than being positioned in either the left or right channel. I could add to this experience further, offering a form of head-tracking that allows users to move within the sonic environment. They would then be able to hear the directionality of each sonic element and direct themselves to it like you would in real life. This would be done stationary however, and although this limits the user to fully move within the soundscape compared to what a virtual reality experience might entail, I can use this limitation to my advantage to make the experience even scarier.
I entitle this project 'The Blindfold Experience'. Influenced by creative works such as 'Kitchen', a PlayStation VR horror experience created by Capcom and 'Immersive Sound, The Art and Science of Binaural and Multichannel Audio' by Agnieszka Roginsky and Paul Geluso, users will be awakened into a horrifying sonic environment. Trapped and blindfolded within the unknown, users are forced to listen to their surroundings.
The 'Resident Evil series has always been a massive influence with every piece of horror work that I create. Revolutionising the horror genre in 1996, Capcom has continued to adapt the series over the years, resulting in experiences such as 'Kitchen' that have introduced many people (including myself) to Virtual Reality. Although the experience is now a standalone application for PlayStation VR users, Kitchen was first introduced as an exhibition piece at the popular gaming event Pax West. Featuring an enclosed room with only a few pieces of furniture and a chair, users were told to place on the virtual reality headset and encounter the ungodly horror presented to them. Although different in many ways, 'The Blindfold Experience' wishes to build upon that experience, allowing users on multiple devices, including an exhibition-style piece, to experience a horror environment through binaural sound design.
To this day, I am still uncertain as to why the Resident Evil series has always had such a profound effect on myself and my work. Japanese culture has always had a huge effect on horror culture, and although somewhat quirky at times (noted by the original Resident Evil’s notoriously bad voice acting), the sense of having immense danger at every moment is something that continuously attracts me. It’s addicting. Similar to the feeling of being on a rollercoaster, the emotion of horror really isn’t for everyone. However, for those who wish to endure, it’s something we can never really escape from. Replayability. Being able to experience something that is just as exciting over and over again, really is an artist’s dream creation. Using these influences, I aim to reproduce these ideas within my own work, with future adaptations of The Blindfold Experience being a definite possibility within the next few years. For now, a more standalone experience is what I wish to achieve. Similar to the tech demo originally produced at Pax West.
Finally, one of the biggest influences for this project was Roginsky's 'Immersive Sound, The Art and Science of Binaural and Multichannel Audio'. As an avid user of new and upcoming technologies, I wanted to experiment with the concept of binaural audio. Roginsky's book has allowed me to gain a much deeper understanding of the technology, including its significant advantages over the 'traditional' listening experience and its effectiveness for creating immersive sound through various mediums. The use of multi-channel audio such as Spatial and Binaural audio allows for a lot more precision in terms of audio directionality, reverb occlusion and audio immersion, something that will of course be hugely advantageous for the listener. In the pursuit of audio realism, Roginsky's book has allowed me to contemplate the planning process, describing that "To create a convincing auditory experience, and one that closely approximates a natural listening environment, careful considerations must be made with respect to binaural synthesis, capture, reproduction methods and tuning".
References:
- Capcom (2016) Kitchen Demo [Video game]. Capcom.
- Glombicki, M. (2016) Hands-on: Capcom's Resident Evil "kitchen" demo is terrifying on PSVR, Road to VR. Available at: https://www.roadtovr.com/hands-on-capcoms-resident-evil-kitchen-demo-is-terrifying/.
- Roginska, A. and Geluso, P. (2018) Immersive sound: The art and science of binaural and multi-channel audio. New York, NY: Routledge, an imprint of the Taylor & Francis Group
Journal 2 - Further Planning.
Like any project, a clear plan of the development cycle must be made before we can even begin to get started. As a result of what was discussed in the last journal entry, I feel that it is necessary to discuss my overall ambition for the project. In its truest form, The Blindfold Experience is an installation featuring a curated environment in tandem with the best-suited hardware. This allows us to produce the best experience physically possible for the project whilst also allowing for real-time feedback to be collected for any future improvements that could be made. With this, however, the experience will also be made available online as a 360-video publication on YouTube. This allows any user to view the experience on any device, with only a slight reduction of immersion. To begin the planning process, a variety of created work has been detailed below:
- Padlet - A webpage which allows various amounts of information to be stored and viewed, similar to rough planning sketches like mind maps or flashcards. This allowed a quick way of adding references, videos, artwork, and fun ideas for the project, which could be easily tracked and shared. Not surprisingly, conventional themes started to emerge. As I wanted to align myself with what various audio expectations may include other similar horror experiences such as ‘Sleep Paralysis’ or ‘The Conjuring 2 Experience Enfield’ gave good insight into the user interaction process, evolving over time as both of the experiences continued to frighten me. Falling into these norms is a general convention within the horror genre, and although various differences will definitely be apparent due to my absence of any major video material, the use of player interaction especially within a 360-video format is something that I had to take inspiration from. Continuing on from this, the documentation strategy of using Padlet for both a note-taking tool and for the categorisation of various assets will continue to be used throughout the remainder of the project.
- Gantt Chart – A project management tool that allows me to create and detail specific tasks throughout the project. As with the Padlet, this will be followed extensively throughout the project, promoting time management and organisation. Although other project management tools are available, I believe I am more adept at creating and using Gantt charts as I was taught to develop them in High School.
- Risk Assessment – A document that analyses any risk that could be associated with or created due to the installation. Due to the requirements of the Glasgow School of Art and by law, the inclusion of a risk assessment document is required.
- Wire Diagram – A diagram showing the installation's technical layout. Although not explicitly required, it has been taught by the lecturers at the GSA that these types of documents should always be created and shared with any members associated with the creation of the installation. This allows everything to be pieced together as efficiently as possible, following the layout presented on the diagram.
- Ethical Assessment and Consent Forms – Documents that request any consensual obligations required for the installation (recording consent etc.), as well as report any ethical issues associated with the installation. Due to the requirements of the Glasgow School of Art and by law, the inclusion of these forms is required.
References:
- (2016) The Conjuring 2 - Experience Enfield VR 360. Available at: https://youtu.be/A6aRkhlqWuE (Accessed: April 28, 2023).
- (2022) Sleep Paralysis. Available at: https://youtu.be/xYq91I5STA0 (Accessed: April 28, 2023).
Journal 3 - Technical Workflow.
This entry will encompass the final part of this project's creation, collating everything we have learned within my previous entries and putting it all together to create the project in line with its initial vision.
Video Editing – The 360 video is more of a conduit to the experience rather than a fully featured film. Because of this, no external media was used in the creation of the 360 video, allowing me to form a dynamic introduction sequence purely on the tools that video editing software such as Premiere Pro provides. Adobe Premiere Pro was chosen due to my familiarity with the software. As a media creator, I have often used track-based software due to its ease of use, and since Premiere Pro is also given for free by my university, it was the best route to go down. To start, a new project file was created using the Apple Pro Res codec, allowing for the most optimal playback for my M1 MacBook. Various pieces of media were then created and sorted within the bin section, allowing me to drag and insert multiple text files into the sequence with ease. Following a plan I had previously created within my head, I began constructing the 360 video, starting with an introduction countdown to generate tension and time for users to get comfortable, along with text elements to introduce what the project may include. The length of the project was also very important. With experience in creating soundscapes for similar projects, I had wanted to make the project last for around 5 minutes. This allowed optimal time for both the introductory sequence and the soundscape portion without making everything feel too dragged out.
Generating the 360 video myself, influenced my decision-making for the project dramatically. Although it would have been great to have recorded my own 360 video, reproducing the environment, along with actors and a set would have caused dramatic time restraints and costs. Using the blindfolded idea allowed me to circumvent this all, adding in an additional element that re-created its own themes, illusions, and imaginative ideas. Users are now able to imagine the scene, playing into their own fears without any visual indication of what is next.
Audio Editing – Once editing was completed, the video was imported into a ready-made Pro Tools session before snapping to the appropriate time code for optimal tracking. After a quick visual to see if everything was playing correctly, any essential sound effects were then sourced online via sites such as YouTube, Soundly and Soundsnap, with each sound being sorted into their appropriate 'SFX' or 'Atmos' folder. Reverberation was then applied to each track via busses, with Pro Tools' 'Space' plugin granting the most realistic reverberation for each of the sound sources in comparison to what an enclosed environment may sound like. The Facebook 360 Spatialisation plugin was then placed onto each track, allowing 3rd order Ambisonic conversion to be applied for each sound source. The session was then mixed appropriately via the faders in the mix window, track automation (including volume and binaural panning automation) and most importantly, clip-based gain, with effects such as equalisation being applied to tracks that required touch-ups. Additionally, each Ambisonic track was then routed to a binaural decoder, allowing me to hear the elements correctly within the spatial environment. Finally, the mix was then bounced out, reimported into the session, and mastered for both BBC's R128 and YouTube's loudness requirements. This was done using Ozone Elements and their maximiser plugin. The Ambisonic mix was then converted to 1st order, before being placed back into Premiere Pro to apply the appropriate spatialisation for the YouTube 360 video.
Using 3rd-order Ambisonic had a significant impact on how I processed sounds for the project. Initially, I had planned to use DearVR for spatialisation, but I found that the Facebook 360 audio suite offered accurate and precise spatialisation for each sound source, despite having only 16 channels. With the help of Pro Tools and Facebook's 360 software, the only limitations that I faced were my own experience with the Ambisonic software and my creativity within the given time frame. With the mix also including third-order ambisonics, I can also replicate the experience on a variety of multi-channel speaker setups, allowing me to reproduce the soundscape in a variety of different ways depending on the exhibition location.
Journal 4 - Before Resubmission.
With the project now finished, I think now is the best time to reflect on past events resulting in the project's state of completion. Due to the issues I with my physical and mental health over the past few months, the project I had wished to submit for the 9th of December has sadly been deemed unsatisfactory within my eyes. Although the submission does still include a 360 video along with a binaural soundscape, the lack of binaural head tracking significantly reduces 'The Blindfold Experience's potential to create a moveable immersive experience for the user, reducing the clarity of certain sounds within the space, along with a reduction of accuracy in how sounds are positioned and played back. When comparing the initial planning documentation to today's product, it is evident that the project is uncompleted. Further work is required, including the implementation of head-tracking for any state of completion to be considered.
I cannot continuously put myself down for this, however. Although I may be unhappy with the project in its current state, the original proposed submission date is only formative. This allows me to continue working on the project over the next few months and act on any feedback I may gain from the formative submission. During this time, I am confident that I can implement any of the missing features, making 'The Blindfold Experience' the best product it can be. As well as this, I have successfully prepared 'The Blindfold Experience' for its next stage in the production process. With feedback being gained, a more heightened experience can be created, allowing me to alter any events that may need little tweaks or fixes to their sound design. With this being altered, tracks can then be easily converted to an ambisonic format, allowing me to integrate both the previously created 360 video and the integral element that is audio-based head-tracking. With the use of a fully featured Facebook 360, sounds will now be more appropriately placed within their space, featuring more accurate sound responses depending on head placement and better control over height, width, and reflection parameters.
Journal 5 - A Resubmission Update.
I'm pleased to announce that 'The Blindfold Experience' is now complete, and I have successfully implemented the changes I mentioned in my previous journal entry. By incorporating binaural head tracking via ambisonics and 360 video, the project has been elevated to a more immersive experience for the user, allowing for more accurate positioning and playback of sounds in the space. With the use of the Facebook 360 plugin, I have been able to fine-tune the sound response depending on head placement, height, width, and reflection parameters, resulting in a more realistic and dynamic soundscape. Reflecting on my journey, I acknowledge that my physical and mental health challenges have presented some personal disadvantages in completing the project within the original submission date. However, I have learned that it's essential to prioritize self-care and seek support when needed. As a personal advantage, I have gained experience in using a variety fo ambisonic plugins, and have been able to push myself creatively within the given time frame. Additionally, the feedback I received during the formative submission has allowed me to make the necessary improvements to create a more polished final product.
In relation to the resubmission date, the previous journals have already been updated to reflect these changes, with Journal 4 now including a more up-to-date process on both the technical and creative aspects of the projects. Overall, I'm proud of what I have achieved with 'The Blindfold Experience.' By taking the time to address the initial issues, I have been able to create an immersive audio-visual experience that delivers on its intended purpose. Hopefully, you enjoy it as much as I enjoyed creating it, and I cannot wait to develop similar projects in the future!
Captions + Audio Source List
Due to the limitations of Jimdo (the website builder used to create my website), extra information such as captions cannot be placed next to images. To combat this, I have added a 'captions' tab to each of my projects, which aims to give better clarity on the images used in comparison to the work created.
These images are labelled in a manner that corresponds to their placement. This should be read from left to right, with image 1, detailing information for the highest left-most image.
- Image 1 - TheBlindfoldExperiencePTTimeline.png - AVID Pro Tools timeline screenshot.
- Image 2 - TheBlindfoldExperienceAPPTimeline - Adobe Premiere Pro timeline screenshot.
- Image 3 - TheBlindfoldExperienceIntroductionScreenshot - A screenshot of one of the introduction scenes.
- Image 4 - TheBlindfoldExperiencePTMixWindow- A screenshot of the AVID Pro Tools mix window as well as the Facebook 360 Spatial plugin being used.
------------------------------------------------------------------------------------------------------------
Audio Source List:
- CloseBreathingFemale[Sound] Soundsnap. Available at: <https://www.soundsnap.com/efx_int_breathing_02_wav> [Accessed 8 December 2022].
- CloseBreathingFemale2[Sound] Soundsnap. Available at: <https://www.soundsnap.com/efx_int_breathing_14_wav> [Accessed 8 December 2022].
- CloseBreathingFemale3[Sound] Soundsnap. Available at: <https://www.soundsnap.com/efx_int_breathing_06_wav> [Accessed 8 December 2022].
- CloseBreathingFemale4[Sound] Soundsnap. Available at: <https://www.soundsnap.com/efx_int_breathing_07_wav> [Accessed 8 December 2022].
- DistantThunder[Sound] Soundsnap. Available at: <https://www.soundsnap.com/thunder_distant_rain_peal_release_to_roll> [Accessed 8 December 2022].
- ElectricTVHum [Sound] Soundsnap. Available at: <https://www.soundsnap.com/electric_tube_tv_humming> [Accessed 8 December 2022].
- GoreMovement [Sound] Soundsnap. Available at: <https://www.soundsnap.com/intestine_gore_movement_3_wav> [Accessed 22 April 2023].
- InsideRainAmbience[Sound] Soundsnap. Available at: <https://www.soundsnap.com/dakota_towns_park_rain_thunder_metal_roof_overhead_loopable_wav> [Accessed 8 December 2022].
- LongHorrorDrone [Sound] Soundsnap. Available at: <https://www.soundsnap.com/dark_horror_drone_pad_rising_evil_oscillating_low_end_distortion_eerie_thriller_3> [Accessed 8 December 2022].
- LowRumble [Sound] Soundsnap. Available at: <https://www.soundsnap.com/haunted_factory_ambience_rumbling_interior_perspective_2> [Accessed 8 December 2022].
- LooseWallClock[Sound] Soundsnap. Available at: <https://www.soundsnap.com/small_and_loose_clock_ticking_in_a_loop> [Accessed 8 December 2022].
- MetalCageRattling [Sound] Soundsnap. Available at: <https://www.soundsnap.com/contact_metal_cage_opening_and_closing_locks_horror_sounds_mono_wav> [Accessed 8 December 2022].
- OldWoodenDoorClosing [Sound] Soundsnap. Available at: <https://www.soundsnap.com/pm_hd_designed_21_doors_old_wooden_scary_horror_close_impact_wav> [Accessed 8 December 2022].
- Payne, V. Constantinople. [Song] Available at: <https://youtu.be/icM09kttaVo>[Accessed 8 December 2022].
- PlasticClockTicking[Sound] Soundsnap. Available at: <https://www.soundsnap.com/designed_clock_plastic_08_wav_0> [Accessed 8 December 2022].
- RadioTuning[Sound] Soundsnap. Available at: <https://www.soundsnap.com/radio_tuning_static_noise_collection_2_wav> [Accessed 8 December 2022].
- SkinPeelingGore [Sound] Soundsnap. Available at: <https://www.soundsnap.com/horror_accent_skin_peeling_2> [Accessed 8 December 2022].
- SkinPeelingGore2[Sound] Soundsnap. Available at: <https://www.soundsnap.com/efx_int_pulling_skin_04_c> [Accessed 8 December 2022].
- SlammingWindowShut[Sound] Soundsnap. Available at: <https://www.soundsnap.com/efx_int_glass_wood_window_01_b> [Accessed 8 December 2022].
- TVTurningOff[Sound] Soundsnap. Available at: <https://www.soundsnap.com/tv_switch_off> [Accessed 8 December 2022].
- WeirdClockChime[Sound] Soundsnap. Available at: <https://www.soundsnap.com/clockfancymantleclock6oclockcyclechimecloseint01_ms_wav> [Accessed 8 December 2022].
- WoodenDoorClosing[Sound] Soundsnap. Available at: <https://www.soundsnap.com/door_heavy_wooden_entrance_door_closing_3> [Accessed 8 December 2022].
- WoodenDoorOpening[Sound] Soundsnap. Available at: <https://www.soundsnap.com/door_heavy_wooden_entrance_door_opening_4> [Accessed 8 December 2022].
- WoodenFootstepsTrainers[Sound] Soundsnap. Available at: <https://www.soundsnap.com/footsteps_trainer_wooden_floor_living_room> [Accessed 8 December 2022].