Video Instrumentalization is a work in progress project and software developed by Guy Fleisher, together with Carlos Campos, which explores producing musical compositions in real time by translating visual material into sound. This software enables new connections between mediums, and correlates the visual domain, and the sonic – the intangible and emotional – coerced into new multidisciplinary compositions.
The software is composed of two parts:
Video analysis (VideoSynth), and Audio comparisons (SoundSynth).
The software analyzes the input materials in a 12-hues color space, exposing the pixels hiding in the gradients of the animations, reminiscing experiments done in the 60s and 70s with electronic music and motion graphics, art cinema, and New Media Art.
The uniqueness of this project is the interconnectedness of sound and image.
I have found that often when the visual composition aesthetics are pleasing, so is the sound composition and vice versa, asserting the correlation between the mediums, as well as making this program a way to create live multimedia compositions, juxtapositioning data points from the visual content into the sound domain. By creating visual planes for exploration and manipulation, the unfolding and envelopment of the composition opens up, allowing for aesthetic decisions to make a lasting effect over sound, similar to playing traditional musical instruments, but in an extended way.
Utilizing this new multidisciplinary “lending” of ideas, I am taking compositional approaches from visual media and repurposing them freely in a new multidisciplinary meta-space, encompassing both the visual content attributes (such as hue, saturation, brightness, amongst others) and sound content attributes (such as frequency, amplitude, coarseness, and higher harmonics content, amongst others.)
The project aims to be an intuitive instrument in experimental audio-visual composition.
I plan to develop the project to enable more visual content-creators to have access to musical composition, using innate visual composition “rules”, as well as known gestalt principles.
Each audio-visual composition is unique, and created simultaneously, in real-time.
The video synth reverses the traditional roles between sound and image; changes made to the visual material generate, effect (and in turn) affect the music and sounds, creating new audio-visual connections in the synchronicity, as well as the synchresis (from Michel Chion) of the resulting image and sound.
In the development, I built 3 working prototypes (presented in a semi-formal presentation/performance as part of Worm Winter Festival at De Liceiras 18 Artist-led community, as well as in Dentro Gallery space – Porto 2021):
Exploring real-time footage material manipulation, focusing on the musicality of the compounded materials, creating abstract audio-visual compositions with rhythmical cues. This is meant as a performative tool.
Exploring real-time abstract color compositions, focusing on the musical output, using the video materials as a base for further exploration. This project is currently being developed further by Guy and Natalie into a full-length audio-visual album, as well as a performance of experimental music.
Exploring the nature of physical objects in real-time using a video camera input for analysis. This is unique in the way the project is performed, using physical objects that filter, reflect, and diffuse light sources, creating layers of colored light.
**Images from the live presentation as part of Worm Winter Festival at De Liceiras 18 Artist-led community, as well as in Dentro Gallery space – Porto, December 2021
Following the exhibition ‘Our Malawi; No Words Needed’, I zoomed out a bit.
While traveling through the country, I collected and curated environmental recordings of Malawi; from the quiet lake-side shores to the villages in the rural areas around Livingstonia, always carrying my trusted Zoom H-6 handy recorder. Having it with me at all times added to many experiences, as I was struggling to explain in my broken Chichewa what the device does, using hand gestures and headphones to explain my stance.
The first album in the series is the Unu House Charity album, performed by the children of the open-house, showcasing their musical afternoon activities, and dance. I explain the album further in my previous post, My Malawi Pt. 1.
Once we settled in N’Khata Bay at the Unu House charity headquarters located in the neighbouring village of Chikale, I took to heart local musician’s nightly hang outs, making music on the lakeshore. First, I met Lumbani ‘JahYouth’ Mkandewire, a guitarist and singer, who was teaching music at the house . Along with him I met , Martha ‘Mama Malawi’ Chinyanja. We jammed and talked long into the night, playing folk tunes and after a while I could convince them to play some of their own compositions. Two days later I met Joffrey Goma, one of JahYouth’s friends, a quiet and slander singer and percussionist. By the fifth day I knew how to play all their songs, and we started talking about the option of recording. I was keen to try and get a ‘stripped-down’ version of the band, mainly catching acoustic versions, no amplification. We set up in the main gallery space of the house, placing the zoom recorder and recorded takes, stopping to discuss the performance, trying different arrangements. For the most part, the first or second takes were chosen, having captured “happy mistakes”. I tried to emphasise the room acoustics, including the noise from outside, as the curios children from the neighbourhood peeked laughingly. All three of them, Jahyouth, Martha and Joffrey, had different vocal textures, ebb and flow. The session took two to three hours, by the end of which we all agreed we must meet again and jam different tunes. Since these are at most Jams, I took liberty with naming the songs, as we never stopped to even talk about the songs themselves, except play them.
Lumbe. Jah Family Band. Recorded at Unu House. This song was captured during the second recording session. Jah Youth proposed it the same day I got back to Unu House. Before we started playing I asked where and when was this song from, when Martha commented that it was a very old song they used to play with their band, but never got to record.
Traditional. Jah Family Band. Recorded at Unu House. This song was captured during the second recording session. I asked Jahyouth if he could play me a traditional folk song, a conversation which prolonged into a deep talk about the state of conservation in Malawi, each throwing around their own thesis about the way it is. Finally, Jahyouth started playing and humming this tune, saying it used to be a traditional song, but has now been disregarded, and all but forgotten. He admitted that he couldn’t remember it properly as well, making up missing parts.
Kalimera. Recorded at Unu House. This song was captured during the second recording session. This was a song Martha wanted to record. She played the song incessantly throughout the entire day, practicing the strumming technique and the chords repeatedly. I heard her play this song with the children in the house (on the first album), so I was familierized with the tune, and had heard many different versions of it. Even in this version she was hesitant on how to end it, and so this track was taken from a first take, as it was raw and interesting. Maybe some day I’ll release the rest of the takes of this song.
Theressa. Recorded at Unu House. This song was captured during the second recording session. This was Joffrey’s song that Jahyouth helped arrange. Joffrey was very excited for the recording, and the option to record his vocals. This was a first take, as he knew exactly how the song should play-out.
Medley. Recorded at Unu House. This was an unexpected surprise, when Joffrey mixed and mashed several of his songs into one. This extended track was so much fun to record, since we got to hear a lot of the repertoire of his songs, all stacked up onto another.
Think of me. Recorded in the back room of Unu House. Mr. Overton Gondwe is a phenomenal musician. As soon as I heard of him, I knew I wanted to jam with him. His songs are in English, since he plays for the most part, in hotels, for tourists. He plays accordion and sings. He has a clear cutting voice, and his accordion skills are very unique. When I asked him to play me some of his songs, he preferred to play religious songs, as he is a very christian person. He recorded the two tracks one after the other, without stopping. One memorable moment was when we listened back to the recording. Overton sat silently, with the headphones over his ears. He seemed concentrated immensely. He did not speak or flinch for the entire length of the track. After we finished playing back the songs, his only reaction was a short and almost inaudible “good”. He was the epitome of a humble musician, only striving to play his music for people, with an open heart, and a unique sound to his playing.
We shall meet again. Recorded in the back room of Unu House. A gospel and a love song performed by Overton.
Baby, Ndiwe wanga. Chifundo Chinoko. Recorded on the Lakeside, at Butterfly Space Hostel. Chifundo was working in the kitchen at a local hostel, when we met. He would play live shows twice a week, at the bar of the hostel, playing guitar and singing. We met through Jahyouth, and we jammed a bit. His English was very good, and he was brilliant at playing the guitar. Funnily enough, he was deeply influenced by American love songs, and pop music, and had written his own music in the same style. This song is a love song he wrote.
Love. Toxic Baby. Recorded near Livingstonia. At the Mushroom Farm, I had met with a local friend of Jahyouth, a rapper, who only identified himself as ‘Toxic Baby’. He wanted to sing and jam with me, but had not come with anything prepared. I told him to just sing me something, so we went in the night to the outskirts of the mountain, far from people, to record some acapella versions. This is a song he wrote, also as a love song.
Unu House anthem. Recorded at Unu House. This song was written by Jahyouth and the children at Uni House, together, as a musical activity. The kids loved to sing it, as it was for almost all of them, the first time they had written music, and played it back.
Sing me some song. Recorded at Unu House. This song is by Jahyouth, and he taught it to the children of the house. They all join in, singing with him for the chorus. Through this song the kids also learned English skills, as well as a way to voice themselves. It was truly incredible to jam with them the song, as they sang it with 30 (!!!) voices. Jahyouth is leading the song form with his guitar.
The third album was a happy coincidence altogether. While traveling to visit the Malawi-Wawi project, we sat in their headquarters, located in the rural community of Kapeska, not far from Chinteche, northern region of Malawi. Just when we were about to leave, Noam turned my attention to some singing, emanating from afar. It was barely audible, but had piqued my interest, as Julius Banda, the local manager of Malawi-Wawi project and I went over to check it out. It turned out to be the local choir’s rehearsal. Situated in an old semi-demolished church, the choir group, (which consisted of the choir master, three men, 4 women and 2 girls), had started their rehearsal for the Sunday mass. Their songs were mostly in Chitonga (local language in Malawi, endemic to the northern region). As soon as we enetered the building, the choir went silent. Julius asked the choir master to allow me to sit in, and record their music. After a short consultation, they all agreed smiling, and so I was able to catch an entire mass, consisting of four different church songs. While the choir was singing, they had orchestrated dance moves, which are salient and audible in the recording, and resemble a percussion sound. These were made with their feet and hands, using body percussion and motion. Once they had finished singing the songs, the choir master had given us the signal to quietly leave, and so we did. I incorporated the environmental recordings from around the Kapeska community, mainly the football field next to where the church was located.
The fourth, and last album of the series, was recorded during our second visit to the country. Between February – March 2020, Noam and I decided to take our exhibition ‘Our Malawi; No words needed’ and show it around the country (I expand on this here). During that month, we had the opportunity to return to the places we stayed in for prolonged periods the prior year. This was a heartfelt visitation, as we reunited with old friends, as well as with new ones. The album came about more concisely, because I knew I wanted to capture more songs of local musicians I had met the year before. The Unu House Chairty had since formed a band, for the kids to play in. I also made contact with new musicians, Namely Andrew ‘Bobo Andy’ Makana. He is a bobo rasta, from Mzuzu, northern region. I had made the connection with Andy through a mutual friend, Hal, AKA Selkie, from RoomTone Records in Bristol. You can (and should) check them out here and here. As soon as we met, we immediately clicked. Andy is a talented musician with a creative force like no other. I also directed, shot and edited a video clip for two of his songs, one of which you can check out here. Continuing the spirit of the first album, Andy and I jammed a few sessions, before we settled on a song we could improvise, with Andy singing lead vocals and me playing the guitar, singing backing vocals and. This album is ridden with strong emotional moments for me, reminding me that I have a new home in Malawi, with all the complexities it brings, it widens my heart in new directions I did not see coming. Leaving Malawi that time, it has taken a special place in my heart, and life. During our exhibitions we met with local artists, poets, musicians, activists. I have collaborated with a local poet, Chris Msosa, in creating sound poetry pieces, which culminated in an evening of poetry and environmental sound-art, which I will share in the future. I feel like this project is a personal recollection of the time spent, and could be a jumping board to a much larger body of work. In the future, I want to return to Malawi, with more equipment, and perhaps enlarge the project, reframe it in different ways, but for now, it represents the local artists and musicians through my eyes and ears. I am forever gratful to have met new friends, which are by now family, in the warm heart of Africa.
1. MOTO. Recorded at Unu House. I first heard this song (and recorded it) on our first visit to Unu House. This time, the kids wanted to perform it themselves, without anyone else helping them with the instruments. They gathered ,singing and dancing, while I stood there as spectator, allowing them to express themselves, led by the older children visiting the house. 2. Kids’ song. Recorded at Unu House. This was another one where the kids had asked to play themselves. They had written this song all alone, following their music lessons. They arranged it themselves as well. 3. Eliza. Song written and arranged by Martha ‘Mama Malawi’ Niyanja. Recorded at Unu House. 4. I initially met Martha when we first came to Malawi. She is truly the mother of the house. When we came back this time, Martha told me she had wrote this song, but hadn’t arranged it. This is a jam rendition of the song, extended alongside Mr. Overton Gondwe, playing the accordion. 5. Unu Charity Jazz Band. This song was written by Martha and the kids together, based loosely on a vocal obstinate Martha had picked up from a Zimbabwean singer she liked very much. The song capitalises the new essence of the house band, with the kids taking a bigger part in making music. Recorded at Unu House. 6. Unu charity jazz band vocals. Recorded at Unu House. This is the original vocal rendition of the inspiration that Martha told me about. As soon as I heard this, I told her I have to record her vocals Acapella, as they moved me so much. 7. Good Vibe. This song came out of a jam session with a new friend, Andrew ‘Bobo Andy’ Makana. Recorded at Andy’s house, Mzuzu. We decided to jam and write a tune together. As we jammed, I played guitar and Andy wrote the lyrics, while we arranged the song ‘on the fly’. 8. Think of me, forget me not. This song is by Mr. Overton Gondwe, and was recorded for the last album as well. Recorded at Unu House. This time, Overton insisted on recording it again, since he thought he had a better delivery of the song now. I joined with a guitar, and followed his instructions, as we jammed the song form. 9. Where are you. This is another of Overton’s songs. Recorded at Unu House. This song is very unique, in the incredibly intricate rhythmic patterns he plays on the accordion, while singing the lyrics. I played along with the guitar, as Overton marked the chord changes with a slight nod of his head.
starting off 2020 in a lockdown in South Africa’s quiet coastel surf town of Jeffrey’s Bay, has thrown my life into a twist. As most, A bunch of future projects and plans went down the drain, along with the pandemic limiting my movement…but not my creative juices!
After a short break back in Israel to work on some personal projects (to be revealed very soon) Noam and I took part in an interesting and quirky artist residency program in Porto. What started as a short lived idea, became a love at first sight, and we extended our stay.
Between September – November 2020 we spent time in oPorto, Portugal. Together with other international artists we co-lived and co-worked on 5 different exhibitions, meeting new people daily and absorbing the lifestyle of the Portuguese northern city of art.
For my part, I tried to emphasize the program’s multi-disciplinary, cooperative approach, combining my love for visual art and sound art into collaborations that bore fruit in the form of new works together with artists from different disciplines.
As 2020 came to an end, I finally took time to recap and bring a closure to a project I started with Noam, titled Our Malawi; No words needed.
This interactive photography exhibition started as a passion project, from an infatuation we’ve developed with the country.
Malawi, a small country in eastern Africa, has a special place in our hearts. While we traveled there in 2019, we had no ambition to make a large project from it. The scenery and the people have imprinted a new look on life in us. We saw beauty everywhere we looked. They don’t call it ‘The warm heart of Africa’ for no reason!
The people, mainly, have made a lasting impression on us.
When we got back to London and started collecting our materials, we realized we have an entire exhibition’s worth of photographs and field recordings, all of which of unique nature. They were from all around; portraits of incredible people we’ve met, landscapes and occasions – from rural areas, villages and city life. Malawi has it all. Also, the third largest lake in the continent!
Unu House; A charity based in Chikale, N’khata Bay region, which serves as a open-house model for the local kids, giving them after-school activities, cultivating excellence in sports, theater, arts and music.
Malawian musicians; A documentation of musician jam session around the country. Every musician I met, I had asked to play me folk songs, or original music, based on the area where they grew up in Malawi. This album is an effort in documenting these songs, making up an album to spread the music out to the world.
Malawi Wawi; This album came out of the Kapeska community, located near Chinteche, in the Northern region. The project is an NGO, working together with the local cummunity to help them build new schools and accomodate training facilities and English studies for older members of the community. The music is from the rehearsal of the local church choir, as well as environmental recordings from the Malawi-Wawi headquarter’s grounds.
The nice reception made us feel like we HAVE TO show it to our newly made family in Malawi, and so we started the preparation for showing in and around Malawi, contacting everyone we could think of. Lucky for us, we have made a bond with Zilanie Gondwe; a human rights activist, working with women empowerment projects in the capitol, Lilongwe. Zilanie treated us like family from the get-go, and she has truly taken a deep place in our hearts. without her, all of this could have never come to life, and for this we are indebted to her forever.
I have made a page on my website on the topic, with all the documentation of the project, from London to Malawi and back here.
Bringing the art back to show it in Malawi was a tremendous feat for us, and allowed us to expand our ideas of the art itself. The reception, the comments and people’s reactions were mainly incredibly good, a bit of an eye-opener on the local art scene and artist led spaces in the country. We’ve met some incredible artists and spaces, and were able to collaborate with them, as well as give talks, lectures and workshops on our own practice.
Each one Teach one, but in the real world. Put our practice where we see instant gratification, on the behalf of others. Exposing people to new art mediums, and get back a smile, a laugh, some nice words.
This project has left a dent in my heart, with a deep yearning to return as much as possible to re-meet with old friends and acquaintances, which have now turned into an extended family.
I hope we can continue this in the future, but first we need to recap on the latest developments in our art practice.
Referring to the idea of sports in school, relating to maintaining the human body through physical exercises.
Travelling through Malawi has taught me a valuable lesson, which is to experience life, or to live the experiences.
Our travels led us to Nkhata Bay, Malawi: one of the biggest tourism spots in northern Malawi. The serene atmosphere on the lakeside created a great bed for tourism in the area. When we reached our hosts in Chikale, we soon realized we were stepping into something bigger than what we had imagined.
Unu House Gallery is an open-house model which started a few years ago for visitors and travellers to live close to the local community and learn about Malawian village life, providing a peaceful environment to children and youth from the villages around. It invites people to come together to share knowledge, skills, experiences, stories and support. The children come to the house to play, draw, sing, dance, drum or to do whatever comes into their mind, in a free and open environment. Since many of the children are exposed to confusing and conflicting occurrences in their daily environments, it’s very important for them to have a child-friendly space where they can gather and just be kids.
The Project was founded by Antje and Phillip, a Swiss-German couple which came to Malawi in 2015 and started renting the house. In 2018 Anton, Annetta and Nicole, three young travellers from Germany, took over the house from Phillip and Antje.
There are many toys, musical instruments, books and painting materials in the house you can share with the kids. The main fields of interest revolves around education, games and group activities.
Many educational misconceptions I came with were confronted immediately upon our arrival; The physicality of the education seemed inescapable. The kids’ participation was mainly manifest in group dance and making fun of my moves. Whilst dancing was the first thing everyone played, it proved to be the key to their musical participation as well – if they could dance it, they would play and sing it, reminiscent of the African churches which they have yet to attend.
After our 2nd day, Noam and I started participating in activities, Noam making art activities with the children, and I tried to help with musical activities. Through playtime, the importance of language was long gone, and we were left to our own devices, improvising dance and song.
A local musician and educator, Lumbani “Jah Youth” M’kandawire, was the kids’ instructor to musical activities. We became fast friends, jamming into the night and having lengthy conversations on the past, the present and the possible future.
The kids were shy at first, but after we had jammed a little and danced a lot they became willing and took active participation in everything we did.
After a week of participating and helping with musical activities, the kids presented a theatre play they had written, with the help of the people of the house. I recorded the entire show, edited and mixed it quickly to play it back to them.
It was the first time some of the kids heard their own voice played back, and it was beautiful.
After Noam and I left Unu House I felt agitated and uneasy. I could not concentrate on relaxing and enjoying the lake, as I felt I had more to do in the house.
I took the Ilala boat ride from Usisya (northern region) back to N’khata Bay by myself, this time with the intention to record the children’s songs they had written for the show, with higher quality.
Once again, the physicality of the experience led me to make some decisions on recording techniques, informing my misled conception of music recording, allowing me to go back to the “old way of recording”. We did placements for each member of the band, we tried different microphone positions and we orchestrated the whole session to fit the children’s short attention span. Everything felt like going back to the 1950’s, concentrating on performance and room tone, not post production; everything had to happen then and there. The physicality of the process had become the product itself.
This collection of recorded songs presented here represents some of the best moments we had in this 3 day recording session.
Some of the songs are written and performed solely by the kids, some present Jah Youth on guitar and some extra vocals, while some are just captures of moments during the process.
Check out the full album on my bandcamp acount, it’s free to download, but please consider giving a donation, all proceedings will go to Unu House for musical equipment❤️
Stay updated, more posts coming soon!
My Malawi Pt. 2
My Malawi Pt. 3
Our Malawi; No Words Needed || interactive audio-visual Photography exhibition
In this blog I will update my travels from London, UK to Malawi, Africa, and to Cape-Town, South Africa, Israel and hopefully back.
As I handed in my final creative project for my Master’s at Goldsmith’s, University of London, I realized that this was just the beginning of the journey. Like many things in life, once you start something and dive deep, only then you realize that you’ve gone down Alice’s rabbit hole, the road seems to open up just enough to show you the tiniest of clues as to the fabric of the research, an exploration into life, and in my case, into perceptual habits, musical conceptualization and composition, and the process of communicating with sound. In order to dive deeper and peel away the coating of everyday life, I opened this blog, in an attempt to use the Tao (the way, in many respects) and record in the process. To keep evolving off the beaten path of localized sound and compositional practice.
Malawi is nicknamed “Nyasaland”, which translates literally to “Land of Lake”. We spent most of our time meeting local friends who could direct us to the landmarks we must see, with an emphasis on landmarks off the beaten track of overland travels through Africa. We landed in Lilongwe and spent two nights at a couchsurfing house hosted by Sisi, a young Zambian living in Malawi. We moved with an old friend, Jam Kawnda for two more nights during which he showed us around the city and took us to see a support rally for the upcoming elections. Then we journeyed from Lilongwe, the capital city, to Zomba, the old capital. Zomba is located in the southern region, and the main language is Chichewa, the most commonly used in Malawi today.
We were hosted by an old friend, and now sister, Olivia. She shared her house with us and we spent roughly 3 days with her and her newborn baby Sophie. Since my wife was volunteering in Malawi around 2011 we had to see all the places she’s been to, and so we found ourselves walking in the rural area around Zomba quite a lot. We hiked the Zomba Plateau, in search of some significant wildlife and crystals around the mountain. We found only crystals.
After that, we left for what would soon be our second home, Monkey Bay.
We stayed at the famous Mufasa Eco Lodge and enjoyed our first few days of peace and quiet, recording a lot of Cicadas at night, along with the various night crawlers around the lake. Monkey Bay is just one of those places where the rhythm of life takes you over, slowing down every small demeanor and accumulated FOMO. I could literally feel the layers of old habits, learned practice and ADD-infused concentration, peel away, slowly but surely. After meeting exceptionally grat humans at the lake, we have decided to start heading north, and to our next stop; Nkhata Bay.
We stayed at a Couchsurfing that turned out to be one of the highlights of our travels, the Uno House Gallery and open house. located in the upper part of the Nkhata Bay village, the open house is a place for children of the neighborhood to come for afternoon activities, learn sports, music, and dance, alongside theater and other programs. The house was jam packed with kids, from the early morning till late at night, bustling with activities, dancing, singing and lots of music! I had the pleasure to record the children in a drama play they wrote and played in, instructed by the house leaders, Anton, Aneta and Nicole, with their local instructors; Lumbani “Jah Youth” and Martha “Mama Malawi”. While Jah would teach the children music, singing and playing, alongside rhythm and composition, Martha would teach them various sports activities and theater drama.
I recorded the children presenting their theater play (including songs!) to their proud parents and immediately mixed it on the spot to share the files, for all to hear. The beauty of sitting and playing back the recording to the kids was one to remember, some were surprised by the sound of their own voice, not to mention, playing and singing! The day turned into night, as we sat, drank tea, and talked about the potential these kids could have, if only there would be government funding for such activities in the touristic village of Nkhata Bay.