Split/Merge AudioVisual 視聴覚の分離と融合

index@kenjikojima.com

Technical Note of "Split/Merge AudioVisual" using LiveCode programming 1, 2, 3




Please consider making a donation
to support Kenji Kojima's works


This is English 3 minutes video. 日本語版(Japanese 3 minutes video)

Software Art: Split/Merge AudioVisual
Artist: Kenji Kojima

Binaries in electronics is one of art materials in 21st century. Cave artists encountered earth as color pigments 40,000 years ago. They created cave paintings. 20th century artists encountered binaries. They were aware art expressions would be beyond the boundaries of audio and visual senses by manipulating zeros and ones. Some artists in the modern art history like a painter Wassily Kandinsky, a composer Alexander Scriabin and others tried to go beyond the the boundaries of audios and visuals. This project has been on the extension of their works. "Split/Merge AudioVisual" processes a bitwise XOR operation on data of a photograph, and makes two same length of data. This method is called one time pad which is used quantum cryptography. The two data are converted to a music and a color pixel files. The project player downloads music and color pixel files from the artist's server, and performs merging audio and visual files into an original photograph. You can appreciate the art works anywhere on the earth through the internet.




Download Full Screen Player "MergeAudioVisual"
MacOS X Only.
If you are a Windows user, please see the videos below.

This application has only player function, does not split audio and visual of an image.
The performance of merging audio and visual files are "Bee Photo Series" by Kenji Kojima.
When you like to stop the player, please command-key down. The application was created by LiveCode 5.5.5 (unlocked).
The latest MacOS plays small volume of MIDI sounds. Please make large sound volume before the application opens.

MergeAudioVisual_Mac06.zip (20140723)
The player application "MergeAudioVisual (20140723)" is under Creative Commons License BY-NC-SA 3.0
You may not use it for commercial purposes.


Please consider making a donation to support Kenji Kojima's works


If an alert asked “MergeAudioVisual.app can’t be opened because it is from an unidentified developer". Please right click or mouse-click with control-key on the app icon, then select "Open" from the contextual menu.



New alert window appears.
Click "Open".

If your MacOS cannot play MIDI files, download QuickTime 7.


Left: Opening the player.
Right: Select split audio and visual files.


Left: Set visual on full screen.
Right: Processing audio and visual. It takes 3 steps, collecting musical notes, converting notes to binary, and merging binaries of audio and visual.





Merge AudioVisual Performance Videos:
The Player "Merge AudilVisual" performs these files. The original art is not a video.


Bee_1380418994
Watch Video (original work is not a video)


Bee_1380587313
Watch Video (original work is not a video)


Bee_1381080098
Watch Video (original work is not a video)


Bee_1381333259
Watch Video (original work is not a video)



Bee_1381521759
Watch Video (original work is not a video)


Bee_1381523174
Watch Video (original work is not a video)



Bee_1382369078
Watch Video (original work is not a video)



Bee_1382730493
Watch Video (original work is not a video)



Bee_1384707307
Watch Video (original work is not a video)





Split:
The software splits a digital image to audio and visual files by a bitwise XOR operation (logical exclusive OR). The bitwise operation takes two bit patterns of equal length and performs exclusive OR on each pair of corresponding bits. The software makes random numbers. It is same numbers of pixels of the original image. A binary of the random number performs on a binary of the image by bitwise XOR. Bitwise XOR makes 0 and 0 to 0, 0 and 1 to 1, 1 and 1 to 0. But the position of the decimal does not change. If a binary of one pixel element is 10010011 and random number is 00100001, the result will be 10110010. See the example of 8 bits binary. It is same length of a color data element.

10010011 (original image)
00100001 (random number)


10110010 (the result)




PLAY MIDI File | STOP

Audio and Visual:
The software converted the random number data to musical notes by Kenji Kojima's RGB Music technology. RGB value 120 is converted to middle C. Each two step values increase or decrease a half tone. One pixel make three notes of harmony. 0,0,0 is no sounds. The note length is determined by the color value. The music is a twelve-tone music. The left sample MIDI file was created from the musical notes of the random numbers. You can listen it. The result of the image would be a new image which processed by bitwise XOR. The perception of human being could not recognize what the image was. We have 2 digital files as audio and visual. You can think that the audio is a key and the visual is a cipher image.




Merge:
The software converts an audio key (music file) to musical notes then makes binary. The binary performs on the binary of cipher image (visual) by bitwise XOR. See the example.

10110010 (split image)
00100001 (split audio key)


10010011 (same binary of the original)

The audio and visual files merge into an original image.




Project File Creation Software: Splitting Audio and Visual Files (left), Merging Files (right)
This application is not distributed.



Technical Note of "Split/Merge AudioVisual" using LiveCode programming 1, 2, 3



Video, midi and photograph files are under Creative Commons License BY-NC-SA 3.0
Key words: Audio, Visual, Perception, Cognition, Technology, RGB Music, Cipher, Bitwise XOR, Transformation
Bee in Central Park Slideshow The Bee Series Photographs are used for this Project