Split/Merge AudioVisual 視聴覚の分離と融合

index@kenjikojima.com

Binary is One of Art Materials.
When we treat different sences of computer files in binary, there is no boundaries between them.
Technical Note of "Split/Merge AudioVisual" using LiveCode programming 1, 2, 3




Please consider making a donation
to support Kenji Kojima's works


Software Art: Split/Merge AudioVisual
Artist: Kenji Kojima

The project "Split/Merge AudioVisual" split a photograph to a random color pixels file and a music file, and merged them to the original photograph. It was an experimentation that different senses "Audio and Visual" erased their boundaries. They might be fundamentally a same perception. It was also a transformation, and an encryption and a decryption of visual data by an audio key. Actually the programming method was one time pad that was a type of encryption scheme. One time pad was used quantum cryptography which was impossible to crack. The project handled a perception of visible(color) and invisible(music), and audible(music) and inaudible(color). It was the related things what was form that was emptiness, or what was emptiness that was form.




Download Full Screen Player "MergeAudioVisual"
MacOS X only.

I am planing Windows version for the next project "Two Music Merge into an Image". But not sure so far. Mac and Widows have different color values of gamma then probably I have to make different documents for Mac & Windows. I hope I could make same music documents.

This application has only player function, does not split audio and visual of an image.
The performance of merging audio and visual files are "Bee Series" by Kenji Kojima.
When you like to stop it, please command-key down. If you are a Windows user, please see the videos below.

MergeAudioVisual_Mac05.zip

Please consider making a donation to support Kenji Kojima's works


If an alert asked “MergeAudioVisual.app can’t be opened because it is from an unidentified developer". Please right click or mouse-click with control-key on the app icon, then select "Open" from the contextual menu.

If your MacOS cannot play MIDI files, download QuickTime 7.


Left: Opening the player.
Right: Select split audio and visual files.


Left: Set visual on full screen.
Right: Processing audio and visual. It takes 3 steps, collecting musical notes, converting notes to binary, and merging binaries of audio and visual.





Merge AudioVisual Performance Videos:
The Player "Merge AudilVisual" performs these files. The original art is not a video.


Bee_1380418994
Watch Video (original work is not a video)


Bee_1380587313
Watch Video (original work is not a video)


Bee_1381080098
Watch Video (original work is not a video)


Bee_1381333259
Watch Video (original work is not a video)



Bee_1381521759
Watch Video (original work is not a video)


Bee_1381523174
Watch Video (original work is not a video)



Bee_1382369078
Watch Video (original work is not a video)



Bee_1382730493
Watch Video (original work is not a video)



Bee_1384707307
Watch Video (original work is not a video)





Split:
The software splits a digital image to audio and visual files by a bitwise XOR operation (logical exclusive OR). The bitwise operation takes two bit patterns of equal length and performs exclusive OR on each pair of corresponding bits. The software makes random numbers. It is same numbers of pixels of the original image. A binary of the random number performs on a binary of the image by bitwise XOR. Bitwise XOR makes 0 and 0 to 0, 0 and 1 to 1, 1 and 1 to 0. But the position of the decimal does not change. If a binary of one pixel element is 10010011 and random number is 00100001, the result will be 10110010. See the example of 8 bits binary. It is same length of a color data element.

10010011 (original image)
00100001 (random number)


10110010 (the result)




PLAY MIDI File | STOP

Audio and Visual:
The software converted the random number data to musical notes by Kenji Kojima's RGB Music technology. RGB value 120 is converted to middle C. Each two step values increase or decrease a half tone. One pixel make three notes of harmony. 0,0,0 is no sounds. The note length is determined by the color value. The music is a twelve-tone music. The left sample MIDI file was created from the musical notes of the random numbers. You can listen it. The result of the image would be a new image which processed by bitwise XOR. The perception of human being could not recognize what the image was. We have 2 digital files as audio and visual. You can think that the audio is a key and the visual is a cipher image.




Merge:
The software converts an audio key (music file) to musical notes then makes binary. The binary performs on the binary of cipher image (visual) by bitwise XOR. See the example.

10110010 (split image)
00100001 (split audio key)


10010011 (same binary of the original)

The audio and visual files merge into an original image.




Project File Creation Software: Splitting Audio and Visual Files (left), Merging Files (right)
This application is not distributed.



Video, midi and photograph files are under Creative Commons License BY-NC-SA 3.0
Key words: Audio, Visual, Perception, Cognition, Technology, RGB Music, Cipher, Bitwise XOR, Transformation
Bee in Central Park Slideshow The Bee Series Photographs are used for this Project