ShadowPlay is an exploration of the architecture and light in outdoor spaces through sound. Using a bicycle-mounted smartphone as a brightness sensor and a bluetooth speaker, the mottled patterns of light and shadow become the musical score as one rides along city streets.
User Guide
Usage
- Place device in bicycle mount with active camera uncovered (either front or rear)
- Open ShadowPlay application
- Calibrate:
- a. Open ... -> Calibrate
- b. Press the Start button
- c. Roll bike over both light and dark areas
- d. Press Stop button when finished
- Choose audio scene in Scenes table
- Ride through areas of light and dark to trigger scene audio
When using the front camera facing upwards, try riding directly under trees, bridges, or overhanging objects to affect the brightness.
Settings
Keep Awake
Camera access for live brightness detection requires the app to be in active use. If the screen turns off, the camera and audio generation will also be paused, therefore, by default, the "Keep screen awake" setting is set to on.
Calibration Values
By default, the current brightness value and range min & max are shown via the horizontal range bar in the Calibrate view. To see the actual numeric values, turn on the "Show calibration values" setting.
Qlist Record Controls
For ShadowPlay audio scene development, a built in sensor data recorder is included which saves brightness update events into a Pure Data qlist text file. Turn on "Show qlist record controls" to show the record & play buttons on the main view.
The current qlist can also be played back after recording to a timestamped text file in the ShadowPlay Documents directory. You can access the saved qlists either via the iOS Files app or Finder/iTunes file sharing and replay them on your desktop computer when composing (see Making Your Own Scenes).
Record Video
A silent mp4 video file can be optionally recorded along with the qlist. This allows for syncing recorded brightness ipdate events to video on your desktop computer.
Making Your Own Scenes
A ShadowPlay audio scene is essentially a Pure Data patch and metadata files in a folder. The patch itself receives brightness update events, processes them, and outputs stereo audio.
You can develop your own scenes using the open source Pure Data computer music environment and upload them to the ShadowPlay Documents folder on your device, either via the Files app or Finder/iTunes file sharing. When the Scenes view is opened, any folders in Documents with the required scene files are automatically included in the list.
Additionally, a qlist recorder is included in ShadowPlay to allow for recording brightness events along a route for playback on your desktop computer during composition (see Qlist Record Controls).
Install Pure Data and download the composer pack with a qlist player and scene template:
Exhibitions
Sep 17 - Oct 17 2021 Seasons of Media Arts: Connected Future, Karlsruhe DE
Papers / Presentations
Nov 18 2022 "ShadowPlay: An exploration of light in outdoor spaces through sound", Piksel 2022 festival, Bergen Academy of Art and Design Bergen NO
Jun 01 2024 "SpeedPitch & ShadowPlay: Two open source mobile-device apps for bicycles" (PDF & video recording), Gulashprogrammiernacht 22, Karlsruhe DE