PdParty User Guide

Version: 1.0.3
Date: 2017-02-14

PdParty is an iOS app that allows you to run Pure Data patches on Apple mobile devices using libpd. It is directly inspired by Chris McCormick's PdDroidParty and the original RjDj app by Reality Jockey. It takes a step further by supporting OSC and MIDI and by implementing the native Pd gui objects for a WYSIWYG patch -> mobile device experience:

This patch in Pure Data...

becomes this on iPhone...

and this on iPad.

Acknowledgments

Table of Contents

Background

Dan Wilcox 2016

robotcowboy is the author's ongoing human-computer wearable performance project. Focusing on the embodiment of computational sound, robotcowboy was originally built in 2006-2007 as an MS thesis project using an industrial wearable computer[^wearable] running GNU/Linux & Pure Data, external stereo USB sound & MIDI interfaces, and various input devices including HID gamepads.

The original robotcowboy system hardware was gigged often, went on a 2 month tour of the United States in 2008, and lasted until the 2011 Pd Convention in Weimar. Around this time, Apple released the iPad 2 which featured a dual core processor and, most importantly, supported USB audio & MIDI interfaces. Seeking an option for new system hardware, the author began on and off development of an iOS application that could perform all of the tasks required for a live robotcowboy performance: run patches, full duplex stereo audio, MIDI, HID game controller support, & Open Sound Control communication.

With PdParty, the author now has a stable low latency mobile/wearable platform with a touchscreen, accelerometer, WiFi networking, and USB MIDI/audio. Here is a belt-based wearable setup using an iPhone, Camera Connection Kit, powered USB hub, Roland Edirol UA-25 USB audio interface, and a Behringer direct box (the latter two are built in the case on the left):

App Layout

TL;DR

There is a root settings screen and a patch/scene browser. Go to your patch and run it. Go back to update settings. Patches/scenes have on screen controls for the input volume, audio processing state (on/off), & recording.

Start Screen


Start screen on iPhone


Start screen on iPad, activate by pressing the "Patches" nav button.

This is the root of the app and is inspired by TouchOSC. Here you can launch the Patch Browser or change app settings.

This is also where you can enable the WebDAV server to access the app Documents folder for adding/updating patches.

Patch Browser

This is a simple "drill-down" file browser in the app Documents folder. Stuff in here is sandboxed between app versions. Simply navigate to patches or scene folders to run them. A "Now Playing" nav button will take you back to the Scene View for the currently playing patch/scene.

It only displays folders and supported file types. File type icons represent the supported files & scene folder types & certain scene types include a thumbnail image & author subtitle.

You can delete items by swiping right to show the delete button. Moving, copying, mass deletion, & renaming files/folders is available via the Edit button.

The default layout is:

Feel free to delete samples and tests. The libs folder contains abstractions needed by PdParty. This folder is exposed to allow you to update/upgrade the global abstractions as well as satisfy the user upgradeability requirement for GPL licensed abstractions. If the libs folder is not found, PdParty falls back to its internal backup copy.

The all folders within the libs folder are automatically added to the PdParty search path so this can also be a location for centralized abstraction libraries.

Note: These default folders can be restored on the Settings screen. So if you accidentally remove everything, you're not out of luck!

Scene View

Running a patch/scene launches the Scene View:

Gui elements work similarly to those in the Pure Data gui, except now you have multitouch instead of the 10 foot pole called a mouse pointer. Also, Numberboxes can be incrementally scrolled using two fingers. Empty space is used for touch events if the patch/scene type supports them.

The desired aspect ratio is inferred from the patch canvas size and the Scene View is automatically rotated. Also, the device orientation is locked depending on if the Scene is in portrait or landscape. The exceptions to this are RjDj scenes which are portrait on iPhone & PdDroidParty scenes which are always landscape:

On iPad, however, RjDJ scenes can be both portrait or landscape:

On Screen Controls

Simple on screen controls inspired by the RjDj app are provided to change the audio input level, enable/disable audio dsp, and start/stop recording.

Patches, DroidParty, & PdParty scenes display these controls in a popover activated by the "Controls" nav bar button:

RjDj scenes always have a square background with the controls located below.

Optionally, a button to launch a Console view to display Pd prints for the current patch/scene can be added to the Controls popover. Enable to Console view in the PdParty Settings.

Recording Scene

There is also a special scene for playing back recordings (aka .wav files). The slider now controls the playback volume and there is also a button for looping the playback.

The cassette background is a placeholder for now. When metadata is added, it will be replaced by the scene icon, etc.

OSC Server Settings

Enable the OSC server here and update its settings: ports, host (destination address), etc. The network IP address of the device itself is shown so you know where to send OSC messages to from another device. The server supports both IPv4 and IPv6.

Send via multicast by setting a multicast group in the Host field such as "224.0.0.0/24."

MIDI Settings

Enable CoreMIDI here and optionally enable Network MIDI with a macOS machine.

Inputs & Outputs are refreshed when a MIDI device is plugged in/out. Currently, PdParty automatically connects to all detected MIDI devices.

App Settings

App Behavior

These settings are mainly for live performance reasons where you don't want the device to go to sleep when you're trying to wow your audience. Obviously, keeping the screen on and audio running will drain your battery but in my tests so far it will still last for quite a while.

OSC Event Forwarding

These are useful options for patch creation & debugging. Basically, you can send accelerometer, touch, key*, and game controller events from the device to your computer while you work on your patch/scene in Pure Data. You can also receive live Pd prints from a running patch/scene in PdParty to make sure everything is working as expected.

The OSC server needs to be enabled and a patch/scene must be running in order for events to be streamed. The easiest option is to run an existing patch or upload an empty one you can run while sending events. Event sending will not work unless a patch or scene is being run.

Note: Most sensor events require the services to be enabled within the patch/scene before any events will be sent. Also, certain scene types do not support all events, please see "Scenes" in "Patching for PdParty" for more info.

* requires a usb/bluetooth keyboard

Audio Latency

This is pretty straight-forward and awesome thanks to CoreAudio. A lower audio latency will give you more responsive audio (nearer to "real time") at the cost of more CPU usage and lower battery life. Higher latencies are good for more simultaneous processing or where realtime response is not as critical (lots of delay, recording only, etc).

By default, the "Choose automatic latency" switch is enabled which tells the app to choose the best latency for your device. If this setting is disabled, you can set your desired buffer size using the radio buttons and you'll see the approximate latency.

Your mileage may vary depending on the device and the complexity of the patch or scene you're running. If you get audio clicks/dropouts, then you need to either increase the buffer size or decrease the processing complexity of your patch.

Copy Default Folders

These buttons allow you to copy the default libs, samples, and test folders into the app Documents folder, just as they were when you first ran PdParty. This is mainly there if you wish to re-copy or update any of the default folders.

Patching for PdParty

TL;DR

PdParty only shows gui objects (numbox, slider, etc) with send or receive names. It does not render your entire patch, so you need to create send and receives through the guis you want to appear on the device when you run your patch/scene.

Download the PdParty Composer Pack which contains the abstractions you need when composing on your computer as well as scene type templates.

Naturally, you can also download the PdParty source and open the test patches & examples to see how this is done: https://github.com/danomatika/PdParty/tree/master/res/patches

Detailed Instructions

Largely borrowed from PdDroidParty

  1. Create a new Pd patch that will contain your GUI objects like sliders, toggles, numberboxes etc. Place your main patch logic inside a subpatch and use the [soundinput] & [soundoutput] rjlib objects in place of [adc~] and [dac] (soundoutput is required for the recording controls).

  2. PdParty will scale GUI objects to fit the screen of the device. Your patch should have the rough dimensions of a phone/tablet in portrait or landscape mode (e.g. 3:2 aspect ratio or e.g. 480x320 should usually work well). If it is not exact it, doesn't matter - the GUI elements will be scaled.

  3. Numberbox, Sliders, Radios, Toggle, Comment, Bang, Canvas, and VU are currently rendered by PdParty and are feature complete (yes, all the edit commands work!). Also, the PdDroidParty GUI abstractions are supported.

  4. All GUI elements should communicate with the main audio patches using send and receive only. You can usually set send and receive for each GUI by right clicking on the object and choosing 'properties' in Pd. Do not directly connect cables to the GUI elements as they won't work. It helps to keep the GUIs on their own in the main patch and have it include the logic of your patch as an abstraction or subpatch containing senders and receivers for interfacing with GUI elements. This is good patch design practice anyway as it is basically a model-view-controller methodology.

  5. Copy the patch and/or its containing directory and any needed abstractions to your iOS device using iTunes File Sharing, via WebDAV over your local network, or through "Open in..." from other apps:

    • iTunes File Sharing

      Plug in your iOS device and open iTunes. Select the device, choose the App tab, and choose PdParty in the File Sharing section. You should then see the contents of the PdParty Documents dir. You can drag and drop items onto this panel and/or use the "Add..." and "Save to..." buttons*.

      Note: You can only see the top most level in the Documents folder and cannot enter subfolders. Sorry, that is simply how the iTunes file sharing system currently works.

    • WebDAV

      1. Enable the WebDAV server on the PdParty start screen on the device and connect to it using a file transfer program or the built in WebDAV support in some operating systems using the address below the WebDAV controls on the Start Screen. If you're using macOS or Linux on a local network, the *.local address should work, otherwise use the ip address (#.#.#.#). The server supports both IPv4 and IPv6.

        • macOS: Finder can mount WebDAV server folders: Go->Connect to Server... CMD+K. Login as a Guest:

        • Linux: both Nautilus (Gnome) & Konqueror (KDE) support mounting WebDAV folders, also FileZilla and other file transfer applications should work

        • Windows: newer versions of Windows Explorer support connecting to WebDAV folders, type the full address (including http://) into the address bar, also Cyberduck and FileZilla work nicely
      2. When the transfer is complete, navigate to the patch folder and run the patch. Don't forget to turn off the WebDAV server when you're done.
    • Open in...

      PdParty registers Pd patch .pd files and Zip archives (.zip, .rjz, .pdz) with iOS as supported types. This allows for each of these file types to be opened in PdParty from another application ie. Mail, DropBox, etc. When choosing "Open in..." via the Share button, the file(s) will be copied into the main PdParty Documents folder. Zip archives can then be unpacked by clicking on them in the Patch Browser.

Scenes

PdParty also supports running "scenes" which are basically folders with a specific layout that are treated as a single entity for encapsulation and have certain event attributes:

Running a regular .pd patch (a Patch scene) is the same as running a PdParty scene.

Pure Data Compatibility

PdParty is built using libpd and can be compared to Pd-vanilla with the following externals:

It is highly recommended that you use a vanilla-based abstraction library like rjlib for expanded functionality.

When patching for PdParty (as with RjDj & PdDroidParty), it is recommended that you work with Pure Data vanilla versions 0.46+. If you are working with Pd-extended, disable all externals in order to help lessen the chance you inadvertently use an object that will not create in PdParty. I actually have separate copies of my Pd settings file, one for desktop development and another for pd-vanilla/libpd work.

expr

[expr], [expr~], & [fexpr~] are included with PdParty. As of Pd versions 0.47+, they are under the BSD license which is compatible with the Apple App Store licensing requirements. This was not the case with earlier versions of Pd/libpd.

Key events

[key] works with an external bluetooth or usb keyboard. [keyup] & [keyname] are not supported as there is currently no official way to intercept raw key events on iOS.

VU Meter

[vu] gui objects do not have a sending name in PdParty, so make sure you're patch doesn't rely on passing values through any meters.

MIDI

All of the midi objects ([notein], [ctlout], etc) work. Obviously you'll need to have a usb MIDI interface (through a USB hub connected to the Apple Camera Connection Kit) or using Network MIDI and macOS.


Midi test Pdparty scene

PdDroidParty Compatibility

PdParty currently supports:

* doesn't do much as audio has to be off for vibrate to happen on iOS

SVG widget styling support is planned, but not an immediate priority as there is no built-in svg handling on iOS and there doesn't seem to be a standout 3rd party SVG library.

[droidnetclient] & [droidnetreceive] are deprecated in PdDroidParty and therefore not supported. Use [netsend] & [netreceive] instead.

RjDj Compatibility

PdParty currently supports:

Currently, scene paging and metadata are not supported.

Testing has been done using the original RjDj composer pack as well as various RjDj scenes including:

Also, thanks to Joe White for providing a copy of the RjDj _getsensors.pd patch by Roman Haefeli, et al. which provided an overview of the extended rj sensor objects.

Events

PdParty returns the following events:

_Note: RjDj scenes receive #touch, #accelerate, & #gyro events by default, DroidParty scenes do not receive any events, PdParty & Patch scenes receive all events. This is mainly for explicit compatibility. Extended RjDj sensor access is made via the [rj_loc] & [rjcompass] abstractions.

Accelerate, Gyro, & Magnet Control


PdParty accelerometer events

Reading accelerometer, gyroscope, and/or magnetometer events will affect battery life, so these must be manually started after the scene is loaded by sending messages to the internal #pdparty receiver:

Note: #touch & #accelerate events are automatically started for RjDj scenes for backward compatibility.

Loc (GPS) Control


Loc test PdParty scene

Loc events are essentially GPS location events, dependent on your device's sensors for accuracy (WiFi only, cell tower + GPS chip, etc). Additionally, speed & altitude events are generated when the location events are enabled.

Since running the GPS location service will affect battery life in most cases, it must be manually started and configured after the scene is loaded by sending messages to the internal #pdparty receiver:

It usually takes a few seconds to fix your position after enabling the location services.

_Note: Loc events are available in PdParty & Patch scene types by default, while the presence of an [rjloc] object enables them in RjDj scenes. These events work best on devices with multiple location sensors (iPhone) and may not work on some devices at all (iPad).

Compass Control


Compass test PdParty scene

A compass event is simply the orientation toward magnetic north with the top of the current UI orientation being at 0 degrees.

Like location events, the tracking the compass requires extra resources so it must be manually started by the scene after it is loaded by sending messages to the internal #pdparty receiver:

_Note: Compass events are only available in PdParty & Patch scene types by default, while the presence of an [rjcompass] object enables them in RjDj scenes. Events work best on devices with a digital compass (phones) and may not work on some devices at all.

Timestamps

Timestamps are sent to the [r #time] receiver with the following argument format:

This is the same format that the RjDj [rj_time] object returns.

Timestamp events must be triggered manually by sending a message to the internal #pdparty receiver in your patches:

Recording

You can manually trigger recording via sending messages to the internal #pdparty receiver in your patches:

Note: Recording will only work if you are using the rjlib [soundoutput] patch instead of [dac~].

Opening a URL

You can launch a web view with a given url via sending a message to #pdparty:

url can be either:

title is an open ended list of arguments that will be appended together and used as the navigation bar title, "URL" is used by default when there are no title arguments

Game Controllers


Controller PdParty scene

Compatible iOS MiFi game controllers can be read in PdParty if your device supports them. If the controller uses Bluetooth, enable Bluetooth in your iOS settings and make sure the controller is paired to your device. Currently, iOS limits the number of simultaneous controllers to 4.

Controller events can be read via the [r #controller] receiver with the following format:

There is no direct control over enabling/disabling game controller support. This is handled by scene type detection as RjDj & DroidParty scenes do not use controller events.

Note: Game controller button & axis names are based on the SDL 2.0 naming where "dpup" refers to digital pad up, "dpleft refers to digital pad left etc. This format is compatible with the OSC messages sent by the joyosc desktop HID device to OSC event daemon.

OSC

PdParty sends and receives OSC (Open Sound Control) messages internally between the PureData instance and the OSC server:

All of the PdParty events can be streamed over OSC, included Pd prints. The receive addresses are as follows:

Note: The argument number and types are equivalent with their receive counterparts, i.e. /pdparty/touch receives the same data as [r #touch].

See docs/composerpack/osc/osc-event-receiver.pd in the PdParty source repository for an event receiver you can use while patching & debugging on your computer:


osc-event-receiver.pd test patch

Also, try the tests/osc-test.pd test patch on your computer with the tests/pdparty/Osc scene on the device for a simple example on two-way communication:


OSC test PdParty scene

Note: PdParty utilizes the OSC library provided by Pure Data vanilla versions 0.46+, mainly the [oscparse] & [oscformat] objects.

Libs Folder

PdParty automatically adds any subfolders inside the libs folder in the PdParty Documents root folder to the current search path. Put any abstractions here if you want them to be globally accessible.

Also, this is where the default PdParty libraries are copied when the app is first run. This allows you to update the newer versions of the core lib patches. Be careful, as the rj patches, for instance, are required for running RjDj scenes etc. If you screw up / delete something, you can copy back the original files in the Settings screen.

TODOs