Remote Pair Programming on Twitch with Zoom

[MISSING TRANSLATION] Ted M. Young

Not long ago, I had the opportunity for Amitai Schleier to remotely pair program with me on my Twitch Live Coding stream. Since I'm on a Mac, getting this set up to work wasn't trivial, but once I figured it out, it wasn't that complicated either, so I hope this helps others do more live coding with remote pair-programming.

Updated April 7, 2019 11:45: Added diagram of audio routing and update video diagram.

Note: I'm assuming you're on a Mac, at least 10.12 (Sierra), and using OBS Studio.

Ingredients

  • Computer: MacBook Pro (Mid-2015, with Sierra, 10.12.6)
    • Haven't tried this setup (yet) on High Sierra or Mojave (let me know if you try it!)
  • External Monitor: Having 2 monitors (laptop + external) helps, but not required
    • Luna Display is good for on-the-road, using iPad as 2nd monitor
  • Headphones: earbuds worked fine, but has to be something besides your laptop speakers
  • Microphone: External USB-based R0DE NT-1
    • Might be able to use the mic in your headphones/earbuds, but haven't tried it
  • Videoconferencing tool: Zoom
  • Audio routing tool: Soundflower

Overview

Here's what the broadcast looked like -- I'm in the lower-right, in my typical location, and Amitai is in the lower-left.

Here's a behind-the-scenes view of the above:

The Video

The main screen that's being broadcast is in the center. The green shaded portion (IntelliJ IDEA in this case) is what Zoom is sharing with my remote partner. In the lower-left is the picture-in-picture (PiP) of my remote partner's webcam, which is layered on top (in front) of the screen share by OBS (which my partner doesn't see).

Note that the PiP only appears this way when screen-sharing is on. Otherwise it appears in videoconferencing mode, with decoration around the video. It's actually nicer for Twitch-Pairing that it appears this way.

My webcam appears in the lower-right, with the chat window, and my stream logo above me. This makes up the rest of my pairing OBS scene for pairing.

The Audio

The audio is a bit more complicated as we have to "route" the audio (like water through pipes) so that I can hear my remote partner, he can hear me, and, of course, the audience can hear both of us. Here's an overview:

There are two audio sources that we need to handle: audio coming from my remote partner via Zoom, and my audio, coming from my R0DE microphone.

Zooming in on Audio

Let's look at Zoom's audio setup first.

As you can see, the Speaker, which is the audio coming from the remote side of Zoom doesn't go to headphones or a speaker, but goes to Soundflower (2ch), which is acting as the router or pipe for the sound to flow through. We'll see that the sound flows through Soundflower (get it?) and gets piped into OBS for it to broadcast to my viewers.

The (2ch) after the Soundflower name indicates that it's the stereo, or 2 Channel, pipe. There's also a 64 Channel (64ch) pipe, but you'd only use that for much more complex routing scenarios than I'm covering here, such as music creation and mixing.

The Microphone is the same one that OBS uses, so the audio from my microphone (the R0DE NT-USB) is going out through Zoom to my remote partner, as well as flowing into OBS to be broadcast.

Flowing Zoom Audio Into OBS

In order to route the audio from Zoom into OBS, we need to tell OBS about this new sound device, Soundflower.

Here I've set the Soundflower as a Desktop Audio Device. I could have set it up as any of the audio devices, but I picked that one (for no good reason, other than it was the first one). My Mic/Auxiliary Audio Device is my R0DE as always (remember, it's shared between OBS and Zoom).

Soundflower on the Scene

Now that Soundflower is available, we need to add it to the "pairing" scene so it can be mixed with my microphone's audio. We add it to the sources, so it shows up like this:

And once it's in there, we'll see it in the Mixer:

I Can't Hear You

The last piece of the puzzle was trying to make it so that I could hear what my remote partner was saying. Remember that the Zoom audio is routed through the "Soundflower 2ch" device. That means my viewers can hear it (because it's mixed in, as we see in the mixer tool), but I couldn't. The solution I found was to go into the Advanced Audio Properties from the gear menu in the Mixer:

Which brings up the following window:

Since I wanted to hear the Soundflower audio (coming from Zoom), I changed the Audio Monitoring from the default of Monitor Off to Monitor Only (mute output) and now I could hear my partner's voice.

There may be a better way to do this (or any of the above), but this worked for me.

Volume Mixing

One thing you'll want to keep an eye on -- well, really your ears -- is balancing the volume of your Zoom partner with your own. When I played back the pairing recording, I found that Amitai's voice was louder than I expected, since I was monitoring what was coming in from Zoom and not what was going out through OBS to Twitch. Next time I'll monitor it from my iPad or my other computer just to get the initial levels. I could monitor the mixed audio that goes out all the time (by changing the monitor settings we saw above), but I'm not used to hearing my voice in my headphones, since I don't monitor at all when I'm solo streaming.

Questions? Comments? Suggestions?

Did this work for you? Or not? Have suggestions for improvements? Let me know via email (ted@tedmyoung.com) or on Twitter.

Want to Pair With Me?

I'm looking for folks to come on and pair with me on future Live Coding streams. Just email me (ted@tedmyoung.com) or find me on Twitter. You don't have to be an expert Java developer, and I'm happy to practice a bit with you beforehand.

Even if you just want to join me by voice (instead of chatting), let me know.